Skip to content

ona-samples/shift-eks

Repository files navigation

shift-eks

Multi-cluster Kubernetes development environment with OpenShift-compatible (OKD), EKS Anywhere, and EKS LocalStack.

Overview

This DevContainer provides three Kubernetes cluster options for development and testing:

  1. OKD (OpenShift-compatible) - Community distribution of OpenShift with 100% API compatibility
  2. EKS Anywhere - AWS-curated Kubernetes distribution with Docker provider
  3. EKS LocalStack - AWS EKS via LocalStack Pro for local AWS development

OKD and EKS Anywhere start automatically when the DevContainer launches. EKS LocalStack can be started manually via Ona Automations.

Note: OKD provides full OpenShift API compatibility without requiring nested virtualization (KVM). It runs on kind (Kubernetes in Docker) with OpenShift components installed.

Note: EKS Anywhere is AWS's official on-premises Kubernetes distribution. We use the Docker provider for local development, which is free and provides the same EKS Anywhere experience.

📚 Documentation:

Prerequisites

Docker Engine

This environment requires Docker to run both clusters:

  • ✅ Works in standard containers (Gitpod, DevContainers)
  • ✅ Works on local machines with Docker installed
  • ✅ No nested virtualization (KVM) required

OKD vs OpenShift CRC

OKD (Used in this environment):

  • ✅ 100% OpenShift API compatible
  • ✅ Runs without nested virtualization
  • ✅ Uses kind (Kubernetes in Docker)
  • ✅ Includes OpenShift operators and CRDs
  • ✅ Works in containers and cloud environments
  • ⚠️ Community distribution (not commercially supported)

OpenShift CRC (Alternative):

  • Requires nested virtualization (/dev/kvm)
  • Full commercial OpenShift 4.x
  • Only works on local machines or cloud VMs with KVM
  • See .devcontainer/start-openshift-crc.sh for CRC setup (requires KVM)

Quick Start

The clusters are automatically configured when you open this project in a DevContainer. After the container starts, you'll have:

  • ✅ OKD cluster (OpenShift-compatible) running on kind
  • ✅ EKS Anywhere cluster running (Docker provider)
  • ✅ kubectl, oc, kind, aws, and eksctl anywhere CLI tools installed
  • ✅ Kubeconfig automatically configured

Note:

  • First-time OKD setup takes 3-5 minutes
  • EKS Anywhere cluster creation takes 10-15 minutes on first start
  • EKS LocalStack requires manual start and LOCALSTACK_AUTH_TOKEN environment variable

Ona Automations

This project includes Ona Automations for managing the Kubernetes clusters as services:

Services:

  • eks-anywhere - Starts automatically on environment start
  • openshift-okd - Starts automatically on environment start
  • eks-localstack - Manual start (requires LOCALSTACK_AUTH_TOKEN)

Managing Services:

# List all services
gitpod automations service list

# Start a service manually
gitpod automations service start eks-localstack

# Stop a service
gitpod automations service stop eks-localstack

# View service logs
gitpod automations service logs eks-anywhere

See .gitpod/automations.yaml for service definitions.

Helper Scripts

  • cluster-manager.sh - Manage both clusters (start, stop, restart, switch)
  • verify-setup.sh - Verify all tools are installed correctly
  • .devcontainer/start-eks-anywhere.sh - Start EKS Anywhere cluster
  • .devcontainer/start-openshift-okd.sh - Start OKD cluster
  • .devcontainer/start-eks-localstack.sh - Start LocalStack EKS cluster

Cluster Management

Using the Cluster Manager

A helper script is provided for common operations:

# Check cluster status
./cluster-manager.sh status

# Switch between clusters
./cluster-manager.sh switch-os      # Switch to OpenShift
./cluster-manager.sh switch-eks     # Switch to EKS

# Restart clusters
./cluster-manager.sh restart-os     # Restart OpenShift
./cluster-manager.sh restart-eks    # Restart EKS/LocalStack

# Stop/Start all clusters
./cluster-manager.sh stop-all
./cluster-manager.sh start-all

# View LocalStack logs
./cluster-manager.sh logs-eks

Manual Commands

OKD Cluster (OpenShift-compatible)

# Check cluster status
kubectl get nodes
kind get clusters

# Use OpenShift CLI
oc get nodes
oc get pods --all-namespaces

# Check OpenShift-compatible features
kubectl get crds | grep openshift
kubectl get crds | grep operator

# View Operator Lifecycle Manager
kubectl get pods -n olm
kubectl get catalogsources -n olm

# Use kubectl (automatically configured)
kubectl get nodes
kubectl get pods --all-namespaces

EKS Anywhere Cluster

# Set kubeconfig
export KUBECONFIG=~/clusters/eks-local/eks-local-eks-a-cluster.kubeconfig

# Use kubectl
kubectl get nodes
kubectl get pods --all-namespaces

# View cluster info
eksctl anywhere get clusters
kubectl cluster-info

# Upgrade cluster
eksctl anywhere upgrade cluster -f ~/clusters/eks-local-config.yaml

Available Tools

  • kubectl - Kubernetes CLI
  • oc - OpenShift CLI (works with OKD)
  • kind - Kubernetes in Docker
  • eksctl - EKS Anywhere CLI (eksctl anywhere)
  • kubectx - Switch between kubectl contexts easily
  • kubens - Switch between Kubernetes namespaces easily
  • docker - Docker CLI
  • docker-compose - Docker Compose

Context and Namespace Switching

Use kubectx and kubens for easy switching:

# List all contexts
kubectx

# Switch to a context
kubectx kind-eks-local-eks-a-cluster

# Switch to previous context
kubectx -

# List all namespaces
kubens

# Switch to a namespace
kubens eksa-system

# Switch to previous namespace
kubens -

Configuration Files

  • .devcontainer/devcontainer.json - DevContainer configuration with features
  • .devcontainer/Dockerfile - Container image with all tools pre-installed
  • .devcontainer/post-create.sh - Configuration script (runs once on container creation)
  • .devcontainer/start-clusters.sh - Orchestrator for starting both clusters
  • .devcontainer/start-openshift-crc.sh - OpenShift CRC startup script
  • .devcontainer/start-eks-anywhere.sh - EKS Anywhere cluster startup script
  • ~/.crc/ - CRC configuration directory
  • ~/clusters/eks-local-config.yaml - EKS Anywhere cluster configuration
  • ~/clusters/eks-local/ - EKS Anywhere cluster files and kubeconfig

Resource Requirements

OKD (OpenShift-compatible on kind)

  • Memory: ~4 GB
  • CPUs: 2 cores
  • Disk: ~5 GB
  • No nested virtualization required

EKS Anywhere (Docker provider)

  • Memory: ~4 GB
  • CPUs: 2 cores
  • Disk: ~5 GB

Troubleshooting

Clusters not starting

# Check Docker
docker ps

# Restart all clusters
./cluster-manager.sh stop-all
./cluster-manager.sh start-all

EKS Anywhere cluster issues

# Check clusters
eksctl anywhere get clusters

# View cluster info
export KUBECONFIG=~/clusters/eks-local/eks-local-eks-a-cluster.kubeconfig
kubectl cluster-info

# Restart cluster
./cluster-manager.sh restart-eks

# Delete and recreate
eksctl anywhere delete cluster eks-local
bash .devcontainer/start-eks-anywhere.sh

# Check Docker containers
docker ps --filter "name=eks-local"

OKD cluster issues

# Check cluster status
kind get clusters
kubectl get nodes

# View cluster logs
docker logs okd-local-control-plane

# Delete and recreate OKD
kind delete cluster --name okd-local
bash .devcontainer/start-okd-single-node.sh

# Check OpenShift components
kubectl get crds | grep openshift
kubectl get pods -n olm

Switching to OpenShift CRC (requires KVM)

If you have nested virtualization available:

# Check for KVM support
ls -la /dev/kvm

# If available, use CRC instead
bash .devcontainer/start-openshift-crc.sh

# Note: Requires pull secret at /usr/local/secrets/OPENSHIFT_PULL_SECRET

Context issues

# List all contexts
kubectl config get-contexts
oc config get-contexts

# View current context
oc config current-context

# Switch context manually
# For OpenShift (context name varies)
oc config use-context $(oc config get-contexts -o name | grep crc)

# For EKS
kubectl config use-context arn:aws:eks:us-east-1:000000000000:cluster/eks-local

Architecture

OKD Cluster (OpenShift-compatible)

  • Community distribution of OpenShift 4.x
  • 100% OpenShift API compatible
  • Runs on kind (Kubernetes in Docker)
  • Includes OpenShift operators and CRDs:
    • Operator Lifecycle Manager (OLM)
    • OpenShift Routes CRD
    • OpenShift Projects CRD
    • OpenShift Router
  • Compatible with both kubectl and oc commands
  • No nested virtualization required

EKS Anywhere Cluster

  • AWS's official on-premises Kubernetes distribution
  • Uses Docker provider for local development
  • Same distribution used in production EKS Anywhere deployments
  • AWS-curated Kubernetes with security patches
  • Supports EKS Anywhere features (upgrades, packages, etc.)
  • Can be upgraded to production providers (vSphere, Bare Metal, etc.)
  • Free for development use

Why EKS Anywhere?

EKS Anywhere is AWS's official on-premises Kubernetes distribution:

Benefits:

  • Official AWS distribution - Same Kubernetes used in production EKS Anywhere
  • AWS-curated - Security patches and updates from AWS
  • Free for development - Docker provider is free (production providers require license)
  • Feature-complete - Supports upgrades, curated packages, GitOps
  • Production parity - Test code that will run on real EKS Anywhere
  • Upgrade path - Can migrate to production providers later

vs LocalStack EKS:

  • LocalStack EKS requires Pro license ($)
  • EKS Anywhere is free for Docker provider
  • EKS Anywhere is the real AWS distribution

vs kind:

  • kind is generic Kubernetes
  • EKS Anywhere is AWS-curated with EKS features
  • EKS Anywhere supports AWS tooling and packages

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •