Amazon EKS

EKS Core Architecture

An EKS cluster is composed of two primary elements: the Control Plane and the Data Plane (worker nodes).

1. EKS Control Plane

The EKS control plane is a highly available and scalable single-tenant environment managed entirely by AWS. It runs the core Kubernetes software components responsible for cluster management.

• Architecture: The control plane infrastructure is not shared with other customers or across other AWS accounts. It consists of multiple master nodes distributed across at least two Availability Zones (AZs), ensuring high availability. AWS manages the health, scaling, and patching of these nodes.
• Components: It runs key Kubernetes objects like:
  • etcd: The consistent and highly-available key-value store for all cluster data.
  • kube-apiserver: Exposes the Kubernetes API, which is the frontend for the control plane.
  • kube-scheduler: Watches for newly created Pods and assigns them to nodes.
  • kube-controller-manager: Runs controller processes.
• Endpoint Access: EKS provides a single API endpoint for your cluster. You can configure endpoint access to be public, private, or a hybrid of both, allowing you to control network access to your cluster's API server.

2. EKS Data Plane (Worker Nodes)

The data plane consists of the worker nodes where your containerized applications (as Pods) are executed. EKS offers several flexible options for provisioning and managing these nodes.

• Function: Worker nodes run the kubelet agent, which communicates with the control plane, and a container runtime (like containerd) to run pods.
• Integration: They must be registered with the EKS control plane to become part of the cluster.

EKS Compute and Node Management

EKS provides three primary models for managing the worker nodes in your data plane.

1. EKS Managed Node Groups

This is the most common and recommended approach for provisioning EC2-based worker nodes.

• Automation: AWS handles the full lifecycle management, including provisioning, automatic scaling, and graceful draining of nodes during updates or termination.
• Instance Types: You can use a mix of On-Demand and Spot Instances within a single node group to optimize costs.
• Customization: You can use custom AMIs to meet specific compliance and security requirements.
• Scaling: Integrates with the Cluster Autoscaler to automatically adjust the number of nodes based on workload demands.

2. Self-Managed Nodes

This option provides maximum control and flexibility over the configuration and management of worker nodes.

• Responsibility: You are responsible for provisioning, configuring, scaling, and patching the EC2 instances.
• Use Case: Ideal for scenarios requiring significant customization of the underlying EC2 instances or for teams that need to follow specific, non-standard node provisioning workflows.
• Tooling: AWS provides optimized AMIs with the necessary dependencies (Docker, kubelet) pre-installed to simplify the setup.

3. AWS Fargate

AWS Fargate is a serverless compute engine for containers that eliminates the need to manage servers or clusters of EC2 instances.

• Serverless Kubernetes: With Fargate, you can run pods without provisioning any worker nodes. Each pod runs in its own isolated, kernel-level environment.
• Fargate Profiles: To direct pods to run on Fargate, you create a Fargate Profile. A profile specifies which pods should run on Fargate based on namespaces and labels.
• Pod-Level Scaling: Billing and scaling are at the pod level. You pay for the vCPU and memory resources consumed by your pods, not for the underlying virtual machine.
• Immutable: Fargate profiles are immutable. If you need to change a profile, you must delete it and create a new one.

Networking in EKS

EKS networking is built on top of the Amazon VPC, providing secure and isolated networking for your clusters.

• VPC Integration: The EKS control plane communicates with worker nodes via network interfaces (ENIs) deployed in the VPC subnets you specify.
• CNI Plugin: EKS uses the Amazon VPC CNI (Container Network Interface) plugin by default. This plugin assigns a routable IP address directly from your VPC to each pod, allowing them to be treated like first-class citizens within the VPC.
• Load Balancing: Integrates seamlessly with AWS Elastic Load Balancing. You can use Application Load Balancers (ALB) via the AWS Load Balancer Controller for Layer 7 routing or Network Load Balancers (NLB) for Layer 4 traffic distribution.
• Security Groups: You can use Security Groups for Pods to define fine-grained network traffic rules at the pod level, enhancing the security posture of your applications.

Storage for EKS Pods

EKS supports various storage options through the Kubernetes Container Storage Interface (CSI).

• Amazon EBS CSI Driver: Allows you to manage the lifecycle of Amazon EBS volumes as persistent storage for your stateful applications. It supports dynamic provisioning, which automatically creates a volume when a PersistentVolumeClaim is made.
• Amazon EFS CSI Driver: Enables pods to mount Amazon EFS file systems, providing shared, persistent storage that can be accessed by multiple pods simultaneously.
  • EFS is ideal for workloads that require a shared file system, such as content management systems or developer tools.
  • Pods running on Fargate can automatically mount an EFS file system for persistent storage.

Security and Authentication

EKS leverages AWS Identity and Access Management (IAM) to provide robust authentication and authorization for your cluster.

• Cluster Authentication: EKS uses IAM for authenticating to the Kubernetes API server. It does not use the traditional certificate-based authentication of a self-managed Kubernetes cluster.
• IAM Roles for Service Accounts (IRSA): This feature allows you to associate an IAM role directly with a Kubernetes service account. This provides a secure, credential-less way for pods to access other AWS services without needing to store AWS credentials inside the pod.
• Authorization: For authorization within the cluster (e.g., what actions a user can perform on Kubernetes resources), EKS relies on the native Kubernetes Role-Based Access Control (RBAC) system.

Monitoring and Logging

EKS provides deep integration with AWS monitoring and logging services.

• Control Plane Logging: You can enable logging for the core control plane components (api-server, authenticator, scheduler, etc.) and send these logs directly to Amazon CloudWatch Logs for analysis, auditing, and debugging.
• Application and System Logs: For monitoring your applications and worker nodes, you can use a combination of tools:
  • CloudWatch Container Insights: A fully managed service that collects, aggregates, and summarizes metrics and logs from your containerized applications and microservices.
  • AWS Distro for OpenTelemetry: An AWS-supported distribution of the OpenTelemetry project, allowing you to collect traces and metrics for monitoring your applications.
  • Prometheus and Grafana: Open-source solutions that are widely used in the Kubernetes community and can be deployed on EKS.
• CloudTrail Integration: All EKS API calls made by users, roles, or AWS services are recorded as events in AWS CloudTrail, providing a complete audit trail of actions taken on your cluster.