Introduction to Kubernetes & EKS: Learnings & Pitfalls

Preface In today's rapidly evolving tech industry, Kubernetes (K8s) has quickly emerged as the cornerstone for container orchestration, offering a robust platform for automating containerised applications' deployment, scaling, and management. This blog post delves into the key insights from my Kubernetes and Amazon EKS (Elastic Kubernetes Service) journey, a real-world customer use case and the learnings and pitfalls I encountered along the way. A Brief History of Kubernetes Kubernetes, often abbreviated as "K8s" (8 is the number of letters between “K” and ”s”), originates from the Greek word "kubernḗtēs," meaning 'helm' or 'pilot'. Originally an in-house-developed Google product used for internal production workloads, it combines 15+ years of Google’s experience running production container workloads. It was (thankfully) open-sourced in 2014 (and accepted into CNCF in 2016 and later graduated CNCF in 2018). Since then, it's been instrumental in managing production container workloads and has become the go-to solution for automating the deployment, scaling, and management of containerized apps. Key Features of Kubernetes Kubernetes boasts several key features that make it a powerful tool for developers and IT professionals: Automated Rollout's & Rollbacks: Ensures seamless updates and rollbacks. Self-Healing: Automatically replaces failed containers. The key ideology is to be resilient by default, e.g. if a container goes down, another container needs to start, Kubernetes handles this for you. Service Discovery & Load Balancing: Efficiently manages service discovery and load balancing. Namespaces: Provides a mechanism for isolating groups of resources within a single cluster. Add-Ons: Extensions that provide additional functionalities to a Kubernetes cluster. More on this later in this article! Deploying to Kubernetes kubectl (AKA Kube/Kubernetes Control) – The main command-line tool to interact with the API to manage Kubernetes resources. IaC - Deploy resources in the form of Kubernetes Manifest files using kubectl. Package Manager - Deploy pre-packaged apps (Helm Charts) using the command-line tool helm The Complexity of Kubernetes Management While Kubernetes simplifies application management, it introduces its own set of complexities. The platform's steep learning curve can be daunting for newcomers, and managing Kubernetes clusters requires a deep understanding of its architecture and components. EKS: Simplifying Kubernetes Management Amazon EKS addresses many of the challenges associated with Kubernetes management by offering a managed service (my two favorite words in AWS!) where AWS handles the control plane components. This allows organizations to focus on deploying and managing their applications without worrying about the underlying infrastructure. In summary, the only thing we need to manage is the Control Plan Kubernetes version, worker nodes and applications! EKS Features EKS integrates seamlessly with various AWS services, providing a comprehensive solution for container orchestration: Compute: Supports AWS Fargate and EKS Managed Node Groups (EC2). Monitoring: Integrates with CloudWatch for control plane and pod logging. Networking: Integrates with AWS VPC for secure networking. Security: Utilizes IAM for access control and GuardDuty for threat detection. Storage: Offers EBS and EFS options for persistent storage. Autoscaling: Supports Cluster Autoscaler and Karpenter for dynamic scaling (via AddOns). Customer Use-Case: Real-World Insights Right, now we know how good EKS/Kubernetes is, let's get stuck into a real-world example. As part of a recent project, one of our customers required the ability to automate the build, deployment and scalability of their Web Application (a Dockerfile), which was fed with IoT Streaming data from their on-site machines. The custom requirements and the solution we built for them can be found below. Customer Requirements Dockerfile - Web app to be developed by the customer and provided as a Dockerfile Build automation to package a Docker image to a registry and deploy to a Kubernetes cluster Re-deployability - Require the ability to easily deploy/destroy several versions of the app in a parallel fashion (i.e. for each developer's feature branch) Accessibility - Each feature would need to be accessible via a custom domain, e.g. -webapp.example.com Solution Delivered An AWS Architecture Diagram can be found below, which depicts the built solution: Solution built and deployed via AWS CodePipeline & AWS CloudFormation as the IaC tool of choice (although I'd recommend Terraform!) CloudFormation used to deploy core/shared resources (e.g. EKS Cluster, VPC, ACM, WAF, Route53) EKS Fargate workers were used to allow for scalability and cost savings as the solution would start small (recommended to the customer that moving over t

Mar 28, 2025 - 22:16

Introduction to Kubernetes & EKS: Learnings & Pitfalls

Preface

In today's rapidly evolving tech industry, Kubernetes (K8s) has quickly emerged as the cornerstone for container orchestration, offering a robust platform for automating containerised applications' deployment, scaling, and management. This blog post delves into the key insights from my Kubernetes and Amazon EKS (Elastic Kubernetes Service) journey, a real-world customer use case and the learnings and pitfalls I encountered along the way.

A Brief History of Kubernetes

Kubernetes, often abbreviated as "K8s" (8 is the number of letters between “K” and ”s”), originates from the Greek word "kubernḗtēs," meaning 'helm' or 'pilot'. Originally an in-house-developed Google product used for internal production workloads, it combines 15+ years of Google’s experience running production container workloads. It was (thankfully) open-sourced in 2014 (and accepted into CNCF in 2016 and later graduated CNCF in 2018).

Since then, it's been instrumental in managing production container workloads and has become the go-to solution for automating the deployment, scaling, and management of containerized apps.

Key Features of Kubernetes

Kubernetes boasts several key features that make it a powerful tool for developers and IT professionals:

Automated Rollout's & Rollbacks: Ensures seamless updates and rollbacks.
Self-Healing: Automatically replaces failed containers. The key ideology is to be resilient by default, e.g. if a container goes down, another container needs to start, Kubernetes handles this for you.
Service Discovery & Load Balancing: Efficiently manages service discovery and load balancing.
Namespaces: Provides a mechanism for isolating groups of resources within a single cluster.
Add-Ons: Extensions that provide additional functionalities to a Kubernetes cluster. More on this later in this article!

Deploying to Kubernetes

kubectl (AKA Kube/Kubernetes Control) – The main command-line tool to interact with the API to manage Kubernetes resources.
IaC - Deploy resources in the form of Kubernetes Manifest files using kubectl.
Package Manager - Deploy pre-packaged apps (Helm Charts) using the command-line tool helm

The Complexity of Kubernetes Management

While Kubernetes simplifies application management, it introduces its own set of complexities. The platform's steep learning curve can be daunting for newcomers, and managing Kubernetes clusters requires a deep understanding of its architecture and components.

EKS: Simplifying Kubernetes Management

Amazon EKS addresses many of the challenges associated with Kubernetes management by offering a managed service (my two favorite words in AWS!) where AWS handles the control plane components. This allows organizations to focus on deploying and managing their applications without worrying about the underlying infrastructure. In summary, the only thing we need to manage is the Control Plan Kubernetes version, worker nodes and applications!

EKS Features

EKS integrates seamlessly with various AWS services, providing a comprehensive solution for container orchestration:

Compute: Supports AWS Fargate and EKS Managed Node Groups (EC2).
Monitoring: Integrates with CloudWatch for control plane and pod logging.
Networking: Integrates with AWS VPC for secure networking.
Security: Utilizes IAM for access control and GuardDuty for threat detection.
Storage: Offers EBS and EFS options for persistent storage.
Autoscaling: Supports Cluster Autoscaler and Karpenter for dynamic scaling (via AddOns).

Customer Use-Case: Real-World Insights

Right, now we know how good EKS/Kubernetes is, let's get stuck into a real-world example. As part of a recent project, one of our customers required the ability to automate the build, deployment and scalability of their Web Application (a Dockerfile), which was fed with IoT Streaming data from their on-site machines. The custom requirements and the solution we built for them can be found below.

Customer Requirements

Dockerfile - Web app to be developed by the customer and provided as a Dockerfile
Build automation to package a Docker image to a registry and deploy to a Kubernetes cluster
Re-deployability - Require the ability to easily deploy/destroy several versions of the app in a parallel fashion (i.e. for each developer's feature branch)
Accessibility - Each feature would need to be accessible via a custom domain, e.g. -webapp.example.com

Solution Delivered

An AWS Architecture Diagram can be found below, which depicts the built solution:

Solution built and deployed via AWS CodePipeline & AWS CloudFormation as the IaC tool of choice (although I'd recommend Terraform!)
CloudFormation used to deploy core/shared resources (e.g. EKS Cluster, VPC, ACM, WAF, Route53)
EKS Fargate workers were used to allow for scalability and cost savings as the solution would start small (recommended to the customer that moving over to EC2-based Node Groups should be considered in future to save on costs).
ECR was used to store our docker images to be deployed to EKS.
An EventBridge rule was created to listen to branch creations in the associated CodeCommit repository (where the Dockerfile lived).
This rule would trigger a Lambda Function to provision an isolated CodePipeline Pipeline (for each developer) based on this branch and provision the infrastructure shown in the above diagram.
CodePipeline included CodeBuild jobs to build docker image, push to ECR and deploy to EKS cluster using kubectl (pointing to image in ECR).
Kubectl (via CodeBuild) was used to connect to and deploy the required Kubernetes resources via Kubernetes manifests.
The following Kubernetes Add-Ons were utilized for this solution which worked in conjunction with one-another. a. AWS Load Balancer Controller - An amazing AddOn which automates the creation of the AWS Application Load Balancer (based on the creation of an Ingress resource) and points this at our application pods. It also magically manages the Target Group for us so that our ALB is always pointing to our live pods. b. External DNS - Updates and manages our Route53 records to point to our load balancer (again, using the config in the Ingress resource).

Learnings & Pitfalls

Throughout our journey with Kubernetes and EKS, we encountered several learnings and pitfalls:

Availability: EKS Fargate is available only on Linux and in private subnets.
Metrics & Logging: Requires additional configuration for CloudWatch observability (especially on EKS Fargate).
CloudFormation Kubernetes Support - Used to be able to use EKS Quick-start but no longer supported (as of 31/03/2023). I would recommend using Terraform to manage this instead, see here.
Cost Management: EKS Cluster's cannot be paused/stopped (supported by other providers). Although can schedule to scale down your compute resources using Karpenter & Cluster Autoscaler.
Maintenance: Regular updates and maintenance are required for Kubernetes versions.
Complexity – Kubernetes/EKS can be hard to grasp due to steep learning curve. Could be made a little bit easier in the EKS console for first timers (e.g. when setting up coredns)

The Future is Bright

AWS continues to enhance EKS with new features and improvements. Recent updates include (but are not limited to): simplified IAM cluster access management, EKS Pod Identity, extended support for Kubernetes versions, EKS Upgrade Insights. These advancements ensure that EKS remains a cutting-edge solution for container orchestration.

Conclusion

Kubernetes and EKS offer powerful tools for managing containerized applications, but they come with their own set of challenges. Organizations can leverage these platforms to enhance their digital presence and improve customer engagement by understanding the key features, complexities, and real-world applications.

For further reading, be sure to check out the Kubernetes documentation and the AWS EKS Advanced Workshop.

Get Kubernetes Certified! (with the Linux Foundation, parent of CNCF)!

If you're looking to get certified, I'd recommend looking at either the Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), or Certified Kubernetes Security Specialist (CKS) certifications (dependent on your interests). I completed the CKA back in late 2023 (see my Credly) and would highly recommend this to anyone interested in this topic.