A Safer Container Runtime

Security experts within the container ecosystem have been sounding the alarm lately: Kubernetes, the darling of container orchestration, is likely to become the next major frontier for cyberattacks. As one prominent DevSecOps engineer recently put it, "We're sitting on a security time bomb with Kubernetes deployments. The complexity that makes K8s powerful also creates an expanding attack surface that most teams aren't prepared to defend."

I couldn't agree more with this assessment. With the use of containerization technology not slowing down anytime soon, it's only a matter of time before threat actors focus their efforts on containers and container orchestration platforms and low hanging fruits will be the first to be grabbed.

Rootless containers provide a way to run containers safely by running the container with a user other than the root user or running the container in privileged mode. By default, docker containers run with root privileges. This is because the Docker daemon is required to run containers and it has to run with root privileges in order to provide some powerful Docker features. You can run a container where the /host directory is mounted on the / directory of the host without restrictions. Such is the power afforded to users of the Docker daemon. However, this adds an added risk of compromise of the host which runs the daemon. But by leveraging rootless containers, your applications can be run in containers that are created, run, and managed by users without admin rights. It also adds a new layer of security by utilising identity management to prevent attackers from gaining root privileges on the host even if the container engine, runtime, or orchestrator is compromised.

Below are ways we can safely run containers to minimise the risks associated with running containers as root or in privileged mode:

• Set containers root filesystem as read-only
• Run as a non-root user
• Build rootless containers using supported tools

Running containers do not need changes to be made to the root filesystem. Attempted changes to the root filesystem often is an indication of compromise and is mostly attempted by malicious actors. The immutable nature of containers should be preserved.

Building Rootless Containers

To build truly secure containers, you need to start with the build process itself. Traditional Docker builds run as root by default, but there are better alternatives:

Kaniko

Kaniko is a tool from Google that builds container images from a Dockerfile inside a container or Kubernetes cluster without requiring privileged mode:

# Example running kaniko in a Kubernetes pod
kubectl run kaniko \
  --image=gcr.io/kaniko-project/executor:latest \
  --restart=Never \
  --env="DOCKER_CONFIG=/kaniko/.docker" \
  --env="GOOGLE_APPLICATION_CREDENTIALS=/secret/kaniko-secret.json" \
  --volumes=kaniko-secret:/secret \
  -- \
  --dockerfile=Dockerfile \
  --context=gs://my-bucket/my-app \
  --destination=my-registry/my-app:latest

Kaniko executes each command within the Dockerfile completely in userspace, which makes it a safer alternative to Docker's build process.

Buildah

Buildah is another tool that can build OCI container images without requiring root privileges:

# Install buildah
sudo apt-get install buildah

# Run buildah in rootless mode
buildah --userns=keep-id build-using-dockerfile -t my-app:latest .

# Push to a registry
buildah push my-app:latest docker://registry.example.com/my-app:latest

Buildah can be configured to run in a fully rootless mode, making it an excellent choice for CI/CD pipelines where security is a priority.

When combined with tools like Podman for running containers, you can achieve a fully rootless container workflow from build to execution.

Confidential Containers

Confidential containers represent an emerging approach to container security. They utilize hardware-based Trusted Execution Environments (TEEs) to protect sensitive data and workloads even from privileged users on the host system. This technology creates a secure enclave where container workloads can run with enhanced privacy and integrity guarantees.

Drop Capabilities

Another important security measure is dropping unnecessary Linux capabilities. Docker containers, by default, run with a reduced set of capabilities compared to a normal root user, but you can further restrict these:

docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE your-image

This removes all capabilities and only adds back the specific ones your application needs, following the principle of least privilege.

Security Scanning

Implement automated container image scanning in your CI/CD pipeline. Tools like Grype, Clair, and Anchore can detect vulnerabilities in your container images before they're deployed to production:

grype image your-image:latest

Use Container-Specific Operating Systems

Minimal, purpose-built operating systems like Alpine Linux, Bottlerocket, or Google's Container-Optimized OS reduce the attack surface by eliminating unnecessary packages and services.

Secure Your Container Runtime

Keep your container runtime up-to-date and properly configured. For example, if using containerd:

# Check for security-related configurations
containerd config default | grep -i security

Network Policies

Implement strict network policies to control the traffic between containers:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

This example blocks all incoming and outgoing traffic for pods by default.

Runtime Security Monitoring

Deploy runtime security tools that can detect and respond to suspicious activities inside containers. Solutions like Falco can provide real-time alerts on container behavior:

- rule: Terminal shell in container
  desc: A shell was used as the entrypoint/exec point into a container
  condition: container and shell_procs and not container_entrypoint
  output: Shell running in a container (user=%user.name container_id=%container.id container_name=%container.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)
  priority: WARNING

OCI Runtime Security Profiles

Leverage OCI runtime security profiles like seccomp and AppArmor to restrict what system calls and operations containers can perform:

docker run --security-opt seccomp=/path/to/seccomp/profile.json your-image

Conclusion

Container security requires a multi-layered approach. By implementing rootless containers, read-only filesystems, runtime security controls, and proper monitoring, you can significantly reduce the risk of container-based attacks. As containerization continues to grow in popularity, investing in these security measures now will help prevent your organization from becoming an easy target for attackers looking to exploit container vulnerabilities.

It is worth remembering that container security should be a continuous process, not just a one-time setup. Regular audits, updates, and staying informed about the latest security best practices and vulnerabilities are essential parts of maintaining a secure container environment.