Lesson 05 beginner

Kubernetes Pods

What a Pod is and what its containers share, how to write a good Pod spec, init containers and sidecars, the Pod lifecycle and how to debug Pending, ImagePullBackOff and CrashLoopBackOff.

A Pod is the smallest thing Kubernetes runs. It wraps one or more containers that are always scheduled together on the same node and share one network address. Most Pods hold a single container, and you will rarely create Pods by hand, but every workload you run ends up as Pods, so understanding them pays off in every debugging session.

Why wrap containers at all?

Some processes belong together. A log shipper has to read the files your app writes. A proxy has to intercept your app’s traffic on localhost. If Kubernetes scheduled these containers independently, they could land on different machines and break. A Pod guarantees they are placed, started and stopped as one unit.

What containers in a Pod share

SharedNot shared
One network namespace: one IP address and one port spaceThe filesystem, except volumes you declare
localhost between containersProcesses (each container sees only its own by default)
Volumes declared in the PodCPU and memory limits (set per container)
Lifecycle: scheduled and deleted togetherRestarts: each container restarts on its own

Because the port space is shared, two containers in the same Pod cannot listen on the same port. The second one fails to bind.

Behind the scenes, a tiny “pause” container holds the network namespace open. That is why your app container can crash and restart without the Pod losing its IP address.

Pods are disposable, and so are their IPs

Every Pod gets an IP address that other Pods can reach directly. That address disappears with the Pod, and Pods are replaced all the time: during deploys, when a node is drained, when the node runs out of memory.

$ kubectl get pods -o wide
NAME                        READY   STATUS    IP            NODE
shop-api-6d9f7b8c4-kq2xv    1/1     Running   10.244.1.17   lab-worker
shop-api-6d9f7b8c4-p8mzt    1/1     Running   10.244.2.9    lab-worker2

Never hardcode a Pod IP. Put a Service in front of your Pods (lesson 7) and use its stable name.

Writing a Pod

Here is a single Pod with the settings every production container should have:

apiVersion: v1
kind: Pod
metadata:
  name: shop-api
  labels:
    app: shop-api
spec:
  containers:
    - name: api
      image: ghcr.io/jeffxl/shop-api:1.2.0
      ports:
        - name: http
          containerPort: 8080
      env:
        - name: LOG_LEVEL
          value: info
      resources:
        requests:
          cpu: 100m
          memory: 128Mi
        limits:
          memory: 256Mi
      readinessProbe:
        httpGet:
          path: /healthz
          port: http
        initialDelaySeconds: 5
        periodSeconds: 10

Three habits to build from day one:

  • Pin the image tag (1.2.0), never latest. Otherwise two replicas can run different code and rollbacks do not mean anything.
  • Always set requests. The scheduler places Pods using requests. A Pod without them is packed badly and is the first to be evicted under pressure.
  • Know what limits do. A memory limit kills the container (OOMKilled) when it is exceeded. A CPU limit only throttles it. Lesson 11 goes deeper.

Multi-container patterns

Init containers

Init containers run one after another, to completion, before the app containers start. Use them for work that must finish first, such as database migrations or waiting for a dependency.

spec:
  initContainers:
    - name: migrate
      image: ghcr.io/jeffxl/shop-api:1.2.0
      command: ["./shop-api", "migrate", "up"]
  containers:
    - name: api
      image: ghcr.io/jeffxl/shop-api:1.2.0

A Pod stuck in Init:0/1 means the init container is failing or still waiting. Read its logs with kubectl logs <pod> -c migrate.

Sidecars

A sidecar runs for the whole life of the Pod next to your app: a log shipper, a metrics exporter or a service mesh proxy. Modern Kubernetes has native sidecars: an init container with restartPolicy: Always starts before the app, keeps running and is stopped after it, so a Job can still complete.

spec:
  initContainers:
    - name: log-shipper
      image: fluent/fluent-bit:3.1
      restartPolicy: Always
      volumeMounts:
        - name: logs
          mountPath: /var/log/app
  containers:
    - name: api
      image: ghcr.io/jeffxl/shop-api:1.2.0
      volumeMounts:
        - name: logs
          mountPath: /var/log/app
  volumes:
    - name: logs
      emptyDir: {}

Adapters

An adapter reshapes what your app produces, for example converting a custom metrics format into one Prometheus understands.

Rule of thumb: if a second container is not an init step, a sidecar or an adapter, it probably belongs in its own Pod.

The Pod lifecycle

PhaseMeaning
PendingAccepted, but not running yet: waiting for a node or pulling images
RunningBound to a node and at least one container is running
SucceededAll containers exited with code 0 and will not restart
FailedAll containers stopped and at least one failed
UnknownThe node stopped reporting

restartPolicy (Always, OnFailure or Never) controls what happens to containers that exit. A Pod itself is never “restarted”: it is deleted and a controller creates a new one with a new name and IP.

Debugging: the three statuses you will see

Always start with kubectl describe pod <name> and read the Events at the bottom.

Pending means the Pod could not be scheduled:

$ kubectl describe pod shop-api-6d9f7b8c4-x7c2d
...
  Warning  FailedScheduling  0/3 nodes are available: 3 Insufficient cpu.

Typical causes: requests larger than any node has free, a nodeSelector or affinity that matches no node, a taint the Pod does not tolerate, or a PersistentVolumeClaim that is not bound.

ImagePullBackOff or ErrImagePull means the node cannot fetch the image: a typo in the name, a tag that does not exist or a private registry with no imagePullSecrets.

CrashLoopBackOff means the container starts, exits and is restarted with growing delays (10 seconds, 20, 40… up to 5 minutes). It is a symptom, not a cause. The current container may have no logs yet, so read the run that crashed:

$ kubectl logs shop-api-6d9f7b8c4-kq2xv --previous
Error: required environment variable DATABASE_URL is not set
$ kubectl get pod shop-api-6d9f7b8c4-kq2xv \
    -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'
Error

Common causes: missing configuration, a dependency that is down at startup, OOMKilled (the reason above shows it) and a liveness probe that fires before the app is ready.

Do not create bare Pods

A Pod created directly has no controller. If it dies or its node fails, nothing replaces it, and you cannot scale or roll it out. In real workloads, let a controller create Pods for you:

  • Deployment for stateless apps (lesson 6).
  • StatefulSet for apps that need stable names and their own storage.
  • DaemonSet for one Pod per node, such as log and monitoring agents.
  • Job and CronJob for work that runs to completion.

A bare Pod is perfect for a one-off debug shell that cleans itself up:

$ kubectl run debug --rm -it --image=nicolaka/netshoot -- bash

Quick reference

TaskCommand
List with IP and nodekubectl get pods -o wide
Details and eventskubectl describe pod <name>
Logskubectl logs <name>
Logs of the crashed runkubectl logs <name> --previous
Logs of one containerkubectl logs <name> -c <container>
Shell insidekubectl exec -it <name> -- sh
Watch changeskubectl get pods -w
Full stored objectkubectl get pod <name> -o yaml
Force delete a stuck Podkubectl delete pod <name> --grace-period=0 --force

Exercise

A Pod shows Init:0/1 for several minutes. Where do you look first?

  1. The init container’s logs, with kubectl logs <pod> -c <init-container>
  2. The main container’s logs
  3. The Service endpoints
  4. kubectl top pod
Show answer

Answer: A. The app containers have not started yet, so they have no logs. The init container is still running or failing, and its own logs say why.

Exercise

Why should production Pods never use the latest image tag?

  1. Kubernetes refuses to pull images tagged latest
  2. Images tagged latest are slower to download
  3. Replicas can end up running different code, and a rollback cannot tell which version to return to
  4. The latest tag disables readiness probes
Show answer

Answer: C. latest is just a movable label. Two Pods started a day apart can pull different images under the same name. Pin a version tag or a digest.

Key takeaways

  • A Pod is the unit of scheduling: its containers share one IP, one port space and the volumes you declare.
  • Pod IPs change all the time; reach Pods through a Service.
  • Pin image tags and always set resource requests.
  • Memory limits kill, CPU limits throttle.
  • Use init containers for setup steps and native sidecars for helpers that run alongside the app.
  • CrashLoopBackOff is a symptom: read kubectl logs --previous.
  • Do not run bare Pods in production; use a controller.

FAQ

What is the difference between a Pod and a container?

A container is one running image. A Pod wraps one or more containers that share a network address, volumes and a lifecycle, and it is what Kubernetes schedules onto nodes.

Why is my Pod stuck in Pending?

The scheduler found no suitable node. kubectl describe pod shows the reason: not enough CPU or memory requests left, a selector or taint mismatch, or an unbound PersistentVolumeClaim.

What does CrashLoopBackOff mean?

The container keeps exiting and Kubernetes waits longer before each restart. Read kubectl logs <pod> --previous and the termination reason to find the real cause.

Why does my Pod keep getting OOMKilled?

The container used more memory than its limit. Check real usage with kubectl top pod, look for leaks and make sure runtimes such as the JVM size their heap from the container limit, not from the node’s memory.

↑ ↓ navigate Enter open Esc close