The same image should run in development, staging and production. What changes between them is configuration: database addresses, feature flags, log levels, API keys. Kubernetes gives you two objects to supply it at runtime: ConfigMaps for ordinary settings and Secrets for sensitive values.
Why configuration does not belong in the image
If you bake DATABASE_URL into the image, the image only works in one environment. The thing you tested in staging is then not the thing you ship to production. Worse, rotating a password means building and deploying a new image. Keeping configuration outside the build is the third factor of the well-known twelve-factor app methodology.
ConfigMap or Secret?
| ConfigMap | Secret | |
|---|---|---|
| For | Non-sensitive settings | Passwords, tokens, keys, certificates |
| Stored as | Plain text in etcd | Base64 in the API, optionally encrypted at rest |
kubectl get -o yaml shows | The values | Base64 that anyone can decode |
| Maximum size | 1 MiB | 1 MiB |
| Access control | Its own RBAC rules | Its own RBAC rules, and this is what matters |
Important: base64 is an encoding, not encryption. echo cGFzc3dvcmQ= | base64 -d prints password. A Secret is a ConfigMap with safer defaults, not a vault.
What Secrets really give you: separate RBAC (people can read ConfigMaps without reading Secrets), values hidden from kubectl describe and most logs, optional encryption in etcd (often with a cloud KMS) and in-memory (tmpfs) volumes on the node. They do not protect anything from someone who is allowed to read Secrets in that namespace.
Creating them
$ kubectl create configmap shop-config --from-literal=LOG_LEVEL=info --from-file=app.yaml
configmap/shop-config created
$ kubectl create secret generic shop-db --from-literal=password='S3cure!pass'
secret/shop-db created
$ kubectl get secret shop-db -o jsonpath='{.data.password}' | base64 -d
S3cure!pass
The same ConfigMap as a manifest:
apiVersion: v1
kind: ConfigMap
metadata:
name: shop-config
data:
LOG_LEVEL: info
app.yaml: |
cache:
ttlSeconds: 300
features:
newCheckout: true
Two ways to use them, and one big difference
As environment variables:
containers:
- name: api
image: ghcr.io/jeffxl/shop-api:1.2.0
env:
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: shop-config
key: LOG_LEVEL
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: shop-db
key: password
Use envFrom to load every key of a ConfigMap or Secret at once.
As files in a volume:
containers:
- name: api
image: ghcr.io/jeffxl/shop-api:1.2.0
volumeMounts:
- name: config
mountPath: /etc/shop
readOnly: true
volumes:
- name: config
configMap:
name: shop-config
Each key becomes a file, so the app reads /etc/shop/app.yaml.
The difference that surprises everyone:
- Mounted files update in running Pods, usually within a minute of changing the ConfigMap.
- Environment variables never update. They are read once when the process starts.
After changing config that is used as environment variables, restart the Pods:
$ kubectl rollout restart deployment/shop-api
deployment.apps/shop-api restarted
Watch out: a volume mounted with subPath (a single file placed into an existing folder) never receives updates either.
Immutable, versioned configuration
Set immutable: true and Kubernetes refuses any change to the data:
apiVersion: v1
kind: ConfigMap
metadata:
name: shop-config-v3
immutable: true
data:
LOG_LEVEL: info
This has two benefits: nobody can change production config in place by accident, and kubelets stop watching the object, which takes load off the API server in big clusters. The pattern is to put a version in the name (shop-config-v4) and update the Deployment to point at it. The config change then becomes a normal rollout, with history and a rollback.
Where real secret management lives
Native Secrets are the delivery mechanism. The source of truth should live somewhere better:
- External Secrets Operator syncs values from AWS Secrets Manager, Parameter Store, HashiCorp Vault, Azure Key Vault or Google Secret Manager into native Secrets. This is the most common choice.
- Secrets Store CSI Driver mounts values straight from the external store as files, without storing them in etcd unless you ask it to.
- Sealed Secrets encrypts Secrets with a key only the cluster has, so the encrypted file is safe to commit. Popular with GitOps.
- Vault Agent Injector adds a sidecar that fetches secrets, including short-lived dynamic database credentials.
Rule: never commit a Secret manifest with real base64 values to Git. Anyone with read access to the repository has the password.
Quick reference
| Task | Command |
|---|---|
| ConfigMap from values | kubectl create configmap <name> --from-literal=KEY=value |
| ConfigMap from a file | kubectl create configmap <name> --from-file=app.yaml |
| Secret from values | kubectl create secret generic <name> --from-literal=KEY=value |
| Registry credentials | kubectl create secret docker-registry <name> --docker-server=ghcr.io --docker-username=<u> --docker-password=<token> |
| TLS certificate | kubectl create secret tls <name> --cert=tls.crt --key=tls.key |
| Read a Secret value | kubectl get secret <name> -o jsonpath='{.data.KEY}' then decode with base64 -d |
| Apply env var changes | kubectl rollout restart deployment/<name> |
Exercise
An app reads /etc/shop/app.yaml from a mounted ConfigMap, without subPath. You change the ConfigMap. What happens?
Show answer
Answer: A. Mounted ConfigMaps are refreshed by the kubelet. Whether the new values take effect depends on the app watching or re-reading the file; environment variables, by contrast, never change.
Exercise
Why give a ConfigMap immutable: true and a version in its name, such as shop-config-v4?
Show answer
Answer: C. Pointing the Deployment at a new versioned ConfigMap creates a new revision, so kubectl rollout undo also brings back the old config. Kubelets also stop watching immutable objects, which reduces API load.
Key takeaways
- Keep configuration outside the image so one image runs everywhere.
- ConfigMaps for settings, Secrets for credentials, and RBAC is the real protection.
- Secrets are not encrypted by default; base64 is only an encoding.
- Mounted files update live, environment variables do not. Restart Pods after env changes.
subPathmounts never update.- Prefer immutable, versioned ConfigMaps and Secrets.
- Use External Secrets, Sealed Secrets or Vault, and never commit real Secrets to Git.
FAQ
Are Kubernetes Secrets encrypted?
Not by default. They are base64-encoded in the API and stored in etcd. Encryption at rest must be enabled on the API server (managed services usually offer it with a KMS key), and RBAC controls who can read them.
How do I store secrets in Git safely?
Commit only something that is useless without the cluster: a SealedSecret encrypted with the cluster’s key, or an ExternalSecret that only references a path in your secret manager.
Should I use environment variables or mounted files?
Files for anything structured or that may change while the app runs, because they update live. Environment variables for small values that only change with a deploy.
Is there a size limit?
Yes, 1 MiB per ConfigMap or Secret, because they live in etcd. Larger data belongs in object storage or a volume.