Externalize configuration and sensitive data. Never hardcode config or secrets in your container images.
Imagine your app is a phone. ConfigMaps are like the phone's Settings app — language, timezone, display preferences. You can change settings without buying a new phone. Secrets are like the passwords stored in your phone's secure keychain — they need extra protection. Both let you configure your app without rebuilding the container image.
Store non-sensitive configuration as key-value pairs. Can be consumed as environment variables, command arguments, or mounted as files inside a pod.
Similar to ConfigMaps but designed for sensitive data (passwords, tokens, keys). Values are base64-encoded (not encrypted by default). Can be encrypted at rest with encryption configuration.
Kubernetes Secrets are base64-encoded, not encrypted by default. Anyone with RBAC access to read Secrets can decode them. For production: enable encryption at rest, use external secret managers (Azure Key Vault, AWS Secrets Manager, HashiCorp Vault), and restrict Secret access via RBAC.
| Method | How | When to Use |
|---|---|---|
| Environment Variables | envFrom or individual env entries | Simple key-value config (DB_HOST, LOG_LEVEL) |
| Volume Mounts | Mount as files in a directory | Config files (nginx.conf, app.properties) |
| Command Arguments | Reference in args | CLI-based apps needing config flags |
# Imperative kubectl create configmap app-config \ --from-literal=DB_HOST=postgres.default.svc \ --from-literal=DB_PORT=5432 \ --from-literal=LOG_LEVEL=info # Verify kubectl get configmap app-config -o yaml
# Create a config file
echo "server {
listen 80;
server_name localhost;
location / {
root /usr/share/nginx/html;
}
}" > nginx.conf
# Create ConfigMap from file
kubectl create configmap nginx-config --from-file=nginx.conf
# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
DB_HOST: "postgres.default.svc"
DB_PORT: "5432"
LOG_LEVEL: "info"
app.properties: |
spring.datasource.url=jdbc:postgresql://postgres:5432/mydb
spring.jpa.hibernate.ddl-auto=update
logging.level.root=INFO
# pod-with-configmap.yaml
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: app
image: my-app:1.0
# Load all keys as env vars
envFrom:
- configMapRef:
name: app-config
# Or load specific keys
env:
- name: DATABASE_HOST
valueFrom:
configMapKeyRef:
name: app-config
key: DB_HOST
apiVersion: v1
kind: Pod
metadata:
name: nginx-custom
spec:
containers:
- name: nginx
image: nginx:1.25
volumeMounts:
- name: config-vol
mountPath: /etc/nginx/conf.d
volumes:
- name: config-vol
configMap:
name: nginx-config
# Imperative (kubectl base64-encodes automatically)
kubectl create secret generic db-credentials \
--from-literal=DB_USER=admin \
--from-literal=DB_PASSWORD=s3cur3P@ss!
# Verify (values shown as base64)
kubectl get secret db-credentials -o yaml
# Decode a specific value
kubectl get secret db-credentials -o jsonpath='{.data.DB_PASSWORD}' | base64 -d
# secret.yaml apiVersion: v1 kind: Secret metadata: name: db-credentials type: Opaque data: DB_USER: YWRtaW4= # base64 of "admin" DB_PASSWORD: czNjdXIzUEBzcyE= # base64 of "s3cur3P@ss!" # Alternatively, use stringData (plain text, encoded on apply): # stringData: # DB_USER: admin # DB_PASSWORD: s3cur3P@ss!
apiVersion: v1
kind: Pod
metadata:
name: app-with-secrets
spec:
containers:
- name: app
image: my-app:1.0
env:
- name: DB_USER
valueFrom:
secretKeyRef:
name: db-credentials
key: DB_USER
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-credentials
key: DB_PASSWORD
# Or mount as files:
volumeMounts:
- name: secret-vol
mountPath: /etc/secrets
readOnly: true
volumes:
- name: secret-vol
secret:
secretName: db-credentials
When a Secret is mounted as a volume, Kubernetes automatically updates the files when the Secret changes (with a small delay). Environment variables from Secrets do NOT auto-update — the pod needs a restart.
Symptom: App starts but can't reach the database. Logs show connection timeout.
# Check what env vars the pod actually has kubectl exec my-pod -- env | grep DB # Check the ConfigMap values kubectl get configmap app-config -o yaml # Common issues: # - Typo in ConfigMap key name # - ConfigMap in wrong namespace # - envFrom vs env mismatch # - ConfigMap updated but pod not restarted (env vars don't auto-update)
Symptom: Pod stuck in CreateContainerConfigError.
# Check pod events kubectl describe pod my-pod # Look for: "secret 'db-credentials' not found" # Fix: Create the missing secret kubectl create secret generic db-credentials ... # Or check if secret is in a different namespace kubectl get secrets -A | grep db-credentials
Root cause: Environment variables are injected at pod creation time and don't update.
Solutions:
kubectl rollout restart deployment/my-appA ConfigMap is a Kubernetes object that stores non-sensitive configuration data as key-value pairs. It decouples configuration from container images, allowing the same image to be used across different environments with different configurations.
A Secret stores sensitive data (passwords, tokens, keys). Key differences: 1) Values are base64-encoded. 2) Kubernetes applies stricter RBAC defaults. 3) Can be encrypted at rest. 4) Less likely to be accidentally logged or exposed. However, Secrets are NOT encrypted by default — base64 is encoding, not encryption.
kubectl create configmap my-config --from-literal=KEY=value for key-value pairs. --from-file=config.txt to load a file. --from-env-file=.env for env file format. Or declaratively with a YAML manifest and kubectl apply.
1) Environment variables: envFrom (all keys) or individual env entries with configMapKeyRef. 2) Volume mounts: Each key becomes a file in the mount directory. 3) Command arguments: Reference ConfigMap values in the pod's args field.
By default, no. They are base64-encoded and stored as plain text in etcd. To encrypt: enable EncryptionConfiguration for etcd at-rest encryption. For stronger security, use external secret managers (Azure Key Vault, AWS Secrets Manager, Vault) with operators like External Secrets Operator.
If consumed as environment variables: they're set at pod creation time and never change. The pod must be restarted. If consumed as a volume mount: Kubernetes updates the files automatically (kubelet sync period, typically ~1 minute), but the app must re-read the files. Many apps only read config at startup, so a restart may still be needed.
Setting immutable: true prevents any changes to the ConfigMap/Secret after creation. Benefits: 1) Protects against accidental changes. 2) Improves cluster performance — Kubernetes stops watching immutable objects for changes. To update: delete and recreate with a new name, then update pod references.
Never commit plain Secrets to Git. Options: 1) Sealed Secrets (Bitnami): encrypts secrets client-side, stores encrypted version in Git, controller decrypts in-cluster. 2) External Secrets Operator: syncs secrets from external stores (Key Vault, Secrets Manager) into K8s Secrets. 3) SOPS: encrypts YAML files with age/PGP keys. 4) Vault Agent Injector: injects secrets at runtime from HashiCorp Vault.
Both have a 1 MiB (1,048,576 bytes) limit per object stored in etcd. For Secrets, the base64 encoding adds ~33% overhead, so the actual data limit is about 750 KB. For larger configs, consider mounting from PersistentVolumes or using init containers to download config.
Opaque (default) — arbitrary user data. kubernetes.io/dockerconfigjson — Docker registry credentials. kubernetes.io/tls — TLS certificates (cert + key). kubernetes.io/basic-auth — basic auth credentials. kubernetes.io/service-account-token — auto-generated for service accounts. Each type has validation rules for required keys.
Options: 1) Azure Key Vault Provider for Secrets Store CSI Driver: mounts secrets as files directly from Key Vault into pods. 2) External Secrets Operator: syncs Key Vault secrets to K8s Secrets, then consumed normally via envFrom/volumes. 3) Workload Identity: pod authenticates directly to Azure AD, app sdk reads from Key Vault at runtime (no K8s Secret needed). Option 3 is most secure — secrets never stored in K8s.
1) Immediately rotate the credential — assume it's compromised. 2) Remove from Git history: git filter-branch or BFG Repo Cleaner. 3) Force push to overwrite history. 4) Implement prevention: Git hook or CI check scanning for secrets (e.g., gitleaks, trufflehog). 5) Switch to Sealed Secrets or External Secrets Operator so plain secrets never enter Git.
1) Create one TLS Secret: kubectl create secret tls shared-cert --cert=tls.crt --key=tls.key. 2) Reference it in each pod spec or Ingress. 3) For auto-renewal, use cert-manager — it provisions and auto-renews certificates from Let's Encrypt or internal CAs. 4) cert-manager creates/updates the TLS secret automatically before expiry. This eliminates manual certificate management entirely.
K8s doesn't do this automatically. Strategies: 1) Checksum annotation: Add a pod annotation with the ConfigMap hash (Helm does this with checksum/config). When the ConfigMap changes, the hash changes, triggering a rollout. 2) Reloader (stakater): A controller that watches ConfigMaps/Secrets and triggers rolling restarts. 3) Versioned names: config-v1, config-v2 — changing the reference triggers a rollout.
This error means a referenced ConfigMap or Secret doesn't exist. Steps: 1) kubectl describe pod — the event will say which ConfigMap/Secret is missing. 2) kubectl get configmaps / kubectl get secrets in the same namespace. 3) Common causes: typo in name, resource in wrong namespace, resource deleted. 4) Check if optional: true should be set if the config isn't required.
A banking application uses a layered configuration strategy:
/etc/app/config.json