Handling Secrets in an Argo CD Manifests Repo
Secrets can't go into Git in plaintext. How do you handle them in an Argo CD setup, and what does that look like in your repo structure?
Secrets can't go into Git in plaintext. How do you handle them in an Argo CD setup, and what does that look like in your repo structure?
Three approaches, in order of how I would pick today. Pick External Secrets Operator for most teams. Pick Sealed Secrets if you do not have an external secrets manager. Avoid SOPS-only setups for teams larger than a handful of people because key distribution gets ugly. External Secrets Operator (ESO). The actual secret lives in AWS Secrets Manager, Vault, GCP Secret Manager, or similar. In Git you commit an ExternalSecret CR that says 'fetch this key from that store and create a Kubernetes Secret named X in namespace Y'. Argo CD syncs the ExternalSecret. ESO does the fetch. No ciphertext in Git, the secrets manager handles rotation, IAM controls access. This is what I default to. Sealed Secrets (Bitnami). Run the sealed-secrets controller in the cluster. It generates a public key. You encrypt secrets locally with kubeseal using that public key. The encrypted SealedSecret CR is safe to commit. The controller decrypts it in-cluster and creates a normal Secret. The public key is per cluster, so a SealedSecret encrypted for dev does not work in prod. That is a feature. Repo layout with ESO (per env, since secret references differ): apps/ web/ base/ external-secret.yaml # template with placeholders kustomization.yaml overlays/ dev/ external-secret-patch.yaml # patches secretStoreRef + remote key path prod/ external-secret-patch.yaml platform/ external-secrets/ cluster-secret-store-dev.yaml cluster-secret-store-prod.yaml Repo layout with Sealed Secrets: apps/ web/ base/ deployment.yaml kustomization.yaml overlays/ dev/ sealed-secret.yaml # encrypted with dev cluster public key prod/ sealed-secret.yaml # encrypted with prod cluster public key Things I never do regardless of approach: 1. No plaintext Secret manifests in Git, even base64-encoded. Base64 is not encryption. 2. No single key shared across all environments. The whole point is that a leaked dev key cannot decrypt prod secrets. 3. No secret values in Argo CD Application parameters or Helm values files committed to Git, because they end up in argocd-server memory and CLI history. 4. No mixing: pick one approach per cluster. Two systems means twice the rotation pain and unclear ownership. One Argo CD-specific gotcha. When ESO or sealed-secrets creates the underlying Secret, Argo CD might show the parent Application as OutOfSync because the Secret it created is not tracked in Git. Add the Secret to ignoreDifferences, or annotate the ExternalSecret/SealedSecret so Argo CD knows the child Secret is managed by another controller. Otherwise self-heal will fight with the secret controller and you get reconcile loops.
Secrets in GitOps is the question that exposes who has actually shipped GitOps versus who has read about it. Strong candidates can name at least two approaches, articulate the trade-offs, and explain the Argo CD-specific sync issues that come up when a child Secret is created by another controller. Weak candidates either say 'put them in a ConfigMap' (terrifying) or 'use sealed-secrets' without being able to explain key rotation. Watch for awareness that base64 is not encryption.
ExternalSecret CR committed to Git, no plaintext anywhere
Creating a SealedSecret with kubeseal
Argo CD Application ignoring Secret diffs created by ESO
- Committing base64-encoded Secret YAML and thinking it is safe because it is not obvious plaintext
- Using one sealed-secrets key across all clusters, defeating the whole point of per-cluster encryption
- Not configuring ignoreDifferences on the Application, then watching Argo CD selfHeal fight with ESO in an endless reconcile loop
- How do you rotate a secret with ESO versus with Sealed Secrets? What is the engineer experience for each?
- A sealed-secrets controller's private key gets compromised. What is your recovery plan?
- You are running multi-cluster Argo CD. How does that affect sealed-secrets specifically?
- Why is putting secrets in a Helm values file in Git, even for dev, still a bad idea?
More GitOps interview questions
Also worth your time on this topic
Argo CD Multi-Environment Repository Structure Checklist
How to organize your Git repositories when running Argo CD across dev, staging, and production. Covers folder layout, app-of-apps, ApplicationSets, secrets, RBAC, and promotion flow.
60-90 minutes
Bootstrapping Argo CD and Letting It Manage Itself
Argo CD manages your apps. Who manages Argo CD? Walk me through how you would bootstrap it from a fresh cluster and where its own config lives in your repo.
junior
GitOps with Argo CD: Structuring Your Repository for Multi-Environment Deployments
A practical guide to laying out your Git repository for Argo CD across dev, staging, and production. See real folder structures, Kustomize and Helm patterns, and the pitfalls that bite teams in production.