Kubernetes GitOps means the desired state of your cluster lives entirely in a Git repository, and a controller inside the cluster keeps the actual state matching it. You no longer deploy with kubectl or from a pipeline, you deploy with a merge. ArgoCD does this reconciliation: it reads the repository, compares it with the cluster and applies the difference.
I run Kubernetes clusters in production and wrote the Kubernetes Practical Guide for Rheinwerk Computing (2024). In my clusters on Hetzner, production applications and production changes run through ArgoCD: everything from cert-manager through monitoring to the applications comes out of Git. Small tests and short-lived experiments sometimes go straight through Helm, but nothing reaches production except through the repository. On the way there I stumbled in a few places myself. You can find all my articles on the topic on the Kubernetes page.
What Kubernetes GitOps solves: pull instead of push
At most teams I have worked with, deployment looked like this: a CI pipeline builds the image, then that same pipeline runs kubectl apply or helm upgrade against the cluster. For that, the pipeline needs a service account with broad rights, and its credentials sit in the CI system. That is the push model.
GitOps reverses the direction. A controller runs inside the cluster, regularly pulls the Git repository and compares it with the current state, correcting any drift, the same principle Kubernetes already uses internally: a ReplicaSet counts its Pods and starts a new one whenever one is missing. GitOps takes that loop up one level, making Git the source of truth for everything in the cluster.
For you, this changes one thing above all: for production changes there is no second way into the cluster. That sounds harsh, but it brings three things worth keeping. Every change carries a commit, an author and, ideally, a review. The cluster needs no credentials in the pipeline, since nobody writes into it from outside. And drift stands out, because the controller reports it.
The price is discipline. If you are used to scaling a Deployment quickly with kubectl in an emergency, you have to unlearn that: the change goes into the repository first, or ArgoCD reverts it.
Installing ArgoCD and creating your first Application
ArgoCD runs as an application inside the cluster itself, in its own namespace. I install it via the official Helm chart, because I later want to manage ArgoCD itself through ArgoCD, and Helm values make that easy. Three commands are enough for a first try:
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm install argocd argo/argo-cd --namespace argocd --create-namespace
The admin password for the UI then sits in a secret in the argocd namespace. For your first steps, a port forward to the server service is enough; in production I put an ingress with a certificate in front, and login goes through the customer's identity provider instead of the local admin account.
The central concept in ArgoCD is the Application, connecting exactly three things: a source in Git (repository, branch or tag, path), a destination (cluster and namespace) and a sync policy. An Application for a backend that lives as a Helm chart in the repository looks like this:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: shop-backend
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/example-org/gitops.git
targetRevision: main
path: workloads/shop-backend
helm:
valueFiles:
- values.yaml
- values-prod.yaml
destination:
server: https://kubernetes.default.svc
namespace: shop
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
Three lines in this manifest decide the behavior in production. automated means ArgoCD applies changes from Git without a click. prune: true deletes objects removed from the repository; skip it and you collect leftovers. selfHeal: true reverts manual changes as soon as ArgoCD notices them. To start, leave automation off and sync by hand in the UI, so you see the diff before every sync.
One trap that surprises many people coming from helm install: ArgoCD renders the chart with helm template and applies the finished manifests. No Helm release is left in the cluster afterward, helm list stays empty. What that means for hooks and upgrades is covered in Helm Charts Explained: Packages for K8s.
A repo structure that grows with your team
How you lay out the GitOps repository depends less on the technology than on your organization. Conway's Law applies here too: a DevOps team with full ownership of its application keeps code and manifests in one repository. If development and operations are separate, there is usually a dedicated infrastructure repository with different permissions. Both paths work, what matters is choosing deliberately.
For the structure inside the repository, there are two basic patterns. Application-oriented: one directory per application, environments underneath. Environment-oriented: one directory per environment, applications underneath. The first fits when applications roll out independently, the second when a whole bundle moves through the stages together. With Helm or Kustomize, you keep one base plus small overlays or values files per environment instead of copying manifests. I compare which tool fits when in Kustomize vs Helm: When to Use Which?.
My layout for the Hetzner clusters separates platform (cert-manager, ingress controller, monitoring, policies) from the customer's workloads. Simplified, it looks like this:
gitops/
├── bootstrap/
│ └── root.yaml # one Application that creates all the others
├── apps/
│ ├── cert-manager.yaml # one Application per building block
│ ├── monitoring.yaml
│ └── shop-backend.yaml
├── platform/
│ ├── cert-manager/ # values or manifests per platform building block
│ └── monitoring/
└── workloads/
└── shop-backend/ # chart or Kustomize base plus values per environment
I used one branch per environment myself for a long time. I advise against it today: changes have to be merged from branch to branch, environments drift apart, and eventually someone develops directly on the production branch. One branch, one directory per environment, one pull request per change: that rule causes the least trouble.
App-of-apps and sync waves: the cluster bootstraps itself
If you create twenty Applications by hand, you have already broken the GitOps principle at the first step. The solution is app-of-apps: a single Application points to a directory full of other Application manifests. ArgoCD creates them, and each one pulls its part of the platform or an application into the cluster.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: root
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/example-org/gitops.git
targetRevision: main
path: apps
destination:
server: https://kubernetes.default.svc
namespace: argocd
syncPolicy:
automated:
prune: true
selfHeal: true
This one file is the only thing I still apply by kubectl after installing ArgoCD. From then on the cluster bootstraps itself from the repository. A new cluster, for me, means Terraform builds the nodes, k3s comes up, ArgoCD gets installed, the root Application gets applied, and I wait until everything turns green. For fleets of clusters, ApplicationSets generate Applications from a list or directories; for a single cluster, app-of-apps is enough.
Order is a real concern here. cert-manager has to be running before an ingress requests a certificate, and a custom resource definition has to exist before an object of that type. ArgoCD solves this with sync waves: an annotation with a number, working through them in ascending order, waiting until one is healthy before the next starts.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cert-manager
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-2"
spec:
project: default
source:
repoURL: https://charts.jetstack.io
chart: cert-manager
targetRevision: "*"
helm:
values: |
crds:
enabled: true
destination:
server: https://kubernetes.default.svc
namespace: cert-manager
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
Without an annotation, wave zero applies. I give the platform negative waves and workloads positive ones, so the foundation always stands before the first application starts. A note on targetRevision: "*": ArgoCD then follows every new chart version automatically, convenient for test environments. In production I pin the version and bump it through a pull request.
Secrets in Kubernetes GitOps: encrypted in the repository
The most common question I get about GitOps is where the passwords go if everything lives in Git. A Kubernetes secret is only base64-encoded, so it cannot go into the repository like that. There are three approaches, all of which I have seen in use.
| Approach |
What sits in Git |
Where it gets decrypted |
Fits when |
| SOPS with age or KMS |
the encrypted YAML file |
in the cluster, the key stays only there |
you want to avoid an extra service and keep the repo as the source of truth |
| Sealed Secrets |
a SealedSecret object, encrypted with the cluster certificate |
by the controller in the cluster |
you accept a separate key per cluster |
| External Secrets Operator |
just a reference to the entry in Vault or a cloud secret store |
the operator fetches the value at runtime |
a central secret store already exists anyway |
In my clusters I use SOPS with age. The values get encrypted while the key names stay readable, so a diff in the pull request still shows which secret changed. The private age key lives exclusively inside the cluster, never in the repository or the CI system. ArgoCD cannot decrypt SOPS files on its own; you need a config management plugin in the repo server that decrypts on render, or an operator that turns the encrypted object into a normal secret. Either one belongs before your first production deployment.
What I no longer do: put secrets into the cluster by hand, bypassing ArgoCD. It works, but those objects go missing at the next cluster rebuild, and nobody remembers the value.
Drift, health checks and rollback in practice
ArgoCD shows two states for every Application. The sync status says whether the cluster matches the repository. The health status says whether the objects work: a Deployment counts as healthy only once all replicas are ready, an Ingress only once it has an address. ArgoCD ships these checks for common resources; for your own custom resources, add them in Lua.
Drift happens when someone changes something bypassing the repository. With selfHeal, ArgoCD reverts that automatically; without it, the Application shows as OutOfSync and waits for you. Start with the display only, switching on selfHeal once kubectl is used for reading and debugging alone. Some differences are intentional, such as a replica count steered by a Horizontal Pod Autoscaler; exclude fields like that with ignoreDifferences.
Rollback is where GitOps shows its strength most clearly. A rollback is a git revert of the commit that introduced the problem, and ArgoCD rolls back as traceably as with any other change. The UI also offers a rollback to an earlier sync, but switches off automated sync while doing so, because Git would otherwise win again right away. I use that only to buy time during an incident, then clean up in Git.
The CI pipeline does not disappear, it just ends earlier. It still builds and tests the image, lints the manifests, and finally writes the new image tag into the GitOps repository, by commit or pull request, then ArgoCD takes over. I cover what the pipeline before that looks like in Kubernetes CI/CD: the Pipeline.
ArgoCD vs Flux: which tool fits you
The two big GitOps tools are ArgoCD and Flux, both at the CNCF, solving the same job with a different character. I chose ArgoCD because its UI makes the entry into GitOps easier for customers: a developer sees their diff, sync status and logs, without touching kubectl. For mixed teams, that outweighs technical nuances.
| Criterion |
ArgoCD |
Flux |
| Interface |
its own web UI with diff, status and logs |
no built-in UI at its core, operated via CLI and kubectl |
| Operating model |
Application objects, projects for tenant separation |
Kustomization and HelmRelease objects, one controller per job |
| Helm |
renders templates, no Helm release in the cluster |
a real Helm release via the helm-controller |
| Multiple clusters |
one instance manages many clusters |
usually one Flux instance per cluster |
| Image updates |
separate Argo CD Image Updater project |
image automation controller as part of Flux |
| Getting started |
faster through the UI |
leaner, feels closer to plain Kubernetes tooling |
Flux fits teams that already do everything through kubectl and Git, run many clusters each with its own instance, or want to keep Helm releases inside the cluster. Both projects are well documented, argo-cd.readthedocs.io and github.com/fluxcd/flux2; try them in a test cluster before you commit. Neither choice is wrong, but deploying with kubectl from a pipeline once a team runs more than a handful of services.
Frequently asked questions
What is the difference between ArgoCD and Flux?
Both implement Kubernetes GitOps: Git is the source, a controller inside the cluster reconciles it. ArgoCD ships a web UI with diff and sync status and manages many clusters from one instance. Flux consists of several small controllers, has no UI at its core and usually runs once per cluster. ArgoCD is usually the easier entry for mixed teams.
Does ArgoCD replace my CI pipeline?
No. The pipeline keeps building and testing your image and linting the manifests. It just stops earlier: instead of kubectl apply against the cluster, it writes the new image tag into the GitOps repository. ArgoCD then rolls it out, and the pipeline no longer needs access to the cluster.
How do secrets get into a GitOps repository?
Never in plain text, and never just base64-encoded. Either encrypt the values with SOPS and age, so the file sits in the repository and gets decrypted only inside the cluster, or use Sealed Secrets, or let the External Secrets Operator pull the values from a central secret store and keep only a reference in Git. The private key belongs exclusively inside the cluster either way.
What happens if I change something by hand with kubectl?
ArgoCD detects the drift at the next reconciliation and marks the Application as OutOfSync. If selfHeal is active, it enforces the state from Git right away and your change is gone. If it is off, the change stays until someone syncs. Exclude intentional drift, such as the replica count under an autoscaler, with ignoreDifferences.
How do I roll back with ArgoCD?
The clean way is a git revert of the faulty commit: ArgoCD rolls out the old state automatically, and the history stays complete. The UI also offers a rollback to an earlier sync, but disables automated sync while doing so, an emergency brake, not a permanent state.
Where to go next
If you want to introduce GitOps, start with a test cluster and a single application: install ArgoCD, create the first Application without automation, look at the diff before every sync. Once the team trusts the flow, app-of-apps, sync waves and automation follow, and finally the rule that kubectl only reads.
I have described the building blocks around this in their own articles: which steps belong in the pipeline before the merge is covered in Kubernetes CI/CD: the Pipeline, how a chart that ArgoCD renders is structured in Helm Charts Explained: Packages for K8s, and the choice between overlays and templates in Kustomize vs Helm: When to Use Which?.
In full detail, with repository structures, branching strategies and pipeline architectures, this is covered in chapter 4 of my Kubernetes Practical Guide (Rheinwerk Computing).
