The client had 120 microservices deployed across three Kubernetes clusters. Deployments were triggered by Jenkins pipelines that ran kubectl apply commands directly. Configuration drift was endemic — what was running in production frequently didn't match what was in Git. Rollbacks took 45 minutes on average and required manual intervention.
The migration to GitOps took six months. By the end, deployment frequency had increased from 12 deploys/day to 47 deploys/day, rollback time dropped to under 3 minutes, and configuration drift was eliminated. This is how we did it.
Structuring the GitOps Repository
Repository structure is the first major decision in any GitOps migration and the one most teams get wrong. We evaluated three patterns: mono-repo (all services, all environments), poly-repo (one repo per service), and a hybrid (one config repo per cluster).
For 120 services, the hybrid pattern won. Each Kubernetes cluster has a dedicated config repository. Service teams own their application manifests in their service repository; a promotion pipeline creates PRs against the cluster config repository when a new image is ready. This creates a clear separation between application code and deployment configuration while maintaining auditability.
The Kustomize Overlay Strategy
We used Kustomize rather than Helm for most services — a deliberate choice. Helm charts introduce templating complexity that obscures the actual Kubernetes objects. For the 20% of services with genuine parametric variation across environments, Helm was appropriate. For the other 80%, Kustomize overlays expressing only the actual differences between environments were clearer and easier to review.
The overlay structure is simple: a base/ directory with the complete service definition, and overlays/dev/, overlays/staging/, overlays/prod/ containing only the fields that differ (replica counts, resource limits, image tags, environment-specific secrets references).
Progressive Delivery with Argo Rollouts
The most impactful addition beyond basic GitOps was Argo Rollouts for progressive delivery on tier-1 services. Rather than deploying a new version to 100% of pods simultaneously, we implemented canary deployments: 5% traffic → 20% → 50% → 100%, with automated analysis steps that check error rates and latency at each stage before proceeding.
The first time a canary analysis caught a regression that would have caused a production incident at full rollout, the engineering team's confidence in the platform transformed from sceptical to evangelical. That was month three. After that, adoption of Argo Rollouts spread organically to services we hadn't prioritised.
- Hybrid repo structure (cluster config repos) scales well for 50+ services
- Prefer Kustomize over Helm unless you genuinely need parametric templating
- Automated rollback via ArgoCD self-healing eliminates manual recovery steps
- Add Argo Rollouts for tier-1 services — canary analysis pays for its complexity
- Measure deployment frequency and change failure rate before and after: the numbers justify the migration