From Traditional Deployments to Git-Driven Operations
Civica Training Program
Your team has decided to adopt GitOps. Here's the journey from zero to production...
Your team manages 12 microservices across dev, staging, and production on AKS. Deployments are manual, error-prone, and nobody knows what's actually running in production. Last week, a rogue kubectl apply brought down the payment service for 45 minutes.
How do we make deployments repeatable, auditable, and recoverable? How do we make the cluster's state as trustworthy as our codebase?
The answer: GitOps.
GitOps is an operational framework that applies DevOps best practices for infrastructure automation using Git as the single source of truth.
GitOps uses Git repositories as the source of truth for defining the desired state of infrastructure and applications. An automated process ensures the actual state matches the desired state declared in Git.
The term "GitOps" was coined by Weaveworks in 2017. It evolved from their experience running Kubernetes in production and wanting to apply Git workflows to operations.
Now an industry-standard practice endorsed by the CNCF.
"If it's not in Git, it doesn't exist. If it's in Git, it should be running."
Defined by the OpenGitOps project (CNCF Sandbox).
A system managed by GitOps must have its desired state expressed declaratively. You describe what you want, not how to get there.
Desired state is stored in a way that enforces immutability, versioning, and a complete history. Git provides this naturally.
Software agents automatically pull the desired state declarations from the source. No manual pushes required.
Software agents continuously observe actual system state and attempt to apply the desired state. Drift is detected and corrected.
GitOps brings software engineering practices to infrastructure management.
Every change is a Git commit with author, timestamp, and message. Compliance built in.
Revert a deployment with git revert. The agent reconciles automatically.
If someone manually changes the cluster, the agent detects drift and corrects it.
Cluster credentials stay in the cluster. No CI/CD pipeline needs direct cluster access.
All environments are managed identically. Same process for dev, staging, and prod.
Developers use familiar Git workflows: PRs, reviews, and merges to deploy.
Understanding where GitOps diverges from traditional deployment pipelines.
| Aspect | Traditional CI/CD | GitOps |
|---|---|---|
| Deployment trigger | Pipeline pushes to cluster | Agent pulls from Git |
| Source of truth | CI/CD pipeline config | Git repository |
| Cluster access | CI needs kubeconfig / credentials | Agent runs inside cluster |
| Drift detection | None (fire-and-forget) | Continuous reconciliation |
| Rollback | Re-run old pipeline or manual | git revert |
| Audit trail | Pipeline logs (often ephemeral) | Git history (permanent) |
| Multi-cluster | Complex pipeline branching | One repo per cluster / overlay |
Test your understanding of GitOps fundamentals.
The story of how a simple idea transformed operations.
kubectl edit and never trackedgit log tells you who changed what, when, and whygit clone and they see the entire systemFollowing a change from developer intent to production reality.
Two fundamentally different approaches to deploying changes.
kubectl apply or helm upgradeRisk: If someone runs kubectl edit in production, the pipeline doesn't know or care.
Benefit: The cluster heals itself. Git is always the authority.
Seeing the architectural difference.
A four-phase continuous loop.
Developer merges a change to the GitOps repo. This could be a new image tag, a config change, or a new resource.
The GitOps agent detects the new commit by polling the repository (typically every 3 minutes) or via a webhook.
The agent compares Git state with cluster state. If there's a difference, it applies the changes to the cluster.
The agent verifies the resources are healthy (pods running, services available). Reports status back to the UI.
This loop runs continuously. It's not a one-time deployment — it's ongoing reconciliation.
What happens when someone makes a manual change?
It's 2 AM. An on-call engineer directly scales a deployment from 3 to 5 replicas using kubectl scale to handle a traffic spike. They forget to update the Git repo.
Push vs Pull models and GitOps workflow.
The ecosystem of tools that implement GitOps principles.
CI/CD + GitOps for Kubernetes. Opinionated, full-stack platform.
GitOps at scale for multi-cluster management.
Enterprise GitOps with built-in CI and ArgoCD integration.
Both are CNCF graduated — choosing between them depends on your needs.
| Feature | ArgoCD | Flux CD |
|---|---|---|
| Web UI | Rich, built-in UI | Third-party (Weave GitOps) |
| CLI | Full-featured CLI | Full-featured CLI |
| Multi-tenancy | Projects with RBAC | Namespace-scoped controllers |
| Templating | Helm, Kustomize, Jsonnet | Helm, Kustomize |
| SSO | Built-in OIDC/SAML | No built-in auth |
| Progressive delivery | Argo Rollouts (separate) | Flagger (integrated) |
| Architecture | Centralised controller | Distributed controllers |
| Learning curve | Moderate (UI helps) | Steeper (CLI-only) |
Making the right choice for Civica's AKS-based platform.
ArgoCD's built-in OIDC support integrates natively with Azure AD for SSO. Map Azure AD groups to ArgoCD RBAC roles — no extra tools needed.
The ArgoCD UI provides real-time application topology, sync status, and health. Non-CLI users (managers, auditors) can see what's deployed.
ArgoCD Projects allow team isolation. Each team sees only their applications. RBAC controls who can sync, override, or delete.
Argo Rollouts for canary/blue-green deployments, Argo Events for event-driven workflows, and a massive community with extensive documentation.
A critical decision: how do you organise your Git repositories?
Your application source code (Go, Java, .NET) and your Kubernetes manifests should live in separate repositories. This separation allows independent versioning, different access controls, and cleaner Git histories.
app-service/ src/ tests/ Dockerfile .github/workflows/ci.yaml
Contains source code, Dockerfile, and CI pipeline. Outputs a container image.
gitops-config/
apps/
app-service/
base/
overlays/
dev/
staging/
prod/
Contains Kubernetes manifests. ArgoCD watches this repo.
Two strategies for organising your GitOps configuration.
All application configs in a single repository.
gitops-config/
apps/
service-a/
service-b/
service-c/
infra/
cert-manager/
ingress-nginx/
Each team or application owns its own config repo.
team-payments-config/ payment-api/ payment-worker/ team-platform-config/ cert-manager/ ingress-nginx/
A balanced approach that works well for most organisations.
gitops-config/ # Cluster infrastructure components infra/ cert-manager/ external-secrets/ ingress-nginx/ monitoring/ # Application deployments apps/ payment-service/ base/ deployment.yaml service.yaml kustomization.yaml overlays/ dev/ kustomization.yaml staging/ kustomization.yaml prod/ kustomization.yaml order-service/ base/ overlays/ # ArgoCD application definitions argocd/ apps.yaml # App of Apps projects.yaml # ArgoCD Projects
Repository strategies and tool choices.
How changes flow from dev to staging to production.
Promote by merging dev branch into staging into prod. Simple but can lead to merge conflicts.
Use Kustomize overlays in directories. Promote by updating the overlay. Recommended approach.
CI and GitOps are complementary, not competing.
CI handles building and testing. GitOps handles deploying and maintaining.
How the image tag update actually happens in the GitOps repo.
# GitHub Actions example
- name: Update image tag
run: |
cd gitops-config
kustomize edit set image \
myapp=myacr.azurecr.io/myapp:$TAG
git commit -am "chore: bump myapp to $TAG"
git push
Most common. CI has write access to the GitOps repo.
# Annotation on ArgoCD Application
annotations:
argocd-image-updater.argoproj.io/
image-list: myapp=myacr.azurecr.io/myapp
argocd-image-updater.argoproj.io/
myapp.update-strategy: semver
ArgoCD watches the registry and auto-updates. Write-back to Git.
One of the most powerful benefits of GitOps.
You deployed version 2.3.0 of the payment service. It has a critical bug causing 500 errors. You need to roll back to 2.2.9 immediately.
# Find the right version... kubectl rollout undo deployment/payment # Or re-run a pipeline... # Or find the old Helm values... # Hope nothing else changed...
Error-prone, stressful, and not audited.
# One command: git revert HEAD git push # ArgoCD auto-syncs # Payment service is back to 2.2.9 # The revert is in Git history forever
Clean, audited, and stress-free.
Reducing the attack surface of your deployment pipeline.
With GitOps, the blast radius of a compromised CI pipeline is limited — it can't directly modify the cluster.
GitOps is powerful, but it's not magic. Be aware of these challenges.
We've covered the "why" — now it's time for the "how".
kubectl configured and workinghelm CLI installed (v3+)az) authenticatedBy the end of this module, you'll have a fully operational GitOps workflow deploying to AKS with ArgoCD.
Key takeaways from GitOps Fundamentals.
Git is the single source of truth for your desired cluster state. An agent ensures reality matches that truth.
Declarative, Versioned & Immutable, Pulled Automatically, Continuously Reconciled.
The agent inside the cluster pulls state from Git. No external system needs cluster access.
Our tool of choice: rich UI, RBAC, SSO, and excellent Kubernetes integration.
App code and K8s config live in separate repos. Kustomize overlays manage environments.
Reduced attack surface. CI can't directly access clusters. Everything is audited in Git.
GitOps Fundamentals
Next up: Installing ArgoCD on AKS