Upgrading

PandoCore ships two things: the admission webhook (cluster-wide, installed once) and the sidecar image (injected into each protected pod). They version independently. This page covers how to upgrade both safely.

# Webhook version
helm list -n pando-system

# Sidecar version running in a specific pod
kubectl get pod <pod> -o jsonpath='{.spec.containers[?(@.name=="pando-sidecar")].image}'

# Latest available version from the portal
curl -s https://portal.pandocore.io/api/version

Upgrade the Admission Webhook

Webhook upgrades are done with helm upgrade. The chart is versioned and pinned in the OCI registry.

# Latest
helm upgrade pando-webhook \
  oci://us-central1-docker.pkg.dev/pandocore-prod/charts/pando-webhook \
  --reuse-values

# Or pin to a specific version
helm upgrade pando-webhook \
  oci://us-central1-docker.pkg.dev/pandocore-prod/charts/pando-webhook \
  --version 0.5.2 \
  --reuse-values

--reuse-values preserves your existing licenseKey, slackWebhookURL, and other overrides so you don't have to re-pass them on every upgrade. If you need to change a value at the same time, use --reuse-values plus additional --set flags.

What happens during a webhook upgrade

1. Helm applies the new chart to the pando-system namespace.
2. The webhook Deployment does a rolling update. The new pod comes up, passes its readiness probe, and the old pod is terminated.
3. Existing protected pods are not affected. The webhook only runs at pod admission time; already-running pods keep running with whatever sidecar version they started with.
4. Any pod that restarts after the upgrade will get the new sidecar image (because the webhook's default sidecar tag is now updated).

Upgrade Sidecars in Protected Pods

Sidecar upgrades happen on pod restart. Restart the deployment and the webhook injects the new sidecar image on each new pod:

kubectl rollout restart deployment/your-app

This is the standard approach. Kubernetes handles the rolling update with your deployment's existing maxSurge/maxUnavailable settings. Zero downtime if your app has more than one replica.

Baseline Considerations

Each sidecar builds its own operational profile on startup. When a pod restarts with a new sidecar version, it runs a fresh learning window against the new process. This means:

• Expect a 30-minute learning window (503s) after each sidecar restart. The pod's main container is ready; the sidecar's /readyz endpoint returns 503 until the profile is built. If you're using the sidecar's readiness for load balancer health, factor this in.
• Profiles are not persisted across upgrades by default. Every new sidecar starts fresh. This is intentional: detection logic evolves between versions and reusing old state would conflict with new scoring.
• Stagger upgrades for critical workloads. Don't restart every protected deployment in the same minute. Rolling them one at a time (or letting Kubernetes do it via rollout restart) keeps enough baselined pods in service to handle traffic.

Version Skew

Sidecar and webhook versions don't need to match exactly, but the webhook's chart version defines the default sidecar image tag it injects. Within a major version, mixing sidecar versions across pods is supported. Across major versions, upgrade the webhook first, then restart workloads to pick up the new sidecar.

Rollback

If an upgrade causes problems, roll back the webhook first, then restart affected workloads:

# Roll back the most recent webhook upgrade
helm rollback pando-webhook -n pando-system

# Restart a workload to pick up the previous sidecar version
kubectl rollout restart deployment/your-app

If the issue is severe enough that you need to unblock pod creation immediately, see Emergency Procedures for disabling PandoCore cluster-wide.

Upgrade Checklist

• Read the release notes for the version you're targeting
• Upgrade in staging or a test cluster first
• helm upgrade pando-webhook ... --reuse-values
• Verify the new webhook pod reaches ready state
• Restart a single non-critical workload and confirm the new sidecar injects and learns cleanly
• Watch the portal for unexpected spikes in drift alerts on that workload
• Roll out to remaining workloads in batches, most critical last