Skip to main content
Docs / Getting Started

Getting Started

Two commands to install, and nothing to label or restart.

PandoCore ships as a node sensor: one DaemonSet pod per node that watches the workloads already running there, plus a cluster controller that owns delivery of evidence. No application code changes, no SDK, no rules to write, and nothing injected into your pods.

Prerequisites
  • Kubernetes 1.25+ with kubectl configured
  • Helm 3.0+
  • Nodes you control. The sensor needs hostPID and a host path, so managed node-less runtimes have no install path: see the note below
  • A PandoCore license key from portal.pandocore.io. Signup issues one immediately
GKE Autopilot and EKS Fargate are not supported

The sensor runs as a hostPID DaemonSet with a host path mount. Neither is permitted on GKE Autopilot or EKS Fargate, so there is no install path on either. Standard GKE is the configuration this release was validated on.

Install

Two charts, controller first. The controller is what the sensor delivers evidence to, so installing it second leaves the sensor with nowhere to send anything.

# 1. The controller: receives evidence and owns every outbound destination
helm install pando-controller \
  oci://us-central1-docker.pkg.dev/pandocore-prod/charts/pando-controller \
  --version 2.0.0 \
  --namespace pando-system --create-namespace \
  --set license.key="YOUR_LICENSE_KEY"

# 2. The node sensor: one pod per node
helm install pando-node-sensor \
  oci://us-central1-docker.pkg.dev/pandocore-prod/charts/pando-node-sensor \
  --version 2.0.0 \
  --namespace pando-system \
  --set license.key="YOUR_LICENSE_KEY"

That is the whole install. There is no third step: the sensor protects the workloads on its node without being told which ones, so no labelling, patching or restarting is involved. Your portal dashboard carries both lines pre-filled with your license key and the current chart version.

--version is not optional

Installing an OCI chart without --version fails outright rather than resolving to the newest release. Take the version from your portal dashboard, or from helm show chart against the same reference.

Keeping the license out of your values file

Both charts accept an externally managed Secret instead of an inline key, which is what you want wherever the values file is committed:

kubectl create secret generic pando-license \
  -n pando-system --from-literal=license-key="YOUR_LICENSE_KEY"

helm install pando-node-sensor ... \
  --set license.existingSecret.name=pando-license

Either way the pod reads it through a secretKeyRef, so the credential is never a literal in the rendered pod spec. Create the Secret with --from-literal rather than from a file: the key is not trimmed, and a trailing newline fails validation.

What Just Happened

The DaemonSet placed one sensor pod on each schedulable node. Each sensor watches the pods on its own node, opt-out, and runs a short learning window per workload (30 minutes, and at least 900 samples, whichever is later) to build an operational profile. Deterministic checks arm when learning completes; the behavioural checks that depend on longer baselines keep warming for hours after that.

Because the model is opt-out, the question is no longer which pods are watched but which are not. Four classes are excluded whatever the configuration says:

metadata:
  labels:
    pandocore.io/protect: "false"
The opt-out string is exact

The comparison is byte-for-byte against "false", with no trimming and no case folding. "False", "0" and "no" do not opt a pod out. The label's sense is the reverse of the deprecated webhook's, where "true" opted a pod in; that old value now has no effect at all.

There is a per-node coverage cap

Each sensor runs at most 70 monitored pods per node by default (maxEngines). Pods beyond the cap get no engine, and they are shed in a stable order, so the same pods stay unwatched across restarts rather than rotating. This is reported rather than silent: the gauge pando_node_sensor_pods_unmonitored{reason="cap"} counts them, and the chart ships a PrometheusRule that alerts on it. Treat a non-zero value as a coverage gap, not a capacity hint.

Verify It's Working

# One sensor pod per node, all Ready
kubectl get pods -n pando-system -l app.kubernetes.io/name=pando-node-sensor -o wide

# Follow a sensor's logs
kubectl logs -f -n pando-system <sensor-pod>

# Readiness: 503 during learning and while refusing, 200 once armed
kubectl exec -n pando-system <sensor-pod> -- wget -qO- http://localhost:9090/readyz

# What this sensor currently sees
kubectl exec -n pando-system <sensor-pod> -- wget -qO- http://localhost:9090/status
Select on the component label, not the protect label

The sensor's own pod carries pandocore.io/protect: "false", because it excludes itself. A Prometheus job or kubectl selector written against protect=true matches nothing and reports a clean scrape of nothing, which looks identical to healthy. Select on app.kubernetes.io/name=pando-node-sensor.

/healthz returns 200 even while the sensor is refusing to monitor, deliberately, so the kubelet does not crash-loop the refusal logs away. /readyz is the one that gates. If pods will not start or the logs show license errors, see Troubleshooting.

Progressive Rollout

Everything above gets you to installed and alerting. The sections below go from there to production, in the order most teams adopt them.

1. Let it respond

A free license detects and alerts, and never contains. The response ceiling is a property of the license, so raising it is a billing change rather than a configuration change: on a paid license the sensor may apply containment, and the install above starts doing so without being reinstalled.

Below that ceiling, mode is set per workload with an annotation:

kubectl annotate deployment your-app pandocore.io/mode=isolate --overwrite
kubectl rollout restart deployment/your-app

The sensor reads a pod's annotations when it starts watching that pod, so the restart is what applies the change to workloads already running. The annotation can only narrow what the license already permits, never widen it, and a value it cannot parse is warned about by name and falls back to the node-wide default rather than leaving the pod unwatched. See Operating Modes for the ladder and Pricing for what each license permits.

What containment does

PandoCore writes a Kubernetes NetworkPolicy that denies the pod's egress except DNS, holding the workload in place without restarting it. The right durable response for stateful workloads where a restart is expensive. If the behaviour subsides, the policy is removed again.

The sensor applies and removes that policy from outside the pod, which is why the policy needs no API-server exception and why ingress can be denied as well. The bidirectional half of that depends on your CNI permitting host-to-pod traffic regardless of policy, which Cilium and therefore GKE Dataplane V2 guarantee; on other CNIs treat containment as egress-only.

2. Point evidence where you want it

Every detection emits a structured, signed JSON evidence record. The controller owns delivery, so SIEM, Slack and webhook destinations are configured once on the controller rather than per workload, and no workload pod ever holds a third-party credential. See Controller for the destination settings and Evidence Format for the schema.

3. Manage behaviour declaratively

For anything beyond a handful of namespaces, use the PandoCorePolicy CRD. One YAML file in your GitOps repo controls mode, sensitivity and alert destinations per namespace. See CRD Policy for the schema.

Per-namespace integrations need the mirror

Integrations declared in a PandoCorePolicy reach the controller through the destinations mirror, which the sensor chart installs by default. If you set mirror.enabled=false, nothing writes the destinations Secret and those per-namespace integrations will not take effect. The controller's own destinations are unaffected.

4. Decide what is out of scope

Eleven namespaces are excluded by default, covering kube-system and the GKE managed namespaces. To protect one of them anyway, add it to includedNamespaces.

helm upgrade pando-node-sensor ... \
  --set 'includedNamespaces={my-platform-namespace}'
Two sharp edges in namespace scoping

Do not list pando-system in includedNamespaces. The opt-in is read before the exclusion rules and overrides the own-component exclusion as well as the namespace list, so listing it puts PandoCore's own pods back under monitoring. An early soak run in that configuration took 67 false positives in six minutes, 34 of them from the controller pod.

Setting excludedNamespaces: [] does not mean "exclude nothing". An empty list is read as unconfigured and falls back to the built-in eleven. Use includedNamespaces to protect a default-excluded namespace; emptying the exclusion list will not do it.

5. Tune the cadence, carefully

The sensor samples every 1000 ms by default. The floor is 250 ms, and a value below it is refused by name at startup rather than quietly clamped. Shortening the interval costs roughly four times as much CPU at 250 ms as at the default, so lower it only alongside a larger request.

helm upgrade pando-node-sensor ... --set sampleIntervalMs=1000

6. Alert on the sensor itself

The chart ships PrometheusRules covering the conditions that a sensor cannot report on its own behalf: the series going absent, a refusal at startup, shedding at the cap, and a cadence that has stretched past its configured interval. Absence is the one condition no self-report can distinguish from healthy, which is why it is alerted on from outside. See Production Hardening.

What This Costs Per Node

The sensor requests 250m CPU and 256Mi per node and is limited to 2 cores and 1Gi. One pod per node, whatever that node is running.

Measured warm and enforcing on the shipping build, across two runs: 130.4 mCPU at the p95 with a 106.9 MiB working set over a 24-hour warm window, and 127.3 mCPU with 110.6 MiB over a 34-hour run. The working set stayed flat in both, so there is no leak to grow into the limit, and there was no CFS throttling, no OOMKill and no sensor restart. The request is sized for the 70-pod cap rather than for the engine counts these runs carried.

The install refuses nodes that cannot fit the sensor

A Helm preflight check compares the sensor's CPU request against each node's allocatable CPU and fails the install rather than letting pods sit Pending on nodes that will never fit them, because a green install that leaves a node unwatched is the failure it exists to catch. It runs only against a live cluster, so helm template and --dry-run=client skip it, as does a credential that cannot list nodes. Bypass it deliberately with --set preflight.nodeCapacity=false.

Quick Reference

Task Command
List sensors kubectl get pods -n pando-system -l app.kubernetes.io/name=pando-node-sensor -o wide
View sensor logs kubectl logs -n pando-system <sensor-pod>
Check readiness kubectl exec -n pando-system <sensor-pod> -- wget -qO- http://localhost:9090/readyz
View metrics kubectl exec -n pando-system <sensor-pod> -- wget -qO- http://localhost:9090/metrics
Check coverage gaps pando_node_sensor_pods_unmonitored in Prometheus
Set a workload's mode kubectl annotate deployment <name> pandocore.io/mode=isolate --overwrite
Opt a workload out kubectl label pod <name> pandocore.io/protect=false

Next Steps