#Container Snapshots

In Attack & Defense games, each team gets its own running container per A&D challenge — a box they SSH into, patch, and (try to) defend. When the game ends those containers are torn down. Container Snapshots are this fork's mechanism for preserving the forensic record of what each team did to their box, so organizers can run writeups, settle disputes, and study patches after the event.

This is exclusive to this fork; upstream GZ::CTF has no A&D mode and therefore no snapshotting.

There are two distinct artifacts, captured at different times by different code paths:

The tarball is the heavyweight forensic deliverable (a full filesystem you can docker load and shell into). The manifests are a lightweight timeline of which files changed when, so you can diff a team's box between round X and round Y without storing a tarball per round.

See also /guide/features/attack-defense for the broader A&D model and /config/appsettings for storage configuration.

#The end-of-game tarball

When a game's EndTimeUtc passes, the A&D reconciler (AdContainerManager) walks every team service that still has a live container and, before destroying it, snapshots it — but only if the game's AdAllowSnapshotDownload setting is true and no snapshot was already taken.

The capture (TrySnapshotAsync) does three things, in order:

1. Captures the filesystem diff first via docker diff (InspectChanges) — the writable-layer changes versus the baseline image. This is the "what did the team change" data. It is trimmed to the first 3000 entries and stored as JSON on AdTeamService.SnapshotChanges. A failure here is logged but does not abort the snapshot.
2. Commits the container to an image (docker commit) named ad-snapshot-{serviceId}:{gameId}, with a commit comment recording the team and challenge.
3. Exports + uploads the image with docker save, gzips it (CompressionLevel.Fastest), and writes it to blob storage. The committed local image is then deleted (best-effort) — the tarball in blob storage is the deliverable.

docker diff   →  SnapshotChanges (JSON, ≤3000 entries)
docker commit →  ad-snapshot-{serviceId}:{gameId}
docker save   →  gzip  →  blob  ad-snapshots/{gameId}/{participationId}-{challengeId}.tar.gz

The teardown runs each service under its per-service lock and re-checks the end condition inside the lock, so a snapshot can't race a concurrent self-reset or an extended game.

#Restoring a tarball for forensics

The blob is a standard gzipped docker save archive. To inspect a team's final box:

# Download via the admin endpoint (see "Downloading" below), then:
gunzip -c ad-snapshot-team42-challenge7.tar.gz | docker load
# docker load prints the loaded image ref, e.g. ad-snapshot-13:1
docker run --rm -it ad-snapshot-13:1 /bin/sh

From there you have the team's actual filesystem — their patches, any attacker footholds, dropped files, modified binaries.

#The per-round change manifests

AdSnapshotService is a background service that polls every 30 seconds. For each active game with at least one enabled A&D challenge it finds the latest round, then for every team service with a live container it captures a changed-file manifest.

The manifest is produced by the same diff machinery as the live admin view, ComputeLiveChangesAsync:

• Docker: docker diff (InspectChanges) against the baseline image — precise add / modify / delete (kind 1 / 0 / 2). Ancestor directories are collapsed to leaf paths.
• Kubernetes: an exec'd find / -xdev … -newer /proc/1 -type f in the pod — an mtime heuristic for "files modified since the container started". No add/modify/delete distinction and it misses deletions, so every entry is reported as kind 0 (modified).

Both branches filter out runtime/churn noise so the manifest reflects deliberate team changes, not platform activity. The filtered-out categories are:

flag mount (/flag, /gzctf-flag)
/tmp and /run
/var/log, /var/cache, /var/tmp, /var/run, package caches
/proc, /sys, /dev
Python __pycache__ and *.pyc
directories that only contain a changed file (ancestor dirs)

#Dedup and gating

Manifests are stored deduped: a new AdServiceSnapshot row is written only when the canonicalized changed-file set differs from that service's previous one. So storage tracks distinct states, not ticks — a team that stops touching files stops generating rows.

Two gates keep the scan cheap:

• Per-round: a service is scanned at most once per round (tracked in-memory by highest round number seen), so the expensive scan — on K8s, a whole-rootfs find per pod — doesn't re-run on every 30s poll.
• Bounded parallelism: at most 8 concurrent change scans across all active games, mirroring the SLA checker's bound.

Each row records the round it was captured in (AdRoundId), CapturedAt, and the canonical ManifestJson ([{ "p": path, "k": kind }]).

#Restarting vs. resetting a box

A frequent point of confusion: restarting an A&D container does not preserve its filesystem. There is one restart path, RestartContainerAsync, and it always destroys the container and recreates it from the same challenge image — the box reverts to baseline and gets a new IP. It backs both:

• Player self-reset — POST /api/Game/{id}/Ad/Services/{adTeamServiceId}/Reset. Gated by the per-challenge AdAllowSelfReset flag, the game-wide AdResetCooldownMinutes cooldown, and the game window (only while running).
• Operator force-restart — POST /api/edit/games/{id}/ad/Services/{adTeamServiceId}/Restart (game admin). Bypasses the player cooldown — for when a team's box is wedged and they can't recover it themselves.

Provider note: on K8s a reset/restart still destroys and recreates the pod from the image; there is no in-place state-retaining restart on either provider.

#Downloading snapshots

#Participant download (gated)

Players can pull their own team's end-of-game tarball, but only when all of these hold:

GET /api/Game/{id}/Ad/Services/{adTeamServiceId}/Snapshot   (RequireUser)

• The caller is a member of the team that owns the service (else 403).
• The game has ended (now >= EndTimeUtc) — never mid-game, even if a stale SnapshotBlobKey lingers from a prior, extended game (else 404 "Snapshot is only available after the game ends").
• A snapshot was actually taken — SnapshotBlobKey is set (else 404 "Snapshot not available (game still running, snapshot disabled, or snapshot failed)").
• The blob still exists in storage (else 404 "Snapshot blob is missing — may have been retained-out").

The whole participant path is gated by the game's AdAllowSnapshotDownload setting. That setting is consulted at game end to decide whether to capture the tarball at all — so if it was false when the game ended, no SnapshotBlobKey was ever written and the download 404s.

The response is the gzipped tarball as application/gzip, filename ad-snapshot-team{participationId}-challenge{challengeId}.tar.gz.

#Admin download and inspection (any team)

Game admins are not team-scoped — they can pull any team's snapshot and inspect changes. The admin endpoints live under /api/edit/games/{id}/ad/… (where … below stands for that prefix):

Per-file content (current vs. baseline + unified diff) and the in-browser shell on live boxes round out the admin forensics surface.

#Storage and retention

The end-of-game tarballs live in whatever blob storage the deployment is configured with (the same IBlobStorage used for uploads). Tarball size scales with image size, so large challenge images produce large snapshots — budget storage accordingly.

Retention is governed per game by AdSnapshotRetentionDays:

// Game.cs
public bool AdAllowSnapshotDownload { get; set; } = true;   // capture + expose tarballs?
public int? AdSnapshotRetentionDays { get; set; }            // null = keep forever

Both settings are part of the game definition: they round-trip through game export/import and can be set in a .gzevent manifest (SnapshotRetentionDays) for repo-bound games, or edited in the admin game-info UI.

#Quick reference

• Capture (tarball): automatic, once, at game end — Docker only, gated by AdAllowSnapshotDownload.
• Capture (manifests): automatic, every 30s poll / once-per-round-per-service, deduped — Docker + K8s.
• Player download: GET /api/Game/{id}/Ad/Services/{adTeamServiceId}/Snapshot — own team, post-game, gated.
• Admin: GET /api/edit/games/{id}/ad/Services/{adTeamServiceId}/Snapshot plus …/Snapshot/Changes, …/Snapshots, …/SnapshotDiff — any team.
• Restart/reset: always reverts the box to its image; only snapshots survive a wipe.

Related: /guide/features/attack-defense · /config/appsettings