Signing Container Images With Sigstore

This post started as a note to my future self. Every few months I relearn the same lessons about dependency pinning, so this time I'm writing them down.

The setup was ordinary: Sigstore at the edges, Wireshark doing most of the heavy lifting, and a handful of scripts nobody wanted to own. It worked well until usage roughly doubled over a single quarter.

How we got here

Most of the complexity was accidental. Over the years we'd layered three different approaches to dependency pinning on top of each other, and each one made sense at the time.

The first symptom was subtle. Nothing failed outright, but dependency pinning kept drifting in the wrong direction, and every fix seemed to make the next one harder.

The fix

We started by writing down the one rule we actually cared about, then deleted every bit of code that didn't serve it. It's not glamorous, but it works.

Here's a trimmed-down version of what we ended up with:

cosign sign --yes ghcr.io/acme/api@sha256:3f1c...
cosign verify \
  --certificate-identity-regexp 'https://github.com/acme/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/acme/api@sha256:3f1c...

Nothing about this is clever, and that's the point. Anyone on the team can read it in a minute and know exactly what it does.

A few things that helped along the way:

• Make the default the safe choice, not just the documented one.
• Write the migration guide before writing the migration.
• Agree on what "done" means before the first pull request.

Trade-offs

None of this is free. The new approach is a bit more verbose, and newcomers occasionally ask why we don't just use OWASP ZAP for everything. That's a fair question with a boring answer: we tried, and it made dependency pinning harder to reason about.

There are cases where this falls apart. If your team is very small or your traffic is very spiky, the extra moving parts may not pay for themselves.

Wrapping up

That's it for now. Next time I'll dig into session management, which turned out to be closely related.