CI cache correctness for distributed software delivery

Key Stats

• Warm, clean, lockfile, toolchain, and source cases compared
• Cache key inputs and restored paths recorded
• Artifact provenance checked separately from duration

Key Takeaways

• Treat a cache hit as reuse, not proof.
• Test invalidation for every material input.
• Keep protected-branch assurance explicit.

Cache evidence beyond duration

Compare warm and clean executions while changing the lockfile, toolchain, runner image, generated schema, and a source fixture that should alter output. Inspect resolved versions, restored paths, artifact identity, test selection, and trust scope, not only elapsed time. A cache hit proves reuse but not correctness. A distributed developer can document the invalidation matrix so an offline owner sees why the result remains trustworthy. The repository owner retains the decision about protected checks and acceptable speed tradeoffs.

Cache evidence boundary

If a material input changes while the restored result does not, treat that mismatch as a failure of evidence. Preserve a clean baseline and narrow or remove reuse until the invalidation rule is understood. Speed is a secondary signal to artifact correctness and provenance.

Research question and evidence scope

A fast pipeline is not useful if its cache lets an old dependency or generated artifact stand in for the current change. This research asks whether one cached job proves the same claim as a clean or deliberately invalidated execution. Examine cache key inputs, hit and miss behavior, dependency revision, toolchain version, artifact provenance, and result agreement. The scope is one pipeline boundary, not a claim about every CI provider. A distributed developer can document and test cache behavior; the repository owner decides merge protection, release policy, and exceptions.

Methodology

Run the target job with a warm cache, an empty cache, a changed lockfile, a changed toolchain input, and a changed source file that should affect output. Compare logs, resolved dependencies, generated artifacts, test results, and duration. Inspect whether a cache is immutable, scoped, restorable across branches, or shared across trust boundaries. GitHub Actions caching documentation explains mechanics; NIST SSDF frames verification and provenance; Google SRE material helps evaluate reliability tradeoffs. These sources do not prove a cache key is correct. Correctness must be demonstrated with an invalidation matrix and a clean comparison.

Findings and analysis

A cache key should encode every input whose change can alter the job’s claim, while avoiding needless misses that make the signal unusable. Dependency locks, compiler versions, operating-system images, generated schemas, environment flags, and workspace layout may matter differently for different jobs. A cache hit is evidence of reuse, not evidence that reuse is valid. For a Philippines-based team, written cache inputs prevent an overnight developer from interpreting a suspiciously quick green job as proof. The useful artifact records the key, hit state, restored paths, and why the output remains trustworthy.

Counterevidence and role boundary

Look for stale generated code, an untracked lockfile, cache poisoning across branches, a partial restore, and a test that does not execute the restored artifact. Delete or narrow a cache when the invalidation experiment changes a result. Do not fix flaky CI by broadening cache scope without identifying the missing input. The developer can propose a key and run controlled comparisons. The internal owner decides whether a slower clean check is required for protected changes and whether an exception is acceptable.

Limitations

A finite matrix cannot cover every runner image, parallel race, registry response, or future toolchain change. CI provider documentation can change and may not describe local wrappers or self-hosted runners. Timing improvements also vary with load, so duration alone is weak evidence. State the job, runner, key inputs, artifact paths, dependency revision, invalidation cases, and observation window. Do not infer that one successful clean run proves future cache correctness or that every cache is harmful because one key was incomplete.

Evidence-led conclusion

A cache is justified when a controlled invalidation test shows that every material input changes the key or otherwise invalidates the restored result, and when provenance remains visible. If cached and clean executions disagree, narrow or remove the cache until the difference is understood. The evidence-led practice is to retain a representative clean comparison for protected paths and let owners decide the speed-versus-assurance tradeoff. Explicit cache evidence helps distributed developers work asynchronously without turning CI convenience into an invisible release risk.

Cache decision boundary

A faster job is not a better job if reuse is unexplained. Change the lockfile, toolchain, runner image, generated schema, source fixture, and relevant environment flag one at a time, then compare resolved versions, artifact identity, test selection, and failure behavior as well as duration. For an offshore developer, the written record should name restored paths, trust scope, clean baseline, and the owner of protected-branch policy. A green cached result is evidence of execution, not proof that the current change was exercised.

Evaluating speed against proof

A faster job is not a better job if its reuse boundary is unknown. Examine what the cache restores, whether those paths are trusted, how keys are scoped, and whether a partial restore can look like a complete hit. Change one material input at a time: the lockfile, compiler or runtime, runner image, generated schema, source fixture, and relevant environment flag. Compare not only duration but resolved versions, artifact hashes, test selection, and failure behavior. A green result that came from stale generated output is counterevidence, not a performance win. For a distributed developer working across time zones, cache evidence should be readable without a live explanation. State the key inputs, restored paths, trust boundary, clean baseline, and the owner of the protected-branch policy. The developer can run the matrix and narrow an unsafe cache. The repository owner decides whether a clean check is mandatory for sensitive changes and whether the time tradeoff is acceptable. Do not widen a cache to hide flaky tests or use a duration chart as a reliability claim. The research supports an invalidation matrix and provenance record; it does not establish that one successful run proves future correctness.

Evidence table

Good distributed work is observable at the handoff: the result, evidence, limitations, and next owner are all explicit.

Frequently asked questions

Related Research

• Research library