Explain what a release checklist cannot learn from filenames
Also published in our Blogger archive.
Direct answer
Filename checks are observations about a tree, not observations about execution. A checklist should state both sides: the two filename signals it inspected and the three categories it did not inspect. The explicit not_checked list stops a reader from treating a small local inventory as evidence about runtime behavior, dependency resolution, or deployment configuration.
The edge case is a repository with every expected file but a broken start command. It can pass this filename fixture while failing the missing category. Keeping absence of evidence visible is more honest than adding vague confidence language to a score.
The example does not walk a real project, execute commands, resolve lockfiles, or contact a deployment service. Add those as separate checks with their own evidence contracts. A scanner should never imply it learned a fact merely because a similarly named file exists.
Complete example
checked = ["README filename", "LICENSE filename"]
not_checked = ["runtime behavior", "dependencies", "deployment"]
assert len(checked) == 2
assert "deployment" in not_checked
print("checked=2 not_checked=3")Expected stdout:
checked=2 not_checked=3Sources
Prepared with AI assistance. The example uses synthetic data; its stated limits apply.