Design ignore lists for local project scans

By Batu · English technical notes

Also published in our Blogger archive.

Direct answer

To exclude generated directories with os.walk, edit the supplied directory list in place before the next iteration. The slice assignment changes the list that the walker will consult. Assigning a new local variable would leave the walker's original list unchanged and would not prune traversal.

The fixture creates three files under src, build, and __pycache__. Comparing each directory name against IGNORED prevents traversal into the latter two, leaving src/app.py in the result. Filtering only the final file list would hide findings after the work had already been done.

These are exact, case-sensitive basenames at every visited level. A legitimate source directory named build would also be excluded, while build-tools would remain eligible. Keep that tradeoff visible in the scanner's documentation. Add a separate policy for root-relative exclusions if a name should be ignored in only one location. This example asserts the selected file, not the safety or quality of files in skipped directories.

Complete example

import os
from pathlib import Path
from tempfile import TemporaryDirectory

IGNORED = {"__pycache__", "build"}

with TemporaryDirectory() as directory:
    root = Path(directory)
    for name in ("src/app.py", "build/app.js", "__pycache__/app.pyc"):
        path = root / name
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_text("x", encoding="utf-8")
    found = []
    for current, directories, names in os.walk(root):
        directories[:] = [name for name in directories if name not in IGNORED]
        found.extend((Path(current) / name).relative_to(root).as_posix() for name in names)
    assert found == ["src/app.py"]
    print("included=src/app.py")

Expected stdout (for a platform supporting the demonstrated operation):

included=src/app.py

Sources

- os.walk

Prepared with AI assistance. The example uses synthetic data; its stated limits apply.