Limit an lru_cache for bounded input varieties
Also published in our Blogger archive.
Limit an lru_cache for bounded input varieties
By Batu
An unbounded memoization cache can retain one result for every distinct argument tuple. If inputs can keep varying, that means memory use can keep increasing. functools.lru_cache(maxsize=2) instead retains at most two recent keys. This recipe caches a deterministic label for a two-letter language code and records real function-body executions in computed.
The first calls for "en" and "tr" fill the cache. Repeating "en" marks it as recently used. When "de" arrives, "tr" is the least recently used entry and is evicted. Calling "tr" again executes the function body a second time, which is visible as a second compute:tr line. cache_info().currsize is asserted to be no more than two; it is an observation about this wrapper, while the trace demonstrates the particular access sequence.
maxsize=2 bounds the number of cached entries, not the size of each returned object. A few very large results can still consume substantial memory. Arguments must be hashable, so passing a list raises TypeError before caching. Caching also does not make an impure function safe: a function reading time, random state, or mutable globals can return a stale value for an otherwise identical key.
AI assistance was used to draft this article.
from functools import lru_cache
computed = []
@lru_cache(maxsize=2)
def language_label(code: str) -> str:
computed.append(code)
return code.upper()
assert language_label("en") == "EN"
assert language_label("tr") == "TR"
assert language_label("en") == "EN"
assert language_label("de") == "DE"
assert language_label("tr") == "TR"
assert computed == ["en", "tr", "de", "tr"]
assert language_label.cache_info().currsize <= 2
for code in computed:
print(f"compute:{code}")Expected stdout:
compute:en
compute:tr
compute:de
compute:tr