Limit a JSON fixture before decoding from StringIO

By Batu · English technical notes

Also published in our primary archive.

Limit a JSON fixture before decoding from StringIO

To limit a JSON fixture before decoding from StringIO, call read(limit + 1), reject text longer than limit, and invoke json.loads only after that check. For a text stream, this is a character limit: StringIO.read(65) requests up to 65 characters, not 65 bytes.

The fixture is a valid 65-character JSON string and the application policy is 64 characters. The contrasting direct path calls json.load(stream). Its tracking stream records -1, the default unbounded read argument, and the decoder successfully receives the complete fixture. This observation is specific to this synthetic StringIO subclass; it is not a throughput or memory measurement.

The corrected loader reads exactly limit + 1 characters. One extra character is enough to establish that the policy is exceeded, so it raises before calling the injected decoder. The assertions verify both boundary facts: the bounded read was 65, and decoder calls remained zero. A real application still needs to choose a limit appropriate to its own data contract, and this pre-read approach does not impose limits on nesting or individual JSON values.

The json.load documentation specifies a read-supporting file-like input, while io.StringIO is an in-memory text stream. No newer APIs are used; this example runs on Python 3.6+.

AI assistance disclosure: this article was drafted with AI assistance and runs entirely on a synthetic in-memory stream.

import io
import json


class TrackingStringIO(io.StringIO):
    def __init__(self, text):
        super().__init__(text)
        self.read_sizes = []

    def read(self, size=-1):
        self.read_sizes.append(size)
        return super().read(size)


payload = '"' + ("x" * 63) + '"'
regular = TrackingStringIO(payload)
value = json.load(regular)

assert len(payload) == 65
assert value == "x" * 63
assert regular.read_sizes == [-1]
print("unbounded read size:", regular.read_sizes[0])


def load_limited(stream, limit, decoder):
    text = stream.read(limit + 1)
    if len(text) > limit:
        raise ValueError("JSON input exceeds 64 characters")
    return decoder(text)


calls = 0


def counting_loads(text):
    global calls
    calls += 1
    return json.loads(text)


bounded = TrackingStringIO(payload)
try:
    load_limited(bounded, 64, counting_loads)
except ValueError as error:
    print("bounded:", error)

assert bounded.read_sizes == [65]
assert calls == 0
print("decoder calls:", calls)

unbounded read size: -1
bounded: JSON input exceeds 64 characters
decoder calls: 0