Store a last-error summary without storing sensitive payloads

By Batu · English technical notes

Also published in our Blogger archive.

Direct answer

A last-error field should answer what failed without becoming a second copy of the request that failed. Store a short normalized summary, not a traceback, headers, body, or raw exception object. The example’s summarize function replaces one token-shaped field before the INSERT, then asserts that the secret literal cannot appear in the value selected from SQLite.

The ordering matters: redaction happens before persistence. Redacting only when rendering a dashboard leaves the database with material a later export or debug query can disclose. The 120-character bound also prevents one exceptional response from turning a diagnostic column into unbounded storage.

The edge case here is deliberately narrow: a whitespace-delimited token= value. It does not recognize every credential format, nested JSON, or secrets embedded in arbitrary prose. A production system needs a documented allowlist of fields it retains and tests for each connector-specific failure shape. SQLite simply stores the string supplied to it; it cannot decide which part of an error is safe to retain.

Complete example

import re
import sqlite3


def summarize(message: str) -> str:
    redacted = re.sub(r"token=[^\s]+", "token=[redacted]", message)
    return redacted[:120]


connection = sqlite3.connect(":memory:")
connection.execute("CREATE TABLE job_errors (job_id INTEGER, summary TEXT)")
raw = "request failed token=secret-12345 because endpoint refused input"
summary = summarize(raw)
assert "secret-12345" not in summary
assert summary == "request failed token=[redacted] because endpoint refused input"
connection.execute("INSERT INTO job_errors VALUES (?, ?)", (7, summary))
stored = connection.execute("SELECT summary FROM job_errors").fetchone()[0]
assert stored == summary
print(stored)

Expected stdout:

request failed token=[redacted] because endpoint refused input

Sources

- Official API documentation

Prepared with AI assistance. The example uses synthetic data; its stated limits apply.