Detect duplicate CSV headers before mapping fields
Also published in our Blogger archive.
Quick answer
csv.reader returns each parsed record as a list, so it preserves header position before a mapping discards it.
Example
The code counts every raw header and reports one-based positions for name. A mapping layer should stop when this finding exists; choosing the leftmost label only makes an accidental policy invisible.
import csv, io
rows = list(csv.reader(io.StringIO('id,name,name\n1,A,B\n')))
headers = rows[0]
dups = {h: [i + 1 for i, x in enumerate(headers) if x == h] for h in headers if headers.count(h) > 1}
assert dups == {'name': [2, 3]}
print(dups)Expected stdout:
{'name': [2, 3]}Reading the result
A case-insensitive or trimmed duplicate is a different rule. Run that normalization check separately so the report can name both the raw labels and the proposed normalized collision.
Before mapping, also retain the original header row in the finding. That lets a reviewer distinguish two exact name labels from labels that only look identical after a later display transformation.
Do this before DictReader or any dictionary comprehension. Once the repeated header has become a mapping key, the position information needed to resolve it is no longer available.
Sources
- Python csv module documentation
Prepared with AI assistance. The example uses synthetic data; its stated limits apply.