Count CSV records when quoted fields span physical lines

By Batu · English technical notes

Also published in our Blogger archive.

Quick answer

csv.reader iterates logical records, while line_num records physical lines consumed from the input.

Example

After skipping the header, the fixture enumerates two parsed records. One quoted note spans two physical lines, so reader.line_num ends at four. The printed values show why a record number must not be labeled as a source line number.

import csv, io
text = 'id,note\n1,"two\nphysical lines"\n2,one\n'
reader = csv.reader(io.StringIO(text, newline=''))
next(reader)
records = list(enumerate(reader, 1))
assert len(records) == 2 and reader.line_num == 4
print(f'records={len(records)} physical_lines={reader.line_num}')

Expected stdout:

records=2 physical_lines=4

Reading the result

For precise operator diagnostics, capture parser position at the time of each finding. This short example only demonstrates the distinction after all records have been consumed.

A report can carry both record and physical_line fields when an operator needs each. Never use a bare number whose coordinate system the reader must guess.

The logical enumeration begins at one after the header, which is a readable record coordinate for a data report. It is intentionally separate from reader.line_num.

Sources

- Python csv module documentation

Prepared with AI assistance. The example uses synthetic data; its stated limits apply.