Use newline handling correctly with Python csv readers

By Batu · English technical notes

Also published in our Blogger archive.

Quick answer

The csv documentation says file objects should be opened with newline="" so csv can handle newline characters itself.

Example

The fixture contains a CRLF inside a quoted field. Passing newline="" to StringIO lets reader return one record with that embedded CRLF still in the note. A physical line break is therefore not automatically a record boundary.

import csv, io
text = 'id,note\r\n1,"first\r\nsecond"\r\n'
rows = list(csv.reader(io.StringIO(text, newline='')))
assert rows == [['id', 'note'], ['1', 'first\r\nsecond']]
print(len(rows), repr(rows[1][1]))

Expected stdout:

2 'first\r\nsecond'

Reading the result

Use this opening rule with real files as well. It does not validate quotes, widths, or encodings; those are separate preflight decisions.

Test the exact newline forms an exporter emits, including CRLF and LF. The parser setting protects record interpretation, but a downstream application may still have its own newline expectations.

The assertion includes the CRLF characters in the resulting field. That matters because a validator should not accidentally normalize the content while it is proving record structure.

Sources

- Python csv module documentation

Prepared with AI assistance. The example uses synthetic data; its stated limits apply.