Agree the file format before importing records
Before a CSV import, confirm the text encoding, field separator, quoting rules and expected header. Parse a small representative file and compare its fields with known expected values. Then check the full file's structure. Keep the original bytes and reject an uncertain interpretation before it can change live records.
CSV means comma-separated values, but a file named .csv may use semicolons or tabs. An encoding controls how bytes become characters. A delimiter separates fields. These are different settings. A file can decode without an error and still use the wrong delimiter or map values into the wrong columns.
This guide covers the file-reading boundary. The required-fields guide covers business validation after fields have been read. The supplier-update guide covers later matching, calculations and release controls.
Write one small import agreement
Ask the sender for an export specification and an example that contains accents, separator characters and any multiline fields they actually use. Confirm whether there is a header and whether its names and order are stable. Keep that agreement with the import configuration.
| Setting | Record this decision |
|---|---|
| Encoding | For example, UTF-8 with an explicitly agreed treatment of a byte order mark |
| Delimiter and quote character | For example, semicolon fields with double-quoted text |
| Header | Exact expected names, order and duplicate-name rule |
| Record structure | Expected field count; blank-record and multiline-field rules |
| Limits | Permitted file size, record count and field length for this importer |
| Failure owner | Who corrects the export when the agreement is not met? |
Do not infer the complete format from a filename or the spreadsheet application's display. Keep the source bytes unchanged. If a correction is agreed, save it as a separately identified input so that a later reviewer can explain which file was used.
Decode deliberately and keep identifiers as text
Python's Unicode HOWTO explains how encodings translate bytes into text. Use the agreed encoding and a strict error policy. If decoding fails, show the problem and ask for a corrected export or a confirmed encoding. Silently discarding or replacing characters can alter names and identifiers.
A byte order mark, or BOM, can appear at the start of a text file. Python documents the utf-8-sig codec, which skips a leading UTF-8 signature when decoding. Use it when that is the agreed input format. It is not a detector for arbitrary encodings.
Successful decoding alone cannot prove that the sender chose the intended encoding. Include known text in a representative example and compare it. Keep product codes, account references and other identifiers as text through this stage. The identifier 0007 must not become 7 because a spreadsheet guessed its type.
Separate punctuation from number interpretation
A comma inside a price can be a decimal separator in the sender's chosen format. It does not establish which character separates columns. Agree the delimiter first. Apply the separate number-format and currency rules after parsing. Do not globally replace commas or semicolons to make a file appear to fit.
Use a CSV parser instead of splitting lines
A field may contain the separator itself or a line break. RFC 4180 describes comma-separated records, quoted fields and doubled double quotes inside quoted text. It is an informational description of a common format, not proof that every exporter follows one universal dialect. A semicolon export needs its own explicit settings.
Use a real CSV parser with the agreed settings. Python's csv documentation describes configurable dialects and recommends opening file objects with newline='' so that the parser can handle newline behavior. Its normal reader returns text fields unless a conversion option is selected. Do not count physical lines as business records when quoted fields can contain newlines.
A format guess can help investigate an unfamiliar sample, but it should not replace the input agreement. Python describes header inference as a heuristic that can be wrong. An apparently consistent parse still needs exact header and known-value checks before release.
Fictional example: two records in a semicolon file
This example is fictional. An invented stationery exporter supplies a UTF-8 file with a leading BOM, semicolon separators, a three-field header and double-quoted text. The header must be exactly sku, label, note in that order. No live product or supplier file is used.
The file text below excludes the invisible UTF-8 signature. The second note contains a real line break.
sku;label;note
0007;"Café; small";"Check ""blue"" cover"
0008;Plain;"Line one
Line two"
| sku | label | note |
|---|---|---|
| 0007 | Café; small | Check "blue" cover |
| 0008 | Plain | Line one, followed by an embedded line break, then Line two |
The semicolon in the first label belongs to that field. Quoting prevents it from creating an extra column. The double quotes around blue are data, represented with doubled quotes in the file. The newline in the second note belongs to one field, so the parsed result contains two business records even though the source occupies more physical lines.
A comma-delimited interpretation does not match the required header. A new row with four fields fails the three-field agreement. An invalid UTF-8 byte sequence fails strict decoding. Each case should stop before changing a destination. These are proposed import controls; the example does not establish a connected production import.
After parsing, verify required values, duplicates and permitted lengths separately. A structurally correct file can still contain an unknown product or an invalid business value. For larger files, keep work bounded and report failures by stable record identifiers or parser positions without exposing private field contents in public logs.
Check eight cases before allowing the import
- The agreed example produces the exact header and expected field values.
- Accented text and an optional agreed UTF-8 signature are handled correctly.
- Identifiers retain leading zeros and stay separate from numeric values.
- A quoted separator stays within its field.
- An embedded quote and an agreed multiline field retain their intended contents.
- A wrong delimiter, missing header or duplicate header is rejected under the agreement.
- A record with too many or too few fields is reported before any destination update.
- A decoding failure or exceeded file limit leaves the original input available for correction and causes no partial unreviewed import.
Record actual results rather than ticking this list from a screenshot. Test the same upload and parser path that the real importer uses. A sample can show a particular case; it does not prove that the full file or a connected destination is correct.
CSV INPUT AGREEMENT — SYNTHETIC WORKSHEET Export owner and original file identifier: Encoding and signature policy: Delimiter / quote character / embedded-quote rule: Newline and multiline-field behavior: Exact header names and order: Blank and duplicate-header rules: Expected field count per record: Identifiers that must remain text: File / record / field limits: Known example values and parsed record count: Full-file validation and business checks: Failure owner and no-write boundary: Case | expected fields | actual fields | pass/fail/blocked/not run 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
Bring a small redacted file and its intended columns to a free fit call. ZenOut can scope seller operations tools with an explicit import agreement, visible exceptions and a reviewed release step.