Identifiers with zeroes
Fictional example: 0012 and 12 remain distinct strings here. Convert to numbers only if the column definition allows it; review matches that depended on the zero.
In-depth practical guides
Diagnose a CSV file before using it
Check structure, missing values, duplicates and keys without changing the original file.
Updated :
AI-generated illustration · Reports and charts on a data review desk. Illustrative scene with no real data.
Can this table support the intended question?
A file that opens in a spreadsheet can lose leading zeroes, change dates or hide irregular records. Work on a copy and describe issues before correcting them. RFC 4180 documents a common CSV format; the W3C CSVW model describes delimiters, types, null values and keys. The profile below checks observed structure, not the truth of the data.
Keep the original
Record origin, download date, licence and file hash. Keep received files separate from transformations. Opening and saving in a spreadsheet can already change identifiers.
Set the dialect
Choose comma, semicolon or tab explicitly. Quoted fields can contain delimiters or newlines, so counting physical lines is insufficient. This tool requires UTF-8 and a header with distinct names.
Profile the columns
Count blank cells, distinct values and surrounding whitespace. Blank does not necessarily mean zero. NA, null and - remain values unless a dictionary defines them as missing.
Test a composite key and proposed cleaning
Use several columns for repeated observations, such as station and date. The tool checks tuples of exact values without concatenating them. A blank component makes the key incomplete. Fill rate is not a validity rate; with no records it is not calculable. Before trimming whitespace, inspect collisions: A and A with surrounding spaces would become identical.
Decide before correcting
An identical record and a repeated key are different findings. Establish the grain: company, establishment or dated observation. Document each correction and compare counts before and after.
Profile a CSV file
Local processing in your browser: the file is never uploaded. UTF-8, up to 2 MB, 10,000 data records and 200 columns. The export includes file and column names but no cell values.
Situations and decisions
Typical situations for preparing a check. They do not describe completed assignments or actual observations.
Repeated key, different observations
In an illustrative table, a station appears every day. Station code alone is not unique; station + date may be appropriate. A repeated key does not justify deleting an observation.
Record to retain
- Origin, licence and original hash
- Dialect and missing-value dictionary
- Grain, selected key and observed issues
- Transformation rules and compared counts
Official references
Related source profiles
Related protocols
Checks before reaching a conclusion
- Do identifiers retain their zeroes?
- Are blanks distinguished from zeroes?
- Does the grain justify the key?
- Are corrections traceable and reversible?
An exportable JSON profile without cell values, with a decision on the transformations needed.
Frequently asked questions
Should all duplicates be deleted?
No. Check grain, versions and columns before deduplicating. Identical records may represent separate events if an event identifier is missing.
Can an empty column be removed?
Only after checking the schema and intended use. It may be required by an import or indicate an uncollected variable. Record the decision in the dictionary.