Project

General

Profile

Actions

Defect #41434

open

CSV import fails with error for quoted fields containing newlines and CRLF line endings

Added by Go MAEDA about 1 month ago. Updated about 4 hours ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Importers
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Affected version:

Description

When you attempt to import the attached CSV file (test.csv), the following error message is displayed:

The file is not a CSV file or does not match the settings below (Unquoted fields do not allow new line <"\r\n"> in line 1.)

This error occurs when the CSV file contains values with newline characters ("\n") inside double-quoted fields, and the lines end with DOS-style line endings ("\r\n"). If the file uses UNIX-style line endings (\n), the error does not occur.

This type of CSV file can be generated by Microsoft Excel when cells contain line breaks.


Files

test.csv (62 Bytes) test.csv Go MAEDA, 2024-10-04 23:51
clipboard-202410050652-xaakj.png (131 KB) clipboard-202410050652-xaakj.png Go MAEDA, 2024-10-04 23:52
spreadsheet.png (16.7 KB) spreadsheet.png Go MAEDA, 2024-10-04 23:54
41434-csv-crlf.patch (1.17 KB) 41434-csv-crlf.patch Go MAEDA, 2024-11-04 06:07
Actions #1

Updated by Go MAEDA 28 days ago

I found that the error occurs only when quoted values that contain "\n" exist in the first line.

redmine-app(prod)> CSV.parse(%Q|FieldName1,"Field\nName2"\r\nValue1,"Value2"|)
/Users/maeda/redmines/gems/ruby/3.3.0/gems/irb-1.14.1/lib/irb.rb:1260:in `full_message': Any value after quoted field isn't allowed in line 1. (CSV::MalformedCSVError)

No problem when it exists in the second line or later.

redmine-app(prod)> CSV.parse(%Q|FieldName1,"FieldName2"\r\nValue1,"Value\n2"|)
=> [["FieldName1", "FieldName2"], ["Value1", "Value\n2"]]

Setting the row_sep option also avoids the error.

redmine-app(prod)> CSV.parse(%Q|FieldName1,"Field\nName2"\r\nValue1,"Value2"|,  row_sep: "\r\n")
=> [["FieldName1", "Field\nName2"], ["Value1", "Value2"]]

Actions #2

Updated by Go MAEDA about 4 hours ago

The attached patch should fix the issue.

Actions

Also available in: Atom PDF