Feature #7037
CSV export encoding and excel. UTF-8 and BOM
| Status: | Closed | Start date: | 2010-12-03 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: |  Jean-Philippe Lang | % Done: | 0% | |
| Category: | Issues | |||
| Target version: | 3.1.0 | |||
| Resolution: | Fixed |
Description
Redmine doesn't add BOM to the UTF-8 exported csv file, so I have to manually enforce csv encoding to windows-1251. In other way Excel will not display it correctly.
Maybe there is some setting to enforce BOM while doing csv export?
References:7037-csv-bom.diff 
(4.53 KB)
7037-csv-bom-1.diff
(4.53 KB)
7037-csv-bom-2.diff
(4.54 KB)
excel2007.png (38.4 KB)
libreoffice.png (46.7 KB)
Office2013Pro.png (41.6 KB)
Related issues
Associated revisions
Adds a class for handling CSV generation (#7037).
Add BOM to UTF-8 encoded CSV (#7037).
History
#1
Updated by Terence Mill over 11 years ago
Is that fixed?
#2
Updated by John Yani over 11 years ago
Well, the fix exists. But AFAIK it is not merged yet.
The correct way would be to add some setting to enable/disable BOM instead of just hard-coding it.
#3
Updated by Alexander Kriventsov about 11 years ago
John Yani wrote:
Redmine doesn't add BOM to the UTF-8 exported csv file, so I have to manually enforce csv encoding to windows-1251. In other way Excel will not display it correctly.
How did you do this?
#4
Updated by John Yani about 11 years ago
Alexander Kriventsov wrote:
John Yani wrote:
Redmine doesn't add BOM to the UTF-8 exported csv file, so I have to manually enforce csv encoding to windows-1251. In other way Excel will not display it correctly.
How did you do this?
I've edited the "general_csv_encoding" value in corresponding "/config/locales/*.yml" file.
#5
Updated by Etienne Massip about 11 years ago
- Category set to Issues
#6
Updated by Nick Tan over 9 years ago
hi,
i have the exactly same issue, with the plugin https://github.com/sprosin/redmine_csv_with_bom now the csv export works as expected.
but i think this patch should be merge into the redmine core, not in form of a plugin with monkey patching style
#7
Updated by Yuri Rumega over 7 years ago
The problem is still actual in 2.6. Had to adapt "redmine_csv_with_bom" plugin.
#8
Updated by Toshi MARUYAMA over 7 years ago
- Tracker changed from Defect to Feature
#9
Updated by Toshi MARUYAMA over 7 years ago
- Related to Defect #19523: problem in exporting Arabic text to csv added
#10
Updated by Toshi MARUYAMA over 7 years ago
- File 7037-csv-bom.diff added
- Target version set to 3.1.0
This is patch for trunk r14155.
#11
Updated by Toshi MARUYAMA over 7 years ago
- File 7037-csv-bom-1.diff added
Removed extra white space from previous patch.
#12
Updated by Toshi MARUYAMA over 7 years ago
- File 7037-csv-bom-2.diff added
Previous patch has mistake.
This patch fixed.
-+ encoding = Setting.csv_utf8_bom ? "UTF-8" : l(:general_csv_encoding)
++ encoding = Setting.csv_utf8_bom? ? "UTF-8" : l(:general_csv_encoding)
#13
Updated by Toshi MARUYAMA about 7 years ago
- Related to Defect #11116: CSV export encoding problem when issues and interface languages are not equal added
#14
Updated by Jean-Philippe Lang about 7 years ago
- File excel2007.png added
Doesn't work for me when opening the CSV with Excel 2007 (which is supposed to support UTF-8 with BOM), tested with 7037-csv-bom-2.diff applied :
#15
Updated by Toshi MARUYAMA about 7 years ago
Could you check "csv_utf8_bom" is on in setting?
#16
Updated by Jean-Philippe Lang about 7 years ago
Yes, it was checked. The csv file is recognized as UTF-8 with BOM by my text editor but not by Excel 2007.
#17
Updated by Toshi MARUYAMA about 7 years ago
- File libreoffice.png added
$ hexdump -C issues.csv 00000000 ef bb bf 23 3b 54 72 61 63 6b 65 72 3b 53 74 61 |...#;Tracker;Sta| 00000010 74 75 74 3b 50 72 69 6f 72 69 74 c3 a9 3b 53 75 |tut;Priorit..;Su| 00000020 6a 65 74 3b 41 73 73 69 67 6e c3 a9 20 c3 a0 3b |jet;Assign.. ..;| 00000030 4d 69 73 2d c3 a0 2d 6a 6f 75 72 0a 38 37 3b 42 |Mis-..-jour.87;B| 00000040 75 67 3b 4e 65 77 3b 4e 6f 72 6d 61 6c 3b 4e 6f |ug;New;Normal;No| 00000050 6e 2d 41 53 43 49 49 20 c3 a7 c3 a9 c3 a0 c3 b9 |n-ASCII ........| 00000060 3b 22 22 3b 31 30 2f 30 35 2f 32 30 31 35 20 31 |;"";10/05/2015 1| 00000070 39 3a 32 35 0a |9:25.| 00000075 $ hexdump -c issues.csv 0000000 357 273 277 # ; T r a c k e r ; S t a 0000010 t u t ; P r i o r i t 303 251 ; S u 0000020 j e t ; A s s i g n 303 251 303 240 ; 0000030 M i s - 303 240 - j o u r \n 8 7 ; B 0000040 u g ; N e w ; N o r m a l ; N o 0000050 n - A S C I I 303 247 303 251 303 240 303 271 0000060 ; " " ; 1 0 / 0 5 / 2 0 1 5 1 0000070 9 : 2 5 \n 0000075
$ libreoffice --version LibreOffice 4.0.4.2 $ libreoffice issues.csv
#18
Updated by Toshi MARUYAMA about 7 years ago
- File Office2013Pro.png added
I can open on my Japanese MS Office 2013 Professional with general_csv_separator change.
diff --git a/config/locales/fr.yml b/config/locales/fr.yml
--- a/config/locales/fr.yml
+++ b/config/locales/fr.yml
@@ -158,7 +158,8 @@ fr:
general_text_no: 'non'
general_text_yes: 'oui'
general_lang_name: 'French (Français)'
- general_csv_separator: ';'
+ # general_csv_separator: ';'
+ general_csv_separator: ','
general_csv_decimal_separator: ','
general_csv_encoding: ISO-8859-1
general_pdf_fontname: freesans
#19
Updated by Toshi MARUYAMA about 7 years ago
The delimiter semicolon is expected when file is in ANSI.
The delimiter TAB is expected when file is in Unicode.
I don't know about French.
#20
Updated by Jean-Philippe Lang about 7 years ago
Is there any situation where we want to use UTF-8 without BOM?
Because I'm not sure that a new setting is required, we could add the BOM when CSV is encoded in UTF-8 just like the redmine_csv_with_bom plugin does.
#21
Updated by Jean-Philippe Lang about 7 years ago
- Status changed from New to Closed
- Assignee set to Jean-Philippe Lang
- Resolution set to Fixed
#22
Updated by Go MAEDA almost 7 years ago
- Duplicated by Feature #11301: Add "byte order mark" to the export CSV file added
#23
Updated by Toshi MARUYAMA over 6 years ago
- Related to Patch #17902: CSV encoding should be UTF-8 in French locale added