Defect #17826

PDF export fails if there is U+FFFD in issue data or journals

Added by Vincent Robert over 3 years ago. Updated over 3 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Toshi MARUYAMA% Done:

0%

Category:PDF export
Target version:2.4.7
Resolution:Fixed Affected version:2.5.2

Description

When users add special characters to issues (often by copy/pasting text from terminals to issue journals), they cannot export these issues to PDF and get an error (invalid byte sequence in UTF-8).

Here is an example. With these special characters in it, an error should occur when we try to export this issue to PDF.

Archivage automatique          D�sactiv
S�quence de journal en ligne
S�quence de journal courante

pdf-issue-notes.png (22 KB) Toshi MARUYAMA, 2014-09-11 14:52

Associated revisions

Revision 13368
Added by Toshi MARUYAMA over 3 years ago

pdf: add test to export issue pdf with UTF-8 U+FFFD (#17826)

Revision 13369
Added by Toshi MARUYAMA over 3 years ago

pdf: fix r13368 test name (#17826)

Revision 13370
Added by Toshi MARUYAMA over 3 years ago

2.5-stable: pdf: fix failure exporting issue pdf with UTF-8 U+FFFD (#17826)

This revision has trunk r13368 and r13369 test.

Revision 13371
Added by Toshi MARUYAMA over 3 years ago

2.4-stable: pdf: fix failure exporting issue pdf with UTF-8 U+FFFD (#17826)

This revision has trunk r13368 and r13369 test.

History

#1 Updated by Vincent Robert over 3 years ago

New test :

Archivage automatique D�sactiv
Destination de l'archive /uarch
S�quence de journal en ligne la plus ancienne
S�quence de journal courante

#2 Updated by Vincent Robert over 3 years ago

The problem occurs on my server (with Redmine 2.5.2 / ruby 2.0.0 and no plugin) but everything seems to work fine here...

#3 Updated by Jean-Baptiste Barth over 3 years ago

  • Status changed from New to Needs feedback
  • Assignee set to Jean-Baptiste Barth
  • Target version set to Candidate for next minor release

Can you try to reproduce: 1/ on redmine 2.5.2 with ruby 1.9.3, 2/ on current trunk ? Anyway the stack trace would be helpful (I have access to this through our common instance but not other contributors ;)). Maybe we can force encoding somewhere.

Note that I'd like to see it reproduced on current trunk because r13358 has extracted the vendored rfpdf plugin and we will use the gem starting with 2.6.0, so I'd like to see if issue is the same with that. But if we can fix that for 2.5.3, let's go then ;-)

#4 Updated by Vincent Robert over 3 years ago

PDF export works fine with ruby 1.9.3 and redmine 2.5.2.
The error occurs with ruby 2.0.0 and ruby 2.1.2.

Here is the stack trace:

ArgumentError (invalid byte sequence in UTF-8):
lib/plugins/rfpdf/lib/tcpdf.rb:3037:in `gsub'
lib/plugins/rfpdf/lib/tcpdf.rb:3037:in `escape'
lib/plugins/rfpdf/lib/tcpdf.rb:3028:in `escapetext'
lib/plugins/rfpdf/lib/tcpdf.rb:1748:in `Cell'
lib/plugins/rfpdf/lib/tcpdf.rb:2021:in `Write'
lib/plugins/rfpdf/lib/tcpdf.rb:3530:in `block in writeHTML'
lib/plugins/rfpdf/lib/tcpdf.rb:3501:in `each'
lib/plugins/rfpdf/lib/tcpdf.rb:3501:in `writeHTML'
lib/plugins/rfpdf/lib/tcpdf.rb:3612:in `writeHTMLCell'
lib/redmine/export/pdf.rb:141:in `RDMwriteHTMLCell'
lib/redmine/export/pdf.rb:601:in `issue_to_pdf'
app/controllers/issues_controller.rb:133:in `block (2 levels) in show'
app/controllers/issues_controller.rb:125:in `show'

I will try with the last version from trunk.

#5 Updated by Toshi MARUYAMA over 3 years ago

Vincent Robert wrote:

PDF export works fine with ruby 1.9.3 and redmine 2.5.2.
The error occurs with ruby 2.0.0 and ruby 2.1.2.

Here is the stack trace:

ArgumentError (invalid byte sequence in UTF-8):
lib/plugins/rfpdf/lib/tcpdf.rb:3037:in `gsub'
lib/plugins/rfpdf/lib/tcpdf.rb:3037:in `escape'
lib/plugins/rfpdf/lib/tcpdf.rb:3028:in `escapetext'
lib/plugins/rfpdf/lib/tcpdf.rb:1748:in `Cell'
lib/plugins/rfpdf/lib/tcpdf.rb:2021:in `Write'
lib/plugins/rfpdf/lib/tcpdf.rb:3530:in `block in writeHTML'
lib/plugins/rfpdf/lib/tcpdf.rb:3501:in `each'
lib/plugins/rfpdf/lib/tcpdf.rb:3501:in `writeHTML'
lib/plugins/rfpdf/lib/tcpdf.rb:3612:in `writeHTMLCell'
lib/redmine/export/pdf.rb:141:in `RDMwriteHTMLCell'
lib/redmine/export/pdf.rb:601:in `issue_to_pdf'
app/controllers/issues_controller.rb:133:in `block (2 levels) in show'
app/controllers/issues_controller.rb:125:in `show'

I will try with the last version from trunk.

I think your ruby 2.0.0 and 2.1.2 are broken.
Because Redmine 2.5 uses 'ASCII-8BIT'.
source:tags/2.5.2/lib/redmine/export/pdf.rb#L787

#6 Updated by Toshi MARUYAMA over 3 years ago

I cannot reproduce.

$ ruby --version
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]

#7 Updated by Vincent Robert over 3 years ago

Did you try with Redmine 2.5.2?

It appears the error is gone with trunk version of Redmine. It only occurs with 2.5.2 and ruby > 2.0.0 (I reproduced the bug on my Mac, but also on a linux server and even on Heroku ;)

#8 Updated by Toshi MARUYAMA over 3 years ago

Try this patch.

diff --git a/lib/plugins/rfpdf/lib/tcpdf.rb b/lib/plugins/rfpdf/lib/tcpdf.rb
--- a/lib/plugins/rfpdf/lib/tcpdf.rb
+++ b/lib/plugins/rfpdf/lib/tcpdf.rb
@@ -1,3 +1,5 @@
+# encoding: ascii-8bit
+
 #============================================================+
 # File name   : tcpdf.rb
 # Begin       : 2002-08-03

#9 Updated by Toshi MARUYAMA over 3 years ago

Vincent Robert wrote:

Did you try with Redmine 2.5.2?

Sorry, I can reproduce on Redmine 2.5.

#10 Updated by Vincent Robert over 3 years ago

Great! It works perfectly with this patch.

#11 Updated by Toshi MARUYAMA over 3 years ago

  • Assignee changed from Jean-Baptiste Barth to Toshi MARUYAMA
  • Target version changed from Candidate for next minor release to 2.5.3

#12 Updated by Toshi MARUYAMA over 3 years ago

  • Subject changed from PDF export fails if there are some special characters in issue data or journals to PDF export fails if there is U+FFFD in issue data or journals

#13 Updated by Toshi MARUYAMA over 3 years ago

  • Target version changed from 2.5.3 to 2.4.7

#14 Updated by Toshi MARUYAMA over 3 years ago

  • Status changed from Needs feedback to Closed
  • Resolution set to Fixed

Fixed in 2.5-stable r13370 and 2.4-stable r13371, thanks.

Also available in: Atom PDF