Defect #13354

PDF export: incompatible character encodings: UTF-8 and ASCII-8BIT

Added by Ivan Cenov over 1 year ago. Updated about 1 year ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:PDF export
Target version:2.2.4
Resolution:Fixed Affected version:

Description

When trying to export an issue to PDF an internal error happens:

Started GET "/issues/1932.pdf" for 192.168.60.101 at 2013-03-04 21:04:45 +0200
Processing by IssuesController#show as PDF
  Parameters: {"id"=>"1932"}
  Current user: jwalker (id=30)
Completed 500 Internal Server Error in 160ms

Encoding::CompatibilityError (incompatible character encodings: UTF-8 and ASCII-8BIT):
  lib/redmine/i18n.rb:30:in `l'
  app/helpers/issues_helper.rb:336:in `show_detail'
  app/helpers/issues_helper.rb:240:in `block in details_to_strings'
  app/helpers/issues_helper.rb:225:in `details_to_strings'
  lib/redmine/export/pdf.rb:687:in `block in issue_to_pdf'
  lib/redmine/export/pdf.rb:680:in `each'
  lib/redmine/export/pdf.rb:680:in `issue_to_pdf'
  app/controllers/issues_controller.rb:124:in `block (2 levels) in show'
  app/controllers/issues_controller.rb:116:in `show'

My system:
Redmine 2.2.3
Ruby 1.9.3
MySQL
Windows 7

gems

actionmailer (3.2.12, 3.2.11)
actionpack (3.2.12, 3.2.11)
activemodel (3.2.12, 3.2.11)
activerecord (3.2.12, 3.2.11)
activeresource (3.2.12, 3.2.11)
activesupport (3.2.12, 3.2.11)
arel (3.0.2)
bigdecimal (1.1.0)
builder (3.0.0)
bundler (1.2.1)
coderay (1.0.8)
daemons (1.1.9)
erubis (2.7.0)
eventmachine (1.0.0 x86-mingw32, 1.0.0.rc.4 x86-mingw32)
factory_girl (4.2.0, 4.1.0)
factory_girl_rails (4.2.1, 4.1.0)
hike (1.2.1)
i18n (0.6.1)
io-console (0.3)
journey (1.0.4)
jquery-rails (2.0.3)
json (1.7.7, 1.7.6)
mail (2.4.4)
metaclass (0.0.1)
mime-types (1.21, 1.19)
minitest (2.5.1)
mocha (0.12.3)
multi_json (1.6.0, 1.5.0)
mysql2 (0.3.11 x86-mingw32)
net-ldap (0.3.1)
pg (0.14.1 x86-mingw32)
polyglot (0.3.3)
rack (1.4.5, 1.4.4)
rack-cache (1.2)
rack-openid (1.3.1)
rack-ssl (1.3.3, 1.3.2)
rack-test (0.6.2)
rails (3.2.12, 3.2.11)
railties (3.2.12, 3.2.11)
rake (10.0.3)
rdoc (3.12.1, 3.12, 3.9.4)
rmagick (2.13.2, 2.13.1)
ruby-ole (1.2.11.6)
ruby-openid (2.1.8)
rubyzip (0.9.9)
shoulda (2.11.3)
simplecov (0.7.1)
simplecov-html (0.7.1)
simplecov-rcov (0.2.3)
spreadsheet (0.7.9, 0.7.6)
sprockets (2.2.2, 2.1.3)
sqlite3 (1.3.7 x86-mingw32)
thin (1.5.0)
thor (0.17.0, 0.16.0)
tilt (1.3.3)
treetop (1.4.12)
tzinfo (0.3.35)
yard (0.8.4.1, 0.8.3)

We use Redmine in Bulgarian language (cyrillic). Because of this, all issue statuses are stored in the database in Bulgarian language.

I did some research. This error does not occur always. I saw that it happens when:
- There is a status change.
and
- The new status is in Bulagrian language.
and
- this status change is reported as an issue note.

I did this test: translated several of the statuses in English and then:

1. Created an issue with status in Bulgarian (cyrillic) language.
2. The issue was expored succesfully.
3. Changed the status of the issue to any of the english-language statuses.
4. The issue was expored succesfully.
5. Changed the same status back to its initial status in Bulgarian (cyrillic) language.
6. The issue was not exported and above internal error occured.

So, I think that this specific error occurs when the new status of an issue is in cyrillic language.

I did SQL dump, and saw that all tables are utf-8 encoded, and cannot tell where this 'ASCII-8BIT' comes from.

Capture.PNG (39.7 KB) Ivan Cenov, 2013-03-06 18:44

Capture.PNG (32.6 KB) Ivan Cenov, 2013-03-06 22:49

issues_helper.rb_r11621.patch Magnifier (570 Bytes) Jun NAITOH, 2013-03-13 13:50

Associated revisions

Revision 11637
Added by Toshi MARUYAMA over 1 year ago

PDF: fix incompatible character encodings: UTF-8 and ASCII-8BIT (#13354)

Contributed by Jun NAITOH.

Revision 11638
Added by Toshi MARUYAMA over 1 year ago

Merged r11637 from trunk to 2.2-stable (#13354)

PDF: fix incompatible character encodings: UTF-8 and ASCII-8BIT.

Contributed by Jun NAITOH.

Revision 11639
Added by Toshi MARUYAMA over 1 year ago

Merged r11637 from trunk to 2.3-stable (#13354)

PDF: fix incompatible character encodings: UTF-8 and ASCII-8BIT.

Contributed by Jun NAITOH.

History

#1 Updated by Ivan Cenov over 1 year ago

This error occurs in current trunk, r11535 too.

#2 Updated by Ivan Cenov over 1 year ago

Category: PDF export

#3 Updated by Ivan Cenov over 1 year ago

Something wrong with 'value' at source:/trunk/app/helpers/issues_helper.rb#L336

          l(:text_journal_changed, :label => label, :old => old_value, :new => value).html_safe
I can't tell much more...

#4 Updated by Toshi MARUYAMA over 1 year ago

  • Category set to PDF export

#5 Updated by Filou Centrinov over 1 year ago

I can't reproduce your error. I used a cyrillic (russian) status. Are you using the export for a single issue or the issue overview export? Could you give a more exact process how I can reproduce this error for a new issue? May also with a screenshot.

#6 Updated by Ivan Cenov over 1 year ago

I tested with issue overview export. Now tested trunk without any plugins:

  1. Created a task with status Назначена (Cyrillic)
  2. Test PDF export = successful
  3. Changed status to In progress (latin)
  4. Test PDF export = successful
  5. Changed status to Приключена (Cyrillic)
  6. Test PDF export = not successful

Here is what I did step by step:

#7 Updated by Filou Centrinov over 1 year ago

It is weird. At first I couldn't reproduce this error. Than I switch to bulgarian profil language and I got also the error. When I switch back to german profil language this error could be reproduced again although I couldn't reproduce this error first.

#8 Updated by Ivan Cenov over 1 year ago

Is it possible something wrong in config/locales/bg.yml to activate this behavior. I am the translator of Redmine to Bulgarian language; looked at bg.yml but did not find anything suspicious in it...

#9 Updated by Ivan Cenov over 1 year ago

tested with Russian locale - the error still here:

#10 Updated by Filou Centrinov over 1 year ago

I got thiserror everytime when I reset the status to a status with non-latain characters. With the initial status there is no problem to receive the pdf file. Also with a german status "Gelöst" it will fail to produce a pdf file. The pdf export succeeds when the last status had been in latain letters.

#11 Updated by Jun NAITOH over 1 year ago

I reproduced your error.

  1. Set English locale.
  2. load English default data.
  3. Select "Administration" -> "Issue statuses" setting. And, "In Progress" was rewritten to "В работе" on Issue statuses setting.
  4. create New Issue (Status : "New")
  5. change status : from "New" to "В работе"
  6. Select My account -> Language : change from English to Russian
  7. Select Issue and export to PDF, but PDF export Error occured.

my environment

  • Redmine 2.2.2 and trunk
  • Ruby 1.9.3-p392
  • CentOS 6

ad-hoc patch

--- app/helpers/issues_helper.rb    (revision 11561)
+++ app/helpers/issues_helper.rb    (working copy)
@@ -333,7 +333,7 @@
       case detail.property
       when 'attr', 'cf'
         if detail.old_value.present?
-          l(:text_journal_changed, :label => label, :old => old_value, :new => value).html_safe
+          l(:text_journal_changed, :label => label, :old => old_value, :new => value.force_encoding('UTF-8')).html_safe
         elsif multiple
           l(:text_journal_added, :label => label, :value => value).html_safe
         else

I wrote ad-hoc patch.
However, I do not know for this problem to occur why, Sorry.

#12 Updated by Filou Centrinov over 1 year ago

With "Jun NAITOH"-Patch you get also an error, when your the initial status had a non-latain letter.

Improvment of quick & dirty patch:

--- app/helpers/issues_helper.rb    (revision 11561)
+++ app/helpers/issues_helper.rb    (working copy)
@@ -333,7 +333,7 @@
       case detail.property
       when 'attr', 'cf'
         if detail.old_value.present?
-          l(:text_journal_changed, :label => label, :old => old_value, :new => value).html_safe
+          l(:text_journal_changed, :label => label, :old => old_value.force_encoding('UTF-8'), :new => value.force_encoding('UTF-8')).html_safe
         elsif multiple
           l(:text_journal_added, :label => label, :value => value).html_safe
         else

#13 Updated by Ivan Cenov over 1 year ago

Jun NAITOH and Filou Centrinov, I confirm your patch helped.

I think (by analogy with your patch) the whole part should be repaired:

    elsif detail.value.present?
      case detail.property
      when 'attr', 'cf'
        if detail.old_value.present?
          l(:text_journal_changed, :label => label, :old => old_value.force_encoding('UTF-8'), :new => value.force_encoding('UTF-8')).html_safe
        elsif multiple
          l(:text_journal_added, :label => label, :value => value).html_safe
        else
          l(:text_journal_set_to, :label => label, :value => value).html_safe
        end
      when 'attachment'
        l(:text_journal_added, :label => label, :value => value).html_safe
      end
    else
      l(:text_journal_deleted, :label => label, :old => old_value).html_safe
    end
should be changed to
    elsif detail.value.present?
      case detail.property
      when 'attr', 'cf'
        if detail.old_value.present?
          l(:text_journal_changed, :label => label, :old => old_value.force_encoding('UTF-8'), :new => value.force_encoding('UTF-8')).html_safe
        elsif multiple
          l(:text_journal_added, :label => label, :value => value.force_encoding('UTF-8')).html_safe
        else
          l(:text_journal_set_to, :label => label, :value => value.force_encoding('UTF-8')).html_safe
        end
      when 'attachment'
        l(:text_journal_added, :label => label, :value => value.force_encoding('UTF-8')).html_safe
      end
    else
      l(:text_journal_deleted, :label => label, :old => old_value.force_encoding('UTF-8')).html_safe
    end
But I think this is repair of the consequences and not of the root cause.

#14 Updated by Albert Shakirov over 1 year ago

I also had this problem, and did the same thing :old => old_value.force_encoding('UTF-8'), :new => value.force_encoding('UTF-8')
one week ago. But today I got a problem with this patch: when I tried to change the "% Done" field from 70% to 100% in the issue, I got another error 500:

ActionView::Template::Error (undefined method `force_encoding' for 70:Fixnum):
    1: <%= l(:text_issue_updated, :id => "##{@issue.id}", :author => @journal.user) %>
    2:
    3: <% details_to_strings(@journal.details, true).each do |string| -%>
    4: <%= string %>
    5: <% end -%>
    6:
  app/helpers/issues_helper.rb:335:in `show_detail'
  app/helpers/issues_helper.rb:238:in `block in details_to_strings'
  app/helpers/issues_helper.rb:223:in `details_to_strings'
  app/views/mailer/issue_edit.text.erb:3:in `_app_views_mailer_issue_edit_text_erb__1046372188_95127910'
  app/models/mailer.rb:417:in `block in mail'
  app/models/mailer.rb:416:in `mail'
  app/models/mailer.rb:76:in `issue_edit'
  app/models/mailer.rb:427:in `initialize'
  app/models/mailer.rb:440:in `method_missing'
  app/models/journal_observer.rb:26:in `after_create'
  app/models/journal.rb:52:in `save'
  app/models/issue.rb:1362:in `create_journal'
  app/models/issue.rb:965:in `block in save_issue_with_child_records'
  app/models/issue.rb:952:in `save_issue_with_child_records'
  app/controllers/issues_controller.rb:176:in `update

So I suggest slightly modificated solution:

      when 'attr', 'cf'
        if detail.old_value.present?
           m_old_value=old_value
           m_old_value.force_encoding("UTF-8") if m_old_value.respond_to?(:force_encoding)
           m_value=value
           m_value.force_encoding("UTF-8") if m_value.respond_to?(:force_encoding)
           l(:text_journal_changed, :label => label, :old => m_old_value, :new => m_value).html_safe
        elsif multiple

I'm not good in programming, so please forgive my not-very-optimal variant.

#15 Updated by Jun NAITOH over 1 year ago

I think that this problem was caused by find_name_by_reflection method.
On pdf export function, sometimes was returned ASCII-8BIT encoding by find_name_by_reflection method.

I rewrote patch.

#16 Updated by Ivan Cenov over 1 year ago

Is it possible to include this patch in version 2.3.0 ?

#17 Updated by Toshi MARUYAMA over 1 year ago

  • Target version set to 2.2.4

#18 Updated by Toshi MARUYAMA over 1 year ago

  • Status changed from New to Closed
  • Resolution set to Fixed

Committed in trunk r11637, 2.2-stable r11638 and 2.3-stable r11639, thanks.

#19 Updated by Alexandr Sobolev about 1 year ago

Hello!
I'm configured notification via mail
My config file:

production:
   email_delivery:
     delivery_method: :smtp
     smtp_settings:
       address: smtp.mail.ru
       domain: mail.ru
       port: 25
       authentication: :login
       user_name: 'test@mail.ru'
       password: 'blabla'

When i send test mail, get error:

Encoding::CompatibilityError (incompatible character encodings: UTF-8 and ASCII-8BIT):
  lib/redmine/i18n.rb:32:in `l'
  app/controllers/admin_controller.rb:68:in `rescue in test_email'
  app/controllers/admin_controller.rb:64:in `test_email'

Environment:
  Redmine version                          2.3.0.stable
  Ruby version                             1.9.3 (x86_64-linux)
  Rails version                            3.2.13
  Environment                              production
  Database adapter                         Mysql2

Help please!
Thanks!

#20 Updated by Toshi MARUYAMA about 1 year ago

Alexandr Sobolev wrote:

Help please!
Thanks!

Do not post to unrelated issue.
See #14009#note-10.

Also available in: Atom PDF