Project

General

Profile

Actions

Patch #15785

closed

Support more character encodings in incoming emails

Patch #15785: Support more character encodings in incoming emails

Added by Felix Schäfer almost 13 years ago. Updated almost 13 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Email receiving
Target version:

Description

The MailHandler currently tries to convert the encoding of the body of incoming emails to UTF-8 by directly passing the charset of the email body to Redmine::CodesetUtil.to_utf8, see source:/trunk/app/models/mail_handler.rb@12473#L413. Redmine::CodesetUtil.to_utf8 in turn only handles encodings known to Ruby (on 1.9+), which might not be the case for the charset of an incoming email.

Planio recently had the case of a user sending emails in Korean with the charset ks_c_5601-1987, which Ruby doesn't know about. This can be mitigated thanks to Mail::RubyVer.pick_encoding which will try to pick a Ruby encoding compatible with the charset of the email (Mail::RubyVer is a wrapper for the correct functions for the current Ruby version).

I will post a patch for this shortly.


Files

15785.patch (2.2 KB) 15785.patch Felix Schäfer, 2013-12-30 11:33

Related issues 1 (0 open — 1 closed)

Related to Redmine - Patch #18047: MailHandler: Don't use String#respond_to?(:force_encoding) to differentiate between Ruby 1.8 and Ruby 1.9ClosedActions

Updated by Felix Schäfer almost 13 years ago Actions #1

Here is the patch with a test.

Updated by Jan from Planio www.plan.io almost 13 years ago Actions #2

  • Status changed from New to Confirmed
  • Target version set to Candidate for next minor release

Updated by Jan from Planio www.plan.io almost 13 years ago Actions #3

  • Tracker changed from Defect to Patch

Updated by Toshi MARUYAMA almost 13 years ago Actions #4

This patch looks good.
But, Korean encoding detection is in mail 2.5.4.
https://github.com/mikel/mail/commit/88457e66ec5b298bd198b5c3b6739f05320a16ee

Many users posted that mail 2.5.4 has trouble on redmine.org.
For test passes, we need to restrict mail 2.5.4 in Gemfile.

Updated by Toshi MARUYAMA almost 13 years ago Actions #5

Toshi MARUYAMA wrote:

This patch looks good.
But, Korean encoding detection is in mail 2.5.4.
https://github.com/mikel/mail/commit/88457e66ec5b298bd198b5c3b6739f05320a16ee

This means in mail 2.5.4, not in mail 2.5.3.

Updated by Toshi MARUYAMA almost 13 years ago Actions #6

It seems Mail::VERSION returns mail version.
https://github.com/mikel/mail/blob/2-5-stable/lib/mail/version.rb

Updated by Felix Schäfer almost 13 years ago Actions #7

Mail::VERSION.version returns the version of the Mail gem indeed.

I know korean would only work with Mail 2.5.4, and we don't mind requiring it in the Planio Gemfiles to make sure it is supported for us. I think the patch is useful even with earlier versions of the Mail gem, as Mail::RubyVer.pick_encoding is available in earlier versions too and still adds more supported encodings. I agree with you that the test I've written would require Mail 2.5.4 though, you are right.

Should I write a new test that makes sure that the patch works but doesn't require 2.5.4?

Furthermore, what type of errors do users encounter with Mail 2.5.4?

Updated by Toshi MARUYAMA almost 13 years ago Actions #8

  • Subject changed from Support more charsets in incoming emails to Support more charctor encoding in incoming emails

Updated by Toshi MARUYAMA almost 13 years ago Actions #9

  • Subject changed from Support more charctor encoding in incoming emails to Support more character encoding in incoming emails

Updated by Toshi MARUYAMA almost 13 years ago Actions #10

  • Subject changed from Support more character encoding in incoming emails to Support more character encodings in incoming emails

Updated by Toshi MARUYAMA almost 13 years ago Actions #11

Felix Schäfer wrote:

Furthermore, what type of errors do users encounter with Mail 2.5.4?

See #13698.

But Rails 3.2.16 requires mail 2.5.4 or higher.
https://github.com/rails/rails/blame/v3.2.16/actionmailer/actionmailer.gemspec#L23

Updated by Toshi MARUYAMA almost 13 years ago Actions #12

  • Target version changed from Candidate for next minor release to 2.5.0

Updated by Toshi MARUYAMA almost 13 years ago Actions #13

  • Status changed from Confirmed to Closed

Committed in trunk r12474 and tests pass, thanks.

Updated by Felix Schäfer almost 13 years ago Actions #14

Great to hear, thanks!

Updated by Anton Nepomnyaschih almost 13 years ago Actions #15

Is it duplicate for #14675 ? Can we update our Redmine and get the Defect fixed? =)

Updated by Toshi MARUYAMA almost 13 years ago Actions #16

  • Has duplicate Defect #14675: redmine:email:receive_imap failed with localizaed characters in mail body added

Updated by Toshi MARUYAMA almost 13 years ago Actions #17

Anton Nepomnyaschih wrote:

Is it duplicate for #14675 ? Can we update our Redmine and get the Defect fixed? =)

Thank you for your pointing.

Updated by Toshi MARUYAMA almost 13 years ago Actions #18

  • Has duplicate deleted (Defect #14675: redmine:email:receive_imap failed with localizaed characters in mail body)

Updated by Toshi MARUYAMA about 12 years ago Actions #19

  • Related to Patch #18047: MailHandler: Don't use String#respond_to?(:force_encoding) to differentiate between Ruby 1.8 and Ruby 1.9 added
Actions

Also available in: PDF Atom