Defect #31365

Issue subject may be broken if the subject field in the receiving email is split into multiple lines

Added by Go MAEDA 4 months ago. Updated 3 months ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Jean-Philippe Lang% Done:

0%

Category:Email receiving
Target version:4.0.4
Resolution:Fixed Affected version:

Description

When you create issues via email, the subject of the created issue may be broken if the subject field in the original email is split into multiple lines like the following.

Date: Mon, 13 May 2019 13:39:29 +0900
Message-ID: <CALXUZfgwpW-hizEabSAjjkykQGV6-V6bwKu693Cr7cDH6DR7pQ@mail.gmail.com>
Subject: =?utf-8?B?zrHOss6zzrTOtc62zrfOuM65zrrOu868zr3Ovs6/z4DPgc+Cz4PPhM+Fz4bP?=
        =?utf-8?B?h8+Iz4k=?=
From: <foo@example.com>
To: <baz@example.com>
Content-Type: text/plain

Split bytes subject.

The subject should be decoded like this:

αβγδεζηθικλμνξοπρςστυφχψω

But the subject of the created issue will be the following:

αβγδεζηθικλμνξοπρςστυφ��ψω

This is due to the bug of mail gem (I tested in 2.7.1).

irb(main):001:0> puts Mail.read('/tmp/split-bytes-subject.eml').subject
αβγδεζηθικλμνξοπρςστυφ��ψω
=> nil

The subject is not split at the boundary between characters but in the middle of bytes that composes a character and mail gem that parses emails does not properly handle such emails. The first line of the decoded subject field is "αβγδεζηθικλμνξοπρςστυφ\xCF" and the second line is "\x87ψω". The character "χ" is consists of two bytes 0xCF and 0x87. These two bytes span two lines. mail gem cannot handle such cases.

split-bytes-subject.eml - Email which reproduces the issue (332 Bytes) Go MAEDA, 2019-05-14 05:49

31365_mail_gem_broken_subject.patch Magnifier (2.38 KB) Yuichi HARADA, 2019-05-16 06:49


Related issues

Related to Redmine - Defect #31582: Issue subject may be truncated if the subject field in th... Confirmed

Associated revisions

Revision 18245
Added by Jean-Philippe Lang 3 months ago

Issue subject may be broken if the subject field is split into multiple lines (#31365).

Patch by Yuichi HARADA.

Revision 18246
Added by Jean-Philippe Lang 3 months ago

Merged r18245 to 4.0-stable (#31365).

Revision 18269
Added by Go MAEDA 3 months ago

Reverts r18245 (#31582, #31365).

Issue subjets are truncated in some cases when they are created via email.

Revision 18270
Added by Go MAEDA 3 months ago

Merged r18269 from trunk to 4.0-stable in order to revert r18246 (#31582, #31365).

History

#1 Updated by Yuichi HARADA 4 months ago

The subject seems to be broken because Mail::SubjectField does not combine multiple lines of subjects into one line.

I created a monkey patch for Mail::SubjectField .

#2 Updated by Go MAEDA 4 months ago

  • Target version set to Candidate for next minor release

#3 Updated by Go MAEDA 4 months ago

I hope to include the workaround in 4.0.4 or 4.1.0.

Fundamentally, mail gem should be fixed and monkey patching in Redmine is not an ideal solution.

However, I think the workaround is beneficial for users who are affected by this problem. Especially East Asian people such as Japanese, Chinese, and Korean are more affected by the issue because they heavily use multi-byte characters in emails.

#4 Updated by Go MAEDA 4 months ago

  • Target version changed from Candidate for next minor release to 4.0.4

Setting the target version to 4.0.4.

#5 Updated by Jean-Philippe Lang 3 months ago

  • Status changed from New to Closed
  • Assignee set to Jean-Philippe Lang
  • Resolution set to Fixed

Patch committed, thanks!

#6 Updated by Go MAEDA 3 months ago

  • Related to Defect #31582: Issue subject may be truncated if the subject field in the receiving email is split into multiple lines in singlebytes and multibytes mixed added

Also available in: Atom PDF