Project

General

Profile

Actions

Defect #17096

closed

Issue emails cannot be threaded by some mailers due to inconsistent Message-ID and References field

Added by Toshi MARUYAMA almost 10 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Email notifications
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Fixed
Affected version:

Description

Redmine 1.4:

Issue adding:

Message-Id: <redmine.issue-1.20140605133813@example.net>

Issue editing:

Message-Id: <redmine.journal-1.20140605134129@example.net>
References: <redmine.issue-1.20140605133813@example.net>

Redmine 2.4:

Issue adding:

Message-ID: <redmine.issue-7.20140605044637.b10ff7cb511bccec@example.net>
References: <redmine.issue-7.20140605044637@example.net>

Issue editing:

Message-ID: <redmine.journal-8.20140605044912.bdbae389b638e71d@example.net>
References: <redmine.issue-7.20140605044637@example.net>

Redmine 2.4 issue adding "Message-ID" and editing "References" are not equal.

So, some mailers cannot thread issue mails.


Files

mail-17096-01.diff (4.64 KB) mail-17096-01.diff Toshi MARUYAMA, 2014-06-05 15:18
mail-17096-20140608.diff (4.71 KB) mail-17096-20140608.diff Toshi MARUYAMA, 2014-06-08 11:47
17096-fix-emails-threading.patch (7.58 KB) 17096-fix-emails-threading.patch Go MAEDA, 2019-02-17 14:56
17096-fix-emails-threading-v2.patch (7.48 KB) 17096-fix-emails-threading-v2.patch Go MAEDA, 2019-03-03 10:01

Related issues

Related to Redmine - Feature #5037: Role-based issue custom field visibilityClosedJean-Philippe Lang2010-03-10

Actions
Related to Redmine - Feature #26791: Send individual notification mails per mail recipientClosedJean-Philippe Lang

Actions
Related to Redmine - Feature #13111: New setting to include the status changes in issue mail notifications subjectClosedGo MAEDA2008-02-04

Actions
Actions #1

Updated by Toshi MARUYAMA almost 10 years ago

  • Description updated (diff)
Actions #2

Updated by Toshi MARUYAMA almost 10 years ago

  • Subject changed from Issue mails cannot be sorting to Issue mails cannot be threaded
  • Description updated (diff)
Actions #3

Updated by Toshi MARUYAMA almost 10 years ago

Behavior changed by #5037 r12012.

test/unit/mailer_test.rb

@@ -231,19 +231,21 @@
   end

   def test_issue_add_message_id
-    issue = Issue.find(1)
-    Mailer.issue_add(issue).deliver
+    issue = Issue.find(2)
+    Mailer.deliver_issue_add(issue)
     mail = last_email
-    assert_equal Mailer.message_id_for(issue), mail.message_id
-    assert_nil mail.references
+    assert_match /^redmine\.issue-2\.20060719190421\.[a-f0-9]+@example\.net/, mail.message_id
+    assert_include "redmine.issue-2.20060719190421@example.net", mail.references
   end

   def test_issue_edit_message_id
-    journal = Journal.find(1)
-    Mailer.issue_edit(journal).deliver
+    journal = Journal.find(3)
+    journal.issue = Issue.find(2)
+
+    Mailer.deliver_issue_edit(journal)
     mail = last_email
-    assert_equal Mailer.message_id_for(journal), mail.message_id
-    assert_include Mailer.message_id_for(journal.issue), mail.references
+    assert_match /^redmine\.journal-3\.\d+\.[a-f0-9]+@example\.net/, mail.message_id
+    assert_include "redmine.issue-2.20060719190421@example.net", mail.references
     assert_select_email do
       # link to the update
       assert_select "a[href=?]",
Actions #4

Updated by Toshi MARUYAMA almost 10 years ago

  • Target version set to 2.6.0
Actions #5

Updated by Toshi MARUYAMA almost 10 years ago

  • Target version changed from 2.6.0 to 2.5.2
Actions #6

Updated by Toshi MARUYAMA almost 10 years ago

This is patch to fix.

Actions #7

Updated by Toshi MARUYAMA almost 10 years ago

This is new patch.
In note 6 patch, mail receiver can guess user id.

Actions #8

Updated by Jean-Philippe Lang almost 10 years ago

  • Subject changed from Issue mails cannot be threaded to Issue mails cannot be threaded by some mailers

As far as I can see, this patch only works if the same users are notified during all the issue thread. But this is not the case.

Actions #9

Updated by Toshi MARUYAMA almost 10 years ago

Jean-Philippe Lang wrote:

As far as I can see, this patch only works if the same users are notified during all the issue thread. But this is not the case.

Yes, it is same behavior before #5037.

Note-7 patch message id:

Issue adding:

Message-ID: <redmine.issue-72.20140627064350.1ebc022e@example.net>

Issue editing:

Message-ID: <redmine.journal-142.20140627064557.1ebc022e@example.net>
References: <redmine.issue-72.20140627064350.1ebc022e@example.net>
Actions #10

Updated by Jean-Philippe Lang over 9 years ago

No it's not the same behaviour as before #5037. Before #5037, we always had a single email notification, the first message-id and all references were fixed, eg. <redmine.issue-72.20140627064350@example.net>.

With the patch in note-7 applied, the references header changes if a new custom field is added or if you change the permissions on custom fields. And that breaks all the thread. Exemple:

Issue creation:

Message-ID: <redmine.issue-16507.20140705102246.3ecc5e01@redmine.org>

First update (OK):

Message-ID: <redmine.journal-55676.20140705102328.3ecc5e01@redmine.org>
References: <redmine.issue-16507.20140705102246.3ecc5e01@redmine.org>

After adding a new custom field, you get:

Message-ID: <redmine.journal-55677.20140705102736.49bb14f7@redmine.org>
References: <redmine.issue-16507.20140705102246.49bb14f7@redmine.org>
Actions #11

Updated by Toshi MARUYAMA over 9 years ago

Jean-Philippe Lang wrote:

With the patch in note-7 applied, the references header changes if a new custom field is added or if you change the permissions on custom fields. And that breaks all the thread.

Yes. But, it keeps while custom field condition.
In other case before #5037, if user not get issue creating mail, then by watching user get issue editing mail, mail thread is broken on some mailer.

This issue is mail thead broken on some mail in spíte of getting issue creating mail and not changing custom field condition.

Actions #12

Updated by Jean-Philippe Lang over 9 years ago

  • Target version changed from 2.5.2 to Candidate for next major release
Actions #14

Updated by Toshi MARUYAMA almost 9 years ago

  • Related to Feature #5037: Role-based issue custom field visibility added
Actions #15

Updated by T E over 7 years ago

Hi,

are there any news on this? I am having problems with my notification mails, too and would love to see that fixed.

Actions #16

Updated by Lukas Zapletal over 7 years ago

are there any news on this? I am having problems with my notification mails, too and would love to see that fixed.

Yeah is gmail related to this? Having troubles too.

Actions #17

Updated by Go MAEDA over 6 years ago

I think that adding consistent Message-Id and References filed will be easier if per user notification sending is implemented by #26791. We don't have to use random_hex in Mailer.token_for method. Instead, we can use EmailAddress#id or some hash value generated from an email address.

The following is an example of the idea.

Issue adding:

Message-Id: <redmine.issue-#{object.id}.#{timestamp}.#{email_address.id}@example.net>

Issue editing:

Message-Id: <redmine.journal-#{object.id}.#{timestamp}.#{email_address.id}@example.net>
References: <redmine.issue-#{object.id}.#{timestamp}.#{email_address.id}@example.net>
Actions #18

Updated by Go MAEDA over 6 years ago

  • Related to Feature #26791: Send individual notification mails per mail recipient added
Actions #19

Updated by Go MAEDA about 5 years ago

Go MAEDA wrote:

I think that adding consistent Message-Id and References filed will be easier if per user notification sending is implemented by #26791. We don't have to use random_hex in Mailer.token_for method. Instead, we can use EmailAddress#id or some hash value generated from an email address.

I wrote a patch.

In Redmine 3.4, Redmine may generate multiple notification messages and each message is sent to multiple recipients grouped by custom fields visibility. For example, if there are 6 recipients and they are grouped to 3 groups by custom fields visibility, Redmine generates 3 messages and then pass those to a mail server. Of course, each message has 2 recipients.

Since multiple messages are generated at once when updating one issue, Redmine adds a random key to the message-id to ensure the message-id is unique. Because the random key is never stored, Redmine cannot reproduce the message-id generated in the past and cannot set exact id to the References field in the notification message when the issue is updated. This is why emails are not properly threaded.

In the latest Redmine 4.0, one message is always sent to one recipient. Since one message is never addressed to multiple recipients, we can use recipient user information like user_id. If we use user_id instead of a random key to make a unique message-id, we can set the correct and exact value to References field in the message when the issue is updated because message ids generated in the past is always reproducible. By doing that, emails should be threaded properly.

The attached patch uses user_id to generate unique message-ids.

Issue added:

Message-Id: <redmine.issue-#{issue.id}.#{issue.created_on}.#{user_id}@example.net>

Issue updated:

Message-Id: <redmine.journal-#{jounal.id}.#{journal.created_on}.#{user_id}@example.net>
References: <redmine.issue-#{issue.id}.#{issue.created_on}.#{user_id}@example.net>
Actions #20

Updated by Marius BĂLTEANU about 5 years ago

  • Related to Feature #13111: New setting to include the status changes in issue mail notifications subject added
Actions #21

Updated by Go MAEDA about 5 years ago

Updated the patch to apply cleanly to the current trunk (r17913).

Actions #22

Updated by Go MAEDA about 5 years ago

  • Target version changed from Candidate for next major release to 4.1.0

Setting the target version to 4.1.0.

Actions #23

Updated by Go MAEDA about 5 years ago

  • Subject changed from Issue mails cannot be threaded by some mailers to Notification emails are not threaded in some email clients due to referencing a nonexistent Message-ID
Actions #24

Updated by Go MAEDA about 5 years ago

  • Subject changed from Notification emails are not threaded in some email clients due to referencing a nonexistent Message-ID to Issue emails cannot be threaded by some mailers due to inconsistent Message-ID and References field
Actions #25

Updated by Go MAEDA about 5 years ago

  • Status changed from New to Closed
  • Assignee set to Go MAEDA
  • Resolution set to Fixed

Committed the patch. The new Message-ID formats are as follows:

Issue added:

Message-Id: <redmine.issue-#{issue.id}.#{issue.created_on}.#{user_id}@example.net>

Issue updated:

Message-Id: <redmine.journal-#{jounal.id}.#{journal.created_on}.#{user_id}@example.net>
References: <redmine.issue-#{issue.id}.#{issue.created_on}.#{user_id}@example.net>
Actions

Also available in: Atom PDF