Defect #12025

rake receive_pop3 is stripping formatting and new lines

Added by Rob D about 5 years ago. Updated over 2 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:Email receiving
Target version:-
Resolution: Affected version:1.4.4

Description

Since upgrading from redmine 1.3.1 to 1.4.4, the redmine:email:receive_pop3 task seems to be stripping all formatting and new lines out of emails, and adding a continuous blob of text as a one-line comment to a redmine issue.

This continuous blob also includes email signatures which should have been truncated as per the settings (which worked fine in 1.3.1, but is not working anymore) Most likely this is due to the fact the stripping of new lines is occuring before trying to truncate, resulting in no lines matching.

We are using exchange for emails, and nothing else has changed apart from the redmine version.

Raw format from telnet retr.txt Magnifier (13 KB) Rob D, 2012-10-24 06:19

Outlook format from sent items.msg (28.5 KB) Rob D, 2012-10-24 06:19

Sample redmine notification showing problem.msg (32 KB) Rob D, 2012-10-24 06:19

1.png (39.9 KB) Rob D, 2013-09-25 04:47


Related issues

Related to Redmine - Feature #16962: Better handle html-only emails Closed
Related to Redmine - Defect #19740: "Truncate emails after one of these lines" setting is not... Closed

History

#1 Updated by Jean-Philippe Lang about 5 years ago

  • Category set to Email receiving

Could you attach one of these emails (in raw format) so I can try to reproduce?

#2 Updated by Rob D about 5 years ago

I hope these help, please let me know if you need anything else.

Is it possible to call the receive_pop3 rake task with a debug option that saves the raw emails?

#3 Updated by Steve Clay over 4 years ago

We're experiencing this problem, but I'm suspicious it's related to the WYSIWYG functionality. Redmine seems to assume plain text entry and a plain-to-HTML filter on output, and when you enable WYSIWYG editing, the system assumes the content will be HTML and does not run it through a plain-to-HTML filter. When we turned on the WYSIWYG after a few weeks of use, all existing content showed up as described here (without any P/BR breaks).

So I think Redmine is receiving the e-mails, normalizing to a plain text format, but since we have the WYSIWYG on, that plain text is not sent through the plain-to-HTML filter. If that's the case we could either allow HTML e-mail received to be saved directly as markup in Redmine, or we could always apply a plain-to-HTML filter to handle old plain text content added by e-mail/before WYSIWYG was enabled. E.g. Markdown permits P elements to passthrough as is.

#4 Updated by Steve Clay over 4 years ago

We've confirmed that disabling the WYSIWYG eliminates the problem, so I'm guessing the fix will be having the WYSIWYG plugin run incoming e-mail content through something like email_content.gsub(/\n/, '<br />')

#5 Updated by Rob D about 4 years ago

Upgraded to 2.3.3 - but the problem still exists.

#6 Updated by Yang Tang over 3 years ago

Hi,

We have the similar issue on Redmine 1.4.

In our case, it caused by HTML email parser in Redmine 1.4 is incompatible with the latest Microsoft Exchange server. To fix our issue, we modified mail_handler.rb file likes below, and then it works well.

Index: mail_handler.rb
===================================================================
--- mail_handler.rb     (revision 9776)
+++ mail_handler.rb     (working copy)
@@ -346,7 +346,8 @@
     if plain_text_part.nil?
       # no text/plain part found, assuming html-only email
       # strip html tags and remove doctype directive
-      @plain_text_body = strip_tags(@email.body.to_s)
+      # @plain_text_body = strip_tags(@email.body.to_s)
+      @plain_text_body = strip_tags(@email.body.to_s.gsub!(/<o:p>/, '<o:p>&nbsp;').gsub!(/<br>/, "\n").gsub!(/&nbsp;/, "\n"))
       @plain_text_body.gsub! %r{^<!DOCTYPE .*$}, ''
     else
       @plain_text_body = plain_text_part.body.to_s

Hope it can help you to resolve issues

Regards,

Yang

#7 Updated by Toshi MARUYAMA about 3 years ago

#8 Updated by Toshi MARUYAMA about 3 years ago

Try #16962 patch.

#9 Updated by Toshi MARUYAMA over 2 years ago

  • Related to Defect #19740: "Truncate emails after one of these lines" setting is not working added

#10 Updated by Toshi MARUYAMA over 2 years ago

  • Status changed from New to Closed

I think this issue is fixed by #16962.
If you still have problems, please create new issue.

Also available in: Atom PDF