Receiving emails and Truncate emails

Added by Anonymous about 9 years ago

Hi,

I am having a weird problem with receiving emails. I've added under "Truncate emails after one of these lines" the line "REPLY ABOVE THIS LINE" and then I have hardcoded

<p style="font-size:80%;color:grey" >REPLY ABOVE THIS LINE</p>

as the first line of issue_edit.text.html.rhtml so that this line appears in the email notification when an issue is edited.

So far so good - notifications of issue edit now contain the line, I can reply and the reply gets added as a comment to the issue BUT the body is not truncated. Not even if I put the truncation line by hand.

It seemed to work for a couple of mails but then I changed the truncation line a bit. Now it doesn't work. I've checked various time that the strings are exactly the same and I've restarted the server many times. No luck.

Where am I making a fool out of myself?

Many thanks!

Replies (19)

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

If you're a fool then so am I, I am using "STOP" and "====" for my delimiters and tested STOP with no dice..

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

I think the issue for me was that strip_tags was leaving whitespace in front of the delimiter, so I changed app/models/mail_handler.rb to allow whitespace before the delimiter:

regex = Regexp.new("^\s*(#{ delimiters.join('|') })\s*[\r\n].*", Regexp::MULTILINE)

Seems to have fixed it.

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

sigh Danm, I knew the day would come when I would really have to learn regular expressions :) http://xkcd.com/208/

I will try that, many thanks for the hints guys!

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

Hm... Didn't work for me. Anyone has another idea?

Another thing that I wonder: all line breaks are stripped from the mail, is that normal behavior??

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

Well that would explain why the regex isn't working, if the line breaks are stripped.

RE: Receiving emails and Truncate emails - Added by Felix Schäfer about 9 years ago

Jan Wedekind wrote:

Another thing that I wonder: all line breaks are stripped from the mail, is that normal behavior??

The line breaks might get eaten by the html rendering once you look at it in the browser, but redmine shouldn't strip them. I'm not sure if the Regex isn't applied after the html has been stripped, have you tried putting just REPLY ABOVE THIS LINE in your line match thingie?

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

Felix Schäfer wrote:

The line breaks might get eaten by the html rendering once you look at it in the browser, but redmine shouldn't strip them. I'm not sure if the Regex isn't applied after the html has been stripped, have you tried putting just REPLY ABOVE THIS LINE in your line match thingie?

Yes, I did that (w/o any special characters in the line). I even put that in the email by hand.

I will put back the mail_handler from the trunk to check if I have touched something without knowing that screws that up.

Is there any prior bad experience with mails sent from Apple Mail or pulled from Exchange 2007 via IMAP? Exchange tends to do weird things (just learned for example, that it strips the pictures from the notification mail grrr!)

RE: Receiving emails and Truncate emails - Added by Felix Schäfer about 9 years ago

Jan Wedekind wrote:

Is there any prior bad experience with mails sent from Apple Mail

Works well here (in general apple will either adhere to a standard or break it enough that they call it something else).

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

Felix Schäfer wrote:

Works well here (in general apple will either adhere to a standard or break it enough that they call it something else).

Well put :).

Still, it won't work here. Attached a screen showing one reply sent from my Gmail account and one from Apple Mail hooked up to Exchange 2007. As you can see, the Gmail one seems almost fine, but I note that it also stripped the first line break (the one before "On Oct...") and the line break in the footer of the notification ("... in it.Change...").

The one from Apple Mail is completely stripped of all line breaks.

Before, I put back the mail_handler.rb from the trunk (although I hadn't touched it afaik). I almost removed any string from the "Truncate after" field in the settings. Then I restarted the server.

Stupid question: according to the Wiki, the "Enable WS for incoming emails" is only relevant for direct mail forwards, right? In any case, even with that activated it doesn't work.

Any ideas where I can start digging?? Since it doesn't appear to be a general bug, I surely don't want to file a bug report.

Many thanks,
Jan

screen.png (87.3 KB)

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

Ok, this is getting a bit esoteric.

  • Replies in plain text are parsed and handled correctly, no matter from where I send them
  • Replies from Outlook 2007 sent via the same Exchange Server I use with Apple Mail are handled correctly

I have no idea what gets garbled in the coding of the email sent from Apple Mail via Exchange.

Felix, could you point me to where this body is cut? Is is l266ff in mail_handler.rb?

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

Are there even any line breaks to begin with? e.g, if the html looks like this

<p>This is a test.</p><p>REPLY ABOVE THIS LINE</p>

Then there simply are no line breaks. In which case I'd add some code before the strip_tags to add line breaks.

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

Colin Mollenhour wrote:

Are there even any line breaks to begin with? e.g, if the html looks like this
[...] Then there simply are no line breaks. In which case I'd add some code before the strip_tags to add line breaks.

Hm... Not sure if I fully understand. Shouldn't it also take paragraph breaks into account and not only <br>? Did you mean adding a <br>? If so, where exactly? Sorry for being a bit slow and stupid :(

Felix, I can narrow down the problem to Apple Mail. Gmail and Outlook work nicely. Apple Mail does only work, if the reply is sent in plain text. Can you confirm that it works using Rich Text Mails from Apple Mail?

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

Look at the raw HTML generated by Apple Mail, I suspect it is lacking whitespace and I suppose strip_tags is not smart enough to add newlines.

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

The raw HTML looks positively ghastly (see below). So, apparently, Redmine strips the whole HTML code in there including the <p> tags.

Stupid question: why is Redmine not simply interpreting the "Content-Type: text/plain;" part of the email message? Wouldn't that be a hell lot easier? Or can I hardcode that behavior somewhere?

Many thanks!

Content-Type: text/html; charset="us-ascii" 
Content-Transfer-Encoding: quoted-printable

<html><head><base href=3D"x-msg://526/"></head><body style=3D"word-wrap: br=
eak-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div>Body from the mail sent from Apple Mail.</div><br><div><div>On Oct 1=
8, 2010, at 18:19, IRIS Project Management wrote:</div><br class=3D"Apple-i=
nterchange-newline"><blockquote type=3D"cite"><span class=3D"Apple-style-sp=
an" style=3D"border-collapse: separate; font-family: Helvetica; font-style:=
 normal; font-variant: normal; font-weight: normal; letter-spacing: normal;=
 line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; w=
hite-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal=
-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decoratio=
ns-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-wid=
th: 0px; font-size: medium; "><div style=3D"font-family: Verdana, sans-seri=
f; font-size: 0.8em; color: rgb(72, 72, 72); "><p style=3D"font-size: 10px;=
 color: grey; ">+++</p><br>Issue #331 has been reported by Jan Wedekind.<hr=
 style=3D"width: 1168px; height: 1px; background-image: initial; background=
-attachment: initial; background-origin: initial; background-clip: initial;=
 background-color: rgb(204, 204, 204); border-top-width: 0px; border-right-=
width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style:=
 initial; border-color: initial; background-position: initial initial; back=
ground-repeat: initial initial; "><h1 style=3D"font-family: 'Trebuchet MS',=
 Verdana, sans-serif; margin-top: 0px; margin-right: 0px; margin-bottom: 0p=
x; margin-left: 0px; font-size: 1.2em; "><a href=3D"http://redmine.iris.cat=
/issues/331" style=3D"color: rgb(42, 86, 133); ">Task #331: Test</a></h1><u=
l><li>Author: Jan Wedekind</li><li>Status: New</li><li>Priority: Normal</li=
><li>Assigned to: Jan Wedekind</li><li>Due date:</li><li>Target milestone:<=
/li></ul><p>Here is the description.</p><hr style=3D"width: 1168px; height:=
 1px; background-image: initial; background-attachment: initial; background=
-origin: initial; background-clip: initial; background-color: rgb(204, 204,=
 204); border-top-width: 0px; border-right-width: 0px; border-bottom-width:=
 0px; border-left-width: 0px; border-style: initial; border-color: initial;=
 background-position: initial initial; background-repeat: initial initial; =
"><span class=3D"footer" style=3D"font-size: 0.8em; font-style: italic; "><=
p>You have received this notification because you have either subscribed to=
 it, or are involved in it.<br>Change your notification preferences:<span c=
lass=3D"Apple-converted-space">&nbsp;</span><a class=3D"external" href=3D"h=
ttp://redmine.iris.cat/my/account" style=3D"color: rgb(42, 86, 133); ">http=
://redmine.iris.cat/my/account</a>.<br>Problems or questions:<span class=3D=
"Apple-converted-space">&nbsp;</span><a class=3D"email" href=3D"mailto:redm=
ine@iris.cat" style=3D"color: rgb(42, 86, 133); ">redmine@iris.cat</a>.</p>=
</span></div></span></blockquote></div><br></body></html>=

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

Is there a plain text part attached? If there is Redmine should already be using it.

RE: Receiving emails and Truncate emails - Added by Felix Schäfer about 9 years ago

Jan Wedekind wrote:

Felix, I can narrow down the problem to Apple Mail. Gmail and Outlook work nicely. Apple Mail does only work, if the reply is sent in plain text. Can you confirm that it works using Rich Text Mails from Apple Mail?

I only use plain text, can't say anything about the rich text ones.

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

Thanks Felix, thought so!

Colin, yes, there is a MIME part that is plain text. Here the full raw text of the mail that of which posted just the HTML abvove. The "+++" is where it should cut.

I was wondering why Redmine wasn't just using that part anyway. Maybe this is a bug after all?

From: Jan Wedekind <jwedekind@company.com>
To: Redmine <redmine@company.com>
Content-Class: urn:content-classes:message
Date: Mon, 18 Oct 2010 12:19:01 -0400
Subject: Re: [Test Project - Task #331] (New) Test
Thread-Topic: [Test Project - Task #331] (New) Test
Thread-Index: Actu4C1ghl4XJ46BS/2jIhw4rMr8dw==
Message-ID: <CEFFDD21-3054-46CF-B8B4-348D111BB345@company.com>
References: <redmine.issue-331.20101018181919@company.com>
In-Reply-To: <redmine.issue-331.20101018181919@company.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator:
Content-Type: multipart/alternative;
    boundary="_000_CEFFDD21305446CFB8B4348D111BB345iriscat_" 
MIME-Version: 1.0

--_000_CEFFDD21305446CFB8B4348D111BB345companycom_
Content-Type: text/plain; charset="us-ascii" 
Content-Transfer-Encoding: quoted-printable

Body from the mail sent from Apple Mail.

On Oct 18, 2010, at 18:19, IRIS Project Management wrote:

+++

Issue #331 has been reported by Jan Wedekind.
________________________________
Task #331: Test<http://redmine.company.com/issues/331>

 *   Author: Jan Wedekind
 *   Status: New
 *   Priority: Normal
 *   Assigned to: Jan Wedekind
 *   Due date:
 *   Target milestone:

Here is the description.

________________________________

You have received this notification because you have either subscribed to i=
t, or are involved in it.
Change your notification preferences: http://redmine.company.com/my/account.
Problems or questions: redmine@company.com<mailto:redmine@company.com>.

--_000_CEFFDD21305446CFB8B4348D111BB345companycom_
Content-Type: text/html; charset="us-ascii" 
Content-Transfer-Encoding: quoted-printable

<html><head><base href=3D"x-msg://526/"></head><body style=3D"word-wrap: br= (...rest of the HTML part)

--_000_CEFFDD21305446CFB8B4348D111BB345iriscat_--

It's no supper biggie because I mainly activated it now as a fallback for external partners who don't use Redmine daily and might just hit "Reply" on the notification. So the main issue really is getting the info into the system and Redmine adds it as a comment to the right issue (and even the attachment!). So I am already happy.

So now my question is more if it's worth filing a bug report on that.

Felix, maybe you can make a quick check with Apple Mail using Rich Text? Just to see if that is a common problem our just ours.

RE: Receiving emails and Truncate emails - Added by Colin Mollenhour about 9 years ago

The code is supposed to prefer the text/plain part, so I definitely think you should file a bug.

    parts = @email.parts.collect {|c| (c.respond_to?(:parts) && !c.parts.empty?) ? c.parts : c}.flatten
    if parts.empty?
      parts << @email
    end
    plain_text_part = parts.detect {|p| p.content_type == 'text/plain'}

RE: Receiving emails and Truncate emails - Added by Anonymous about 9 years ago

I filed a bug report: #6705. Thanks again already for all your input and help, greatly appreciated.

(1-19/19)