Defect #8093

Incoming email with internal emails as attachments isn't processed properly

Added by Sergey Prosin about 2 years ago. Updated 5 months ago.

Status:NewStart date:2011-04-07
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:Email receiving
Target version:-
Affected version:1.1.2 Resolution:

Description

Our remine system checks email account regularly for new emails. If we send any email to special address system processes it very good, but if we send an email with other email inside (we use MS Outlook feature "Send as attachement" for emails) then the email will be processed incorrectly and there won't be any attachements inside created issue. Obviously this is some special case in processing.

letter_example.png - Screenshot of the email processed incorrectly (50.6 KB) Sergey Prosin, 2011-04-07 09:35

This_is_an_issue_title.msg (32.5 KB) Sergey Prosin, 2011-04-12 11:43

This_is_an_issue_title__unicode_.msg - Previous message in unicode format (34 KB) Sergey Prosin, 2011-04-12 11:43

This_is_an_issue_title.eml (5.89 KB) Sergey Prosin, 2011-04-13 11:02

msgToEml.patch Magnifier (1.22 KB) Joel Thompson, 2012-08-22 08:12

outlook_attachment.diff Magnifier - Dealing with outlook element attachments (1.12 KB) Sergey Prosin, 2013-01-28 06:56


Related issues

Duplicates Defect #5994: Creating new issue via email submission strips attached e... New 2010-07-29

History

#1 Updated by Sergey Prosin about 2 years ago

Any thoughts how to fix or check it?

#2 Updated by Etienne Massip about 2 years ago

Could you please post the most simple / light test mail which fail to be processed by Redmine so that the issue can be reproduced it easily ?

#3 Updated by Sergey Prosin about 2 years ago

Sure. This are 2 msg files of the message: one just in Unicode format.

#4 Updated by Sergey Prosin about 2 years ago

Unfortunately, outlook doesn't have built-in capabilities for saving emails as emls.
I've converted the email to eml format with some external tool. I'm not sure if it is the same content as processed by the receiver (We use "rake redmine:email:receive_pop3" approach). Is there any way to catch exactly the same content? (Say, extended information in log or something else)

#5 Updated by Sergey Prosin about 2 years ago

Hi, any update about the issue?
Can I help with it?

#6 Updated by Etienne Massip about 2 years ago

Actually, it's a duplicate of #5994 but I keep it open since there is more details.

#7 Updated by Joel Thompson 10 months ago

I spent half the day learning ruby so I could fix this today. So I have uploaded the patch that I came up with for Redmine 1.3.3 but I made this patch to work with the latest in the git repo.

It's really simple because Tmail has already converted the attachment object to EML and saves it to Redmine as "noname" so windows cannot open the file, I tried manually opening it with outlook and adding .msg to the file but no go. Then I looked up the code where it is saving the attachments and made it write it directly to the file system, then I found that it is already an EML. So all this patch does is if the attachment meets the 'message/rfc822' content type requirement then I name the file attachment.eml before it saves it to Redmine. I have also tried sending attachments inside an email attached as an attachment and it all comes through fine.

The patch is for app/models/mail_handler.rb.

#8 Updated by Sergey Prosin 10 months ago

Unfortunately, that doesn't help in any way. After some tests we discovered that the internal email in this case doesn't look like attachement at least system think so.
We used this code for check (mail_handler.rb)
--------------
def add_attachments(obj)
if email.has_attachments?
email.attachments.each_with_index do |attachment, i|
print "attachment.content_type = "
print attachment.content_type

--------------
and there wasn't any output like attachment.content_type = smth

I should also say that in our case there is no noname file in Redmine after email was processed.

To sum that, I think the problem is that ruby mechanisms can't understand the structure of such kind of emails. And that is very sad.

Looking forward to hearing from you, Joel.

#9 Updated by Joel Thompson 10 months ago

Hmm, interesting.

I'm afraid I cannot help with that one, as the attachments are coming through but Redmine just didn't know how to handle them.

Keep in mind I am using 1.3.3 here, and Redmine uses Tmail for the mail handling.

So you are using an older version which cannot handle the Outlook .msg files at all?

#10 Updated by Sergey Prosin 10 months ago

As I said we are using Redmine 1.1.2. May be there is a simple script which we can use for discovering our issues nature?
This is the list of gems we have:
actionmailer (3.2.8, 2.3.14, 2.3.5, 2.3.2)
actionpack (3.2.8, 2.3.14, 2.3.5, 2.3.2)
activemodel (3.2.8)
activerecord (3.2.8, 2.3.14, 2.3.5, 2.3.2)
activerecord-mysql2-adapter (0.0.3)
activeresource (3.2.8, 2.3.14, 2.3.5, 2.3.2)
activesupport (3.2.8, 2.3.14, 2.3.5, 2.3.2)
allison (2.0.3)
arel (3.0.2)
builder (3.0.0)
bundler (1.1.5)
cgi_multipart_eof_fix (2.5.0)
coderay (1.0.7)
daemons (1.1.9, 1.0.10)
echoe (3.1.1)
erubis (2.7.0)
fastercsv (1.5.5)
fastthread (1.0.1 i386-mswin32)
fiveruns_tuneup (0.8.20)
gem_plugin (0.2.3)
gemcutter (0.7.1)
highline (1.6.13, 1.5.1)
hike (1.2.1)
hoe (3.0.8)
i18n (0.6.0, 0.5.0, 0.4.2)
journey (1.0.4)
json_pure (1.7.5)
mail (2.4.4)
mime-types (1.19)
mongrel (1.1.5 x86-mswin32-60)
mongrel_service (0.3.4 i386-mswin32)
multi_json (1.3.6)
mysql (2.8.1 x86-mswin32, 2.7.3 mswin32)
mysql2 (0.3.11 x86-mswin32-60)
net-ldap (0.3.1)
pik (0.2.8)
polyglot (0.3.3)
rack (1.4.1, 1.1.3, 1.0.1)
rack-cache (1.2)
rack-ssl (1.3.2)
rack-test (0.6.1)
rails (2.3.14, 2.3.5, 2.3.2)
rake (0.9.2.2, 0.8.7)
rmagick (2.10.0 mswin32)
ruby-openid (2.2.0, 2.1.8)
rubyforge (2.0.4, 1.0.3)
sprockets (2.1.3)
sqlite3 (1.3.6 x86-mswin32-60)
sqlite3-ruby (1.2.3 mswin32)
thor (0.16.0)
tilt (1.3.3)
treetop (1.4.10)
tzinfo (0.3.33)
win32-service (0.5.2 mswin32)

Any ideas, Joel?

#11 Updated by Sergey Prosin 5 months ago

Hi again. We've just migrated to latest redmine version (2.2.0 to the moment). The issue is still here. So obviously that wasn't related with old redmine version we used before update.
Current list of gems:
actionmailer (3.2.11)
actionpack (3.2.11)
activemodel (3.2.11)
activerecord (3.2.11)
activeresource (3.2.11)
activesupport (3.2.11)
ansi (1.4.3)
arel (3.0.2)
aws-sdk (1.6.9)
bcrypt-ruby (3.0.1 x86-mingw32)
bigdecimal (1.1.0)
bluecloth (2.1.0 x86-mingw32)
builder (3.0.0)
bundler (1.2.3, 1.0.21)
coderay (1.0.6)
coffee-rails (3.2.2)
coffee-script (2.2.0)
coffee-script-source (1.3.3)
crack (0.3.1)
daemons (1.1.8)
diff-lcs (1.1.3)
erubis (2.7.0)
eventmachine (1.0.0 x86-mingw32)
execjs (1.4.0)
fastercsv (1.5.4)
hike (1.2.1)
hoe (3.0.6)
httparty (0.8.3)
i18n (0.6.0)
io-console (0.3)
journey (1.0.4)
jquery-rails (2.1.3, 2.0.3)
json (1.5.4)
mail (2.4.4)
metaclass (0.0.1)
mime-types (1.19, 1.18)
mini_magick (3.4)
mini_portile (0.3.0)
minitest (2.5.1)
mocha (0.12.3)
multi_json (1.3.6)
multi_xml (0.5.1)
mustache (0.99.4)
mysql2 (0.3.11 x86-mingw32, 0.2.18)
net-ldap (0.3.1)
nokogiri (1.5.5 x86-mingw32, 1.4.7 x86-mingw32)
polyglot (0.3.3)
prototype-rails (3.2.1)
rack (1.4.1)
rack-cache (1.2)
rack-mount (0.8.3)
rack-openid (1.3.1)
rack-ssl (1.3.2)
rack-test (0.6.1)
rails (3.2.11)
railties (3.2.11)
rake (0.9.2.2)
rake-compiler (0.8.1)
rdoc (3.12, 3.9.4)
RedCloth (4.2.9 x86-mingw32)
rmagick (2.13.1)
rspec (2.11.0)
rspec-core (2.11.1)
rspec-expectations (2.11.2)
rspec-mocks (2.11.1)
ruby-openid (2.1.8)
rubygems-update (1.8.24)
rvm (1.11.3.5)
sass (3.1.20)
sass-rails (3.2.5)
sprockets (2.4.5, 2.2.1, 2.1.3, 2.1.2)
sqlite3 (1.3.6 x86-mingw32)
sqlite3-ruby (1.3.2 x86-mingw32)
subexec (0.2.2)
test-unit (2.5.2)
thin (1.3.1)
thor (0.15.4, 0.14.6)
tilt (1.3.3)
treetop (1.4.10)
turn (0.9.6)
tzinfo (0.3.33)
uglifier (1.2.6)
uuidtools (2.1.3)
yard (0.8.1)

#12 Updated by Sergey Prosin 5 months ago

Hi! Finally we've been able to deal with this issue. The problem was this kind of attachments in emails generated by outlook was stored not like attachment but as part of multipart message with the following header

------=_NextPart_000_01C9_01CDF973.EB5E8D50
Content-Type: message/rfc822
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment

We've created special patch for this case (attached)

I'm not an expert in ruby so that is obvoisly not a state of the art piece of code and I would be honored if someone would rewrite this to follow ruby, RoR, and Redmine guidelines.
Also that would be great if this code after revision will be added in Redmine core.

Also available in: Atom PDF