Defect #14491

MailHandler: Unable to determine target project (when allow_override=project and project=unassigned is used)

Added by Deoren Moor almost 5 years ago. Updated over 3 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Jean-Philippe Lang% Done:

0%

Category:Email receiving
Target version:2.6.0
Resolution:Fixed Affected version:2.3.2

Description

We've had a problem with emails being rejected inexplicably for a long while now, but only recently did I realize that I could specify RAILS_ENV="development" to enable more detailed logging. That done, this is what the development.log file had to say:

Receiving message 14135
  ^[[1m^[[35mUser Load (0.7ms)^[[0m  SELECT `users`.* FROM `users` WHERE `users`.`type` IN ('User', 'AnonymousUser') AND (LOWER(mail) = 'jdoe@example.com') LIMIT 1
  ^[[1m^[[36mProject Load (0.2ms)^[[0m  ^[[1mSELECT `projects`.* FROM `projects` WHERE `projects`.`identifier` IS NULL LIMIT 1^[[0m
MailHandler: missing information from John Doe: Unable to determine target project

Here is the scraping script that we use:

#!/bin/bash

# Purpose:   Fetch emails and import into Redmine
# Reference: http://www.redmine.org/projects/redmine/wiki/RedmineReceivingEmails

cd /opt/redmine

sudo -u www-data bundle exec rake -f /opt/redmine/Rakefile redmine:email:receive_imap \
    RAILS_ENV="development" \
    host=mail.example.com username=support@example.com password=PASSWORD \
    move_on_success=accepted_emails move_on_failure=rejected_emails \
    project=unassigned category=unassigned tracker=support \
    no_permission_check=1 unknown_user=accept port=993 ssl=1 \
    allow_override=project,tracker,status,priority,category,assigned_to,\
    fixed_version,start_date,due_date,estimated_hours,\
    done_ratio > /dev/null 2>&1

What is odd is that I can mark the email as unread, place it back in the Inbox and let the script try again and 9/10 times it will successfully pick it on the 2nd try. I haven't been running the scraping process in development mode long enough to see what the other failures log.

debug_options.patch Magnifier (702 Bytes) Jean-Philippe Lang, 2013-08-07 21:56

Associated revisions

Revision 12067
Added by Jean-Philippe Lang over 4 years ago

Use the default project when receiving an email with an invalid project keyword (#14491).

Revision 12068
Added by Jean-Philippe Lang over 4 years ago

Broken tests (#14491).

Revision 13457
Added by Toshi MARUYAMA over 3 years ago

use deep_dup to prevent losing mail receiving options (#14491)

Revision 13459
Added by Toshi MARUYAMA over 3 years ago

Merged r13457 from trunk to 2.6-stable (#14491)

use deep_dup to prevent losing mail receiving options.

History

#1 Updated by Francisco Ramirez almost 5 years ago

I found a problem when i was integrating VTiger and Redmine using email task in a workflow, the problem was that for some reason VTiger email make the value of project to get down one line when you leave spaces between the ":", oddly enought when I resent the same email in thunderbird it works well, I check the "original format ctrl-U" in thunderbird and I see the problem I'm describing, for example

project : my_dear_proyect

becames

project :
my_dear_proyect

And that was the problem, remove all spaces and perhaps you would solve your issue like I did :)

To the great Redmine Team, perhaps you should update your documentations with this little issue as a Note

#2 Updated by Deoren Moor almost 5 years ago

Francisco Ramirez wrote:

And that was the problem, remove all spaces and perhaps you would solve your issue like I did :)

Our users don't include those attributes, instead we rely on the provided script to force all incoming emails that are not already tagged (we do not have problems with those) into an "unassigned" project for manual processing.

Thanks for the tip though.

#3 Updated by Jean-Philippe Lang over 4 years ago

Deoren, that would be great if you could attach a raw email for which you've got the failure.

#4 Updated by Jean-Philippe Lang over 4 years ago

This queries shows that the project keyword is not properly parsed:

SELECT `projects`.* FROM `projects` WHERE `projects`.`identifier` IS NULL LIMIT 1

I've committed a fix in r12067 so that the default is used if an invalid/malformed project keyword is present in the email. But still, I'd really like to get your failling email to see what is going wrong.

#5 Updated by Deoren Moor over 4 years ago

Jean-Philippe Lang wrote:

This queries shows that the project keyword is not properly parsed:

Hi,

Sorry for the late reply. The emails that we have a problem with don't use keywords, they're sent by our users directly to a support address. Our goal with the provided scraping script was to force those emails into an unassigned project where we would review them and assign as needed.

Jean-Philippe Lang wrote:

Deoren, that would be great if you could attach a raw email for which you've got the failure.

I don't have one at the moment and it's hard to reproduce. Once I get one (it could be a while), should I email you a copy of it? I'm hesitant to post the entire raw email here due to spam/etc.

#6 Updated by Jean-Philippe Lang over 4 years ago

You can send it to me by email.

#7 Updated by Deoren Moor over 4 years ago

Jean-Philippe Lang wrote:

You can send it to me by email.

Thanks. I'm sending the email now. Both of the emails were successfully scraped once I marked them as unread and put them back into the Inbox for a second scrape attempt. I did not include the log entries from the successful processing, but let me know and I'll send those your way also.

#8 Updated by Jean-Philippe Lang over 4 years ago

  • Assignee set to Jean-Philippe Lang

Thanks, I've made a few attemps with the first email and they all suceeded. I really can't see why it fails sometimes. I'll merge some changes for 2.3.3 to get more debug information on this issue.

#9 Updated by Deoren Moor over 4 years ago

Jean-Philippe Lang wrote:

I really can't see why it fails sometimes.

Thanks for looking into this. Is there any additional information I can provide about the OS or environment, or is it looking more like a Redmine specific issue? Please let me know if I can help in any other way.

#10 Updated by Jean-Philippe Lang over 4 years ago

It looks like your project=unassigned option is lost. Can you try with the following patch applied. There's a slight change and some debug messages, thanks.

#11 Updated by Deoren Moor over 4 years ago

Jean-Philippe Lang wrote:

It looks like your project=unassigned option is lost. Can you try with the following patch applied. There's a slight change and some debug messages, thanks.

Patch applied and Apache restarted (what this box uses). I'll let you know as soon as I spot the problem again.

#12 Updated by Deoren Moor over 4 years ago

Jean-Philippe,

Since I've applied that patch you provided I've not seen any further emails rejected. I just wanted to post back and mention that. Thanks.

#13 Updated by Miquel Coll over 3 years ago

Deoren Moor wrote:

Jean-Philippe,

Since I've applied that patch you provided I've not seen any further emails rejected. I just wanted to post back and mention that. Thanks.

I have the same problem with lost project (using redmine 2.4.5). It's been working fine until a few days ago, and now no new issues are created. I've applied the patch that Jean-Phillipe suggested and now it works. Odd. My script to get emails by pop3 is this:

/usr/local/bin/rake /app/redmine-2.4.5/Rakefile redmine:email:receive_pop3 RAILS_ENV="production" host=myhost.com port=110 ssl=0 username=redmine password=secret project=hdk unknown_user=accept folder=inbox move_on_success=redmine-processats move_on_failure=redmine-error allow_override=project,tracker,status,priority,category,assigned_to,fixed_version,start_date,due_date,estimated_hours,done_ratio unknown_user=accept no_permission_check=1 --trace >> /app/redmine-2.4.5/log/cron_imap_`date +\%Y\%m\%d`.log &2>&1

#14 Updated by Toshi MARUYAMA over 3 years ago

  • Target version set to 2.6.0

#15 Updated by Jean-Philippe Lang over 3 years ago

  • Status changed from New to Closed

#16 Updated by Toshi MARUYAMA over 3 years ago

  • Status changed from Closed to Reopened

#17 Updated by Toshi MARUYAMA over 3 years ago

note-10 patch is not applied yet.

#18 Updated by Toshi MARUYAMA over 3 years ago

  • Status changed from Reopened to Closed
  • Resolution set to Fixed

Toshi MARUYAMA wrote:

note-10 patch is not applied yet.

It is committed in trunk r13457 and 2.6-stable r13459.

Also available in: Atom PDF