Project

General

Profile

RedmineReceivingEmails » History » Version 93

Joan Cervan, 2023-07-10 14:31
In my own experience: use '' if it fails, ex: 'my-password'

1 92 Alexander Meindl
h1. Receiving emails
2
3
{{>toc}}
4
5
Redmine can be configured to allow issue creation or comments via email. It is also able to recognize and incorporate email replies to forum messages.
6
7
h2. Setup
8
9
You can configure Redmine to receive emails in one of the following ways:
10
11
* Forwarding emails from your email server:
12
13
  * Pros: works with a remote mail server, email are processed instantly, fast (no environment reloading)
14
  * Cons: needs some configuration on your mail transfer agent (eg. Postfix, Sendmail...)
15
16
* Fetching emails from an IMAP or POP3 server:
17
18
  * Pros: easy to setup, no need to configure your MTA, works with a remote mail server
19
  * Cons: emails are not processed instantly (a cron job needs to be added to read emails periodically)
20
21
* Reading emails from standard input:
22
23
  * Pros: fine for testing purpose
24
  * Cons: slow (the environment is reloaded each time an email is read), needs some configuration on your MTA
25
26
h3. Forwarding emails from your email server
27
28
A standalone script can be used to forward incoming emails from your mail server.
29
This script reads a raw email from the standard input and forwards it to Redmine via a HTTP request.
30
It can be found in your redmine directory: @extra/mail_handler/rdm-mailhandler.rb@.
31
32
In order to use it, you have to enable the API that receive emails:
33
Go to _Application settings_ -> _Incoming emails_, check *Enable WS for incoming emails* and enter or generate a secret key.
34
35
36
37
38
39
40
41
42
43
Copy @rdm-mailhandler.rb@ to your mail server, make sure its permissions allow execution (@chmod +x rdm-mailhandler.rb@), and configure your MTA (Mail Transport Agent).
44
45
Usage:
46
47
<pre>
48
Usage: rdm-mailhandler.rb [options] --url=<Redmine URL> --key=<API key>
49
50
Required arguments:
51
    -u, --url URL               URL of the Redmine server
52
    -k, --key KEY               Redmine API key
53
54
General options:
55
        --key-file FILE         full path to a file that contains your Redmine
56
                                API key (use this option instead of --key if
57
                                you don't want the key to appear in the command
58
                                line)
59
        --no-check-certificate  do not check server certificate
60
        --certificate-bundle FILE
61
                                certificate bundle to use
62
    -h, --help                  show this help
63
    -v, --verbose               show extra information
64
    -V, --version               show version information and exit
65
66
User and permissions options:
67
        --unknown-user ACTION   how to handle emails from an unknown user
68
                                ACTION can be one of the following values:
69
                                * ignore: email is ignored (default)
70
                                * accept: accept as anonymous user
71
                                * create: create a user account
72
        --no-permission-check   disable permission checking when receiving
73
                                the email
74
        --default-group GROUP   add created user to GROUP (none by default)
75
                                GROUP can be a comma separated list of groups
76
        --no-account-notice     don't send account information to the newly
77
                                created user
78
        --no-notification       disable email notifications for the created
79
                                user
80
81
Issue attributes control options:
82
        --project-from-subaddress ADDR
83
                                select project from subaddress of ADDR found
84
                                in To, Cc, Bcc headers
85
    -p, --project PROJECT       identifier of the target project
86
    -s, --status STATUS         name of the target status
87
    -t, --tracker TRACKER       name of the target tracker
88
        --category CATEGORY     name of the target category
89
        --priority PRIORITY     name of the target priority
90
        --assigned-to ASSIGNEE  assignee (username or group name)
91
        --fixed-version VERSION name of the target version
92
        --private               create new issues as private
93
    -o, --allow-override ATTRS  allow email content to set attributes values
94
                                ATTRS is a comma separated list of attributes
95
                                or 'all' to allow all attributes to be
96
                                overridable (see below for details)
97
</pre>
98
99
See [[RedmineReceivingEmails#Issue-attributes|Issue-attributes]] for a list of values that can be used for the @--allow-override@ option.
100
101
Examples:
102
103
<pre>
104
  # No project specified. Emails MUST contain the 'Project' keyword:
105
  rdm-mailhandler.rb --url http://redmine.domain.foo --key secret
106
  
107
  # Fixed project and default tracker specified, but emails can override
108
  # both tracker and priority attributes:
109
  rdm-mailhandler.rb --url https://domain.foo/redmine --key secret \\
110
                     --project foo \\
111
                     --tracker bug \\
112
                     --allow-override tracker,priority
113
</pre>
114
115
Here is an example for a Postfix local alias entry:
116
117
<pre>
118
foo: "|/path/to/rdm-mailhandler.rb --url http://redmine.domain --key secret --project foo"
119
</pre>
120
121
This line should go in the aliases file, which is usually specified in @/etc/aliases@. If the location is unknown, use the command @postconf alias_maps@ to find out. After updating the aliases file, be sure to run @newaliases@ to alert Postfix of the new entry.
122
123
If your domain is setup as a virtual mailbox map (so that you use /etc/postfix/virtual_mailbox_maps to do mappings in the form @ user@example.com /path/example.com/user@) you should:
124
125
* create a mapping in @/etc/virtual@ like: @ foo@example.org foo@
126
* modify @/etc/postfix/main.cf@ to specify a transport file: @transport_maps = hash:/etc/postfix/transport@
127
* within the transport file add a line like: @ foo@example.org local:@
128
129
*Explanation:* - When you define virtual_mailbox_maps for a domain the default transport is virtual, which means specifying a local alias in @/etc/postfix/virtual@ will fail (with "unknown user"). To fix this, we override the default transport by specifying a local transport for the email address in question, which means the local alias will resolve correctly, and your script will be executed.
130
131
h3. Fetching emails from an IMAP server
132
133
A rake task (@redmine:email:receive_imap@) can be used to fetch incoming emails from an IMAP server. When you run the rake command from a cron job you can include the switch @-f /path/to/redmine/appdir/Rakefile@ on the rake command, because otherwise the rakefile is not found. This is an example line for a cron file that fetches mails every 30 minutes:
134
135
<pre>
136
*/30 * * * * redmineuser rake -f /path/to/redmine/appdir/Rakefile redmine:email:receive_imap RAILS_ENV="production" host=imap.foo.bar username=redmine@somenet.foo password=xxx 
137
</pre>
138
139
If your setup is working, but you receive mails from the cron daemon, you can suppress the output from the rake command by adding the --silent switch. That should stop cron sending mails on every execution of the command.
140
141
<pre>
142
*/30 * * * * redmineuser rake -f /path/to/redmine/appdir/Rakefile --silent redmine:email:receive_imap RAILS_ENV="production" host=imap.foo.bar username=redmine@somenet.foo password=xxx 
143
</pre>
144
145
The command has to go on a single line in your cronfile. Also see the other examples below, which only show the rake commands without the @-f@ option and without the cron part. 
146
147
For Windows as server "pycron":http://www.kalab.com/freeware/pycron/pycron.htm can be used to schedule a fetch task.
148
149
It can be necessary that you open the firewall on the machine for outgoing TCP connections to IMAP port 143.
150
151
Available IMAP options:
152
<pre>
153
  host=HOST                IMAP server host (default: 127.0.0.1)
154
  port=PORT                IMAP server port (default: 143)
155
  ssl=SSL                  Use SSL? (default: false)
156
  starttls=STARTTLS        Use STARTTLS? (default: false)
157
  username=USERNAME        IMAP account
158
  password=PASSWORD        IMAP password
159
  folder=FOLDER            IMAP folder to read (default: INBOX)
160
  move_on_success=MAILBOX  move emails that were successfully received
161
                           to MAILBOX instead of deleting them
162
  move_on_failure=MAILBOX  move emails that were ignored to MAILBOX
163
</pre>
164
165
User and permissions options:
166
<pre>
167
  unknown_user=ACTION      how to handle emails from an unknown user
168
                           ACTION can be one of the following values:
169
                           ignore: email is ignored (default)
170
                           accept: accept as anonymous user
171
                           create: create a user account
172
  no_permission_check=1    disable permission checking when receiving
173
                           the email
174
  no_account_notice=1      disable new user account notification
175
  default_group=foo,bar    adds created user to foo and bar groups
176
</pre>
177
178
Issue attributes control options:
179
<pre>
180
  project=PROJECT          identifier of the target project
181
  project_from_subaddress=ADDR
182
                           select project from subaddress of ADDR found
183
                           in To, Cc, Bcc headers
184
  status=STATUS            name of the target status
185
  tracker=TRACKER          name of the target tracker
186
  category=CATEGORY        name of the target category
187
  priority=PRIORITY        name of the target priority
188
  assigned_to=ASSIGNEE     assignee (username or group name)
189
  fixed_version=VERSION    name of the target version
190
  private                  create new issues as private
191
  allow_override=ATTRS     allow email content to set attributes values
192
                           ATTRS is a comma separated list of attributes
193
                           or 'all' to allow all attributes to be overridable
194
</pre>
195
196
See [[RedmineReceivingEmails#Issue-attributes|Issue-attributes]] for a list of values that can be used for the @allow-override@ option.
197
198
Examples for the rake command:
199
200
<pre>
201
  # No project specified. Emails MUST contain the 'Project' keyword:
202
  
203
  rake redmine:email:receive_imap RAILS_ENV="production" \\
204
    host=imap.foo.bar username=redmine@somenet.foo password=xxx
205
206
207
  # Fixed project and default tracker specified, but emails can override
208
  # both tracker and priority attributes:
209
  
210
  rake redmine:email:receive_imap RAILS_ENV="production" \\
211
    host=imap.foo.bar username=redmine@somenet.foo password=xxx ssl=1 \\
212
    project=foo \\
213
    tracker=bug \\
214
    allow_override=tracker,priority
215
216
  # Move successful emails to the 'read' mailbox and failed emails to
217
  # the 'failed' mailbox
218
  
219
  rake redmine:email:receive_imap RAILS_ENV="production" \\
220
    host=imap.foo.bar username=redmine@somenet.foo password=xxx \\
221
    move_on_success=read move_on_failure=failed
222
223
</pre>
224
225
226
Ignored emails are marked as 'Seen' but are not deleted from the IMAP server--these include unknown user, unknown project and emails from the redmine emission account.
227
228
The option _allow_override_ is not only for overriding default values given to rake, but for every attribute in a mail. If you want to override the tracker in your mail you have to add _allow_override=tracker_ as a parameter.
229
230
h3. Fetching emails from a POP3 server
231
232
A rake task (@redmine:email:receive_pop3@) can be used to fetch incoming emails from a POP3 server.
233
234
Available POP3 options:
235
<pre>
236
  host=HOST                POP3 server host (default: 127.0.0.1)
237
  port=PORT                POP3 server port (default: 110)
238
  username=USERNAME        POP3 account
239 93 Joan Cervan
  password=PASSWORD        POP3 password (use '' if it fails, ex: 'my-password')
240 92 Alexander Meindl
  apop=1                   use APOP authentication (default: false)
241
  delete_unprocessed=1     delete messages that could not be processed
242
                           successfully from the server (default
243
                           behaviour is to leave them on the server)
244
</pre>
245
246
See the [[RedmineReceivingEmails#Fetching-emails-from-an-IMAP-server|IMAP rake task]] above for issue attributes control options.
247
248
h3. Reading emails from standard input
249
250
A rake task (@redmine:email:read@) can be used to read a single raw email from the standard input.
251
252
See the [[RedmineReceivingEmails#Fetching-emails-from-an-IMAP-server|IMAP rake task]] above for issue attributes control options.
253
254
Examples:
255
256
<pre>
257
  # No project specified. Emails MUST contain the 'Project' keyword:
258
  rake redmine:email:read RAILS_ENV="production" < raw_email
259
260
  # Fixed project and default tracker specified, but emails can override
261
  # both tracker and priority attributes:
262
  rake redmine:email:read RAILS_ENV="production" \\
263
                  project=foo \\
264
                  tracker=bug \\
265
                  allow_override=tracker,priority < raw_email
266
</pre>
267
268
The option _allow_override_ is not only for overriding default values given to rake, but for every attribute in a mail. If you want to override the tracker in your mail you have to add _allow_override=tracker_ as a parameter.
269
270
h3. Enabling unknown users to create issues by email
271
272
Redmine has a feature that provides the ability to accept incoming emails from unknown users. In order to use this feature, an extra parameter has to be included:
273
<pre>
274
unknown_user=ACTION     how to handle emails from an unknown user where ACTION can be one of the following values:
275
                        ignore: the email is ignored (default)
276
                        accept: the sender is considered as an anonymous user and the email is accepted
277
                        create: a user account is created for the sender (username/password are sent back to the user) and the email is accepted
278
</pre>
279
280
Permissions have to be consistent with the chosen option. E.g. if you choose 'create', the 'Non member' role must have the 'Add issues' permission so that an issue can be created by an unknown user via email. If you choose 'accept', the 'Anonymous' role must have this permission.
281
282
If you receive emails via the rake task, the unknown-user option has to be written as:
283
<pre>
284
 unknown_user=[ignore|accept|create]
285
</pre>
286
287
You can disable permission checking using the 'no_permission_check' option:
288
<pre>
289
no_permission_check=1   disable permission checking when receiving the email
290
</pre>
291
292
This, together with the 'unknown-user', provides the ability to let anyone submit emails to a private project. For example:
293
<pre>
294
rdm-mailhandler.rb --unknown-user accept --no-permission-check --project=foo
295
</pre>
296
297
will let anyone submit emails to a private project 'foo'.
298
299
TODO: Is this true and is this related to the @no_permission_check@ option?:
300
> Since Redmine 0.9 the project doesn't have to be public, but authentication required in the Administration-> Settings->Authentication tab has to be unchecked.
301
302
If you do not want an "new account notification email" sent to every newly created user by rdm-mailhandler you must add the option "--no-account-notice". Is implemneted with version#60 in issue #11498. Now an example:
303
<pre>
304
rdm-mailhandler.rb --unknown-user accept --no-permission-check --project=foo --no-account-notice
305
</pre>
306
307
h3. Schedule email receiving with Rufus Scheduler
308
309
Instead of using a cron to trigger the Rake task to retrieve incoming email, you can use Rufus Scheduler which will run the Rake task in the same process as Redmine.
310
This is using less resources and faster than calling the Rake task from a cron (or from the Task Scheduler on Windows).
311
312
To do that, install rufus-scheduler gem:
313
<pre>gem install rufus-scheduler</pre>
314
315
Create a what_you_want.rb file in /config/initializers/ and put the following content in your file. (Here for a POP3 account. Change it with your own parameters and own task.)
316
<pre>
317
require 'rubygems'
318
require 'rake'
319
require 'rufus-scheduler'
320
321
load File.join(Rails.root, 'Rakefile')
322
323
ENV['host']='pop.toto.com'
324
ENV['port']='110'
325
ENV['ssl']='SSL'
326
ENV['username']='redmine@toto.com'
327
ENV['password']='azerty123456'
328
329
scheduler = Rufus::Scheduler.new
330
# Check emails every 10 mins
331
scheduler.interval '10m' do
332
  task = Rake.application['redmine:email:receive_pop3']
333
  task.reenable
334
  task.invoke 
335
end
336
</pre>
337
338
Restart your Redmine instance and your incoming emails will be retrieved on schedule.
339
340
You can check "Rufus-Scheduler":https://github.com/jmettraux/rufus-scheduler for scheduling syntax.
341
342
h2. How it works
343
344
When receiving an email, Redmine uses the From address of the email to find the corresponding user. Emails received from unknown or locked users are ignored.
345
346
If the email subject contains something like "Re: *[xxxxxxx !#123]*" or "[!#123]", the email is processed as a reply and a note is added to issue !#123. Otherwise, a new issue is created.
347
348
Note that, in order to create an issue, all required custom fields must be provided. Without them, issue creation will fail. As an alternative you can ensure that every custom field has a default value which is then used during issue creation.
349
350
h3. Target project
351
352
The target project can be specified using the @project@ option when receiving emails.  This should be the identifier of the project and *not* the name.  You can easily find the identifier in the URL.
353
354
If you don't use this option, users have to specify in the email body which project the issue should be added to. This can be done by inserting a line in the email body like this: @"Project: foo"@.
355
356
Example (email body):
357
358
<pre>
359
This is a new issue that will be added to project foo.
360
Here we have the issue description
361
[...]
362
363
Project: foo
364
</pre>
365
366
You can set a default project using the @project@ option and let users override this default project by using the @allow-override@ option when receiving emails.
367
Example:
368
369
<pre>
370
  # Create issues on project foo by default
371
  rake redmine:email:receive_imap [...] project=foo allow_override=project
372
</pre>
373
374
Of course, user permissions are checked and this email would be ignored if the user who sent this email is not allowed to add issues to project foo.
375
Make sure that the target project doesn't use *required* custom fields with no default value for its issues, otherwise the creation of the issue will fail.
376
377
h3. Target Project from Email Sub-Address
378
379
This feature was introduced in ver 3.2.0 (SVN rev r14687). It may be preferable to have one mailbox for all incoming emails to a Redmine instance, instead of one per project. When receiving emails at redmine@somenet.foo, an email sent to redmine+foo@somenet.foo will cause the issue to be added to project with identifier foo.
380
381
To enable this feature add the @project_from_subaddress=redmine@somenet.foo@ to the command e.g.
382
383
<pre>
384
    rake redmine:email:receive_imap [...] project_from_subaddress=redmine@somenet.foo
385
</pre> 
386
387
If a project is included in the command-line then it becomes the default project that will be used for emails sent to @redmine@somenet.foo@. So with the command:
388
389
<pre>
390
    rake redmine:email:receive_imap [...] project=bar project_from_subaddress=redmine@somenet.foo
391
</pre> 
392
393
emails to @redmine@somenet.foo@ or @redmine+bar@somenet.foo@ will raise issues in project bar; emails to @redmine+foo@somenet.foo@ will go into project foo.
394
395
Some email providers include support for sub-addresses, as per RFCs 3598, 5233. Where this is not supported, forwarding mail addresses to the 'central' address, may work (it does in my case), ie set up an email address forwarding all mails to @redmine@somenet.foo@ from @redmine+foo@somenet.foo@, @redmine+bar@somenet.foo@ etc. This is easy to administer and does not require fetching emails from multiple accounts, each with its own account credentials.
396
397
h3. Issue attributes
398
399
Based on the options you use when receiving emails (see @allow-override@ option), users may be able to override some attributes when submitting an issue. 
400
401
Note: prior to Redmine version:"3.2.0" (#20543) some attributes were _always_ overridable, but this was not documented. This behavior has been changed and all attributes have to be explicitly listed with the @allow_override@ option to be overridable. For those who want all attributes to be overridable, @allow_override=all@ can now be used (#20543).
402
403
Once @allow_override@ has been configured, you can override those attributes by using the appropriate keywords in the email body.
404
405
Example keyword list:
406
407
* @Project@
408
* @Tracker@
409
* @Status@
410
* @Category@
411
* @Priority@
412
* @Assigned to@
413
* @Start date@
414
* @Due date@
415
* @Target version@
416
* @Estimated hours@
417
* @Done ratio@
418
* @<Custom field name>@
419
420
The values available are the ones of the context. E.g. @Status@ available (for this Tracker and this Project) are labels in the localized language, exactly as displayed in the user interface or system default language (even with spaces, without quoting).
421
422
Example (email body):
423
424
<pre>
425
This is a new issue that overrides a few attributes
426
[...]
427
428
Project: foo
429
Tracker: Bug
430
Priority: Urgent
431
Status: Resolved
432
</pre>
433
434
The format for keywords differs from the allowed attribute values for the @allow_override@ option:
435
436
* @project@
437
* @tracker@
438
* @status@
439
* @category@
440
* @priority@
441
* @assigned_to@
442
* @start_date@
443
* @due_date@
444
* @fixed_version@ (aka. Target version)
445
* @estimated_hours@
446
* @done_ratio@
447
* @<custom_field_name>@
448
449
Multiple attribute values can be specified in order to allow only certain keywords to be used.
450
451
Example:
452
453
<pre>
454
  # Allow overriding project, tracker, status & priority
455
  rake redmine:email:receive_imap [...] allow_override=project,tracker,status,priority
456
</pre>
457
458
h3. Watchers
459
460
If the user who sends the email has the 'Add issue watchers' permission, users that are in To or Cc field of the email are automatically added as watchers of the created issue.
461
462
Watchers are added only when the issue is created. To or Cc fields are ignored on replies. See #7017 and #8009.
463
464
h3. Email format and attachments
465
466
Redmine tries to use the plain text part of the email to fill the description of the issue.
467
If a HTML-only email is received, HTML tags are removed from its body.
468
469
Email attachments are automatically attached to the issue, unless their size exceeds the [[RedmineSettings#Attachment-max-size|maximum attachment size]] defined in the application settings.
470
471
h3. Truncate emails
472
473
The Administrator's settings may be used to automatically truncate emails, for example to eliminate quoted messages in forum replies. To do this, set the outgoing email header to something like @--Reply above this line--@ in the Email notifications settings. Then in the Incoming emails settings, enter the same line into the box "Truncate emails after one of these lines." (It is also possible to allow regex to be truncated)
474
475
h2. See also
476
477
* [[MailhandlerSubAddress|Mailhandler Frontend for SubAddressing]]: How to implement sub-addressing in Redmine <3.2