Redmine

Hi Gary Chen,

You are on the right track.

You need to create an API key in Redmine from:
Administration → Settings → API (make sure REST API is enabled),
then go to My account → API access key to generate or copy the key.
This key must be passed with the --key=... option so that Redmine can authenticate the incoming emails.

The command should look like this:

redmine-test:   "|/usr/local/rvm/rubies/ruby-3.2.3/bin/ruby /home/redmine-test/redmine-test/extra/mail_handler/rdm-mailhandler.rb --url https://localhost/redmine-test --key=YOUR_API_KEY --allow-override=all --no-check-certificate" 

If you look at the line in my original message for the postfix aliases file you will see it contains the argument

--key-file /home/redmine-test/redmine-test/config/mail_key

--key my-key

In any case, I tried using --key instead and it still fails with the message:

permission denied. Command output: Request was
denied by your Redmine server. Make sure that 'WS for incoming emails' is
enabled in application settings and that you provided the correct API key.

Is this working for you on a redmine where logins are required, even for public projects?

Receiving mails with the incoming mails webservice (which is used by the rdm-mailhandler.rb script) does not rely on the "regular" API permissions. Thus, for its authentication it does not matter if you require authentication for regular access or if you have set the projects as public or not.

However, you need to use the key of the mail webservice, not a regular API key. You can configure this in Redmine in Administration -> Settings -> Incoming emails. Here, you need to enable the Enable WS for incoming emails checkbox and generate a key for the mail webservice. This key must then be used in your rdm-mailhandler.rb invocation.

Right now, it appears that the mail WS is either disabled, or the key does not match, or you use the "wrong" Redmine (check the URL in the rdm-mailhandler.rb arguments)

holger mareck Thanks I'm not sure what you mean by a "regular API key". In any case, as explained in the original post, I already have enabled the WS for incoming emails checkbox, and the generated key matches the key stored in config/mail_key. That key is also the key specified (either directly, or via file path) in the rdm-mailhandler.rb arguments.

If I do the following:

$ /usr/local/rvm/rubies/ruby-3.2.3/bin/ruby /home/redmine-test/redmine_test/extra/mail_handler/rdm-mailhandler.rb 
    --url https://localhost/redmine-test --key <ws-api-key> --allow-override all --no-check-certificate <TestMail.txt

  Request was denied by your Redmine server. 
  Make sure that 'WS for incoming emails' is enabled in application settings and that you provided the correct API key.

response = Net::HTTPS.post_form(URI.parse(uri), data, headers, :no_check_certificate => no_check_certificate, :certificate_bundle => certificate_bundle)

This turned out to be an issue with an uncaught, or more likely a caught and ignored, exception. I think somewhere in the bowels of

activerecord-6.1.7.10/lib/active_record/persistence.rb

issue.save!.

We had a plugin (redmineup, redmine_contacts_helpdesk) which had a stylesheet asset, which loads assets into

public/plugin_assets/redmine_contacts_helpdesk/stylesheets

The stylesheet had a reference to

../../../../images/xxx.gif

which should have been to

../../../images/xxx.gif

That bogus reference caused some kind of load failure (file not found?), which caused the save! to not complete properly, although it was returning an html status of 200. Once the bad reference was corrected, things worked properly.

This should probably be filed as a bug with the passenger / rails folks.

Correction, the html response code was 201, not 200.

Thanks for this further analysis. This is quite an obscure issue indeed. If you have log entries and stack traces of the specific errors, we may also be able to improve resiliency here. At least it may help other people who come across similar issues to identify them.

My previous comment is incorrect and bogus.
I had modified rdm-mailhandler.rb to provide some diagnostic output, and in the process had added an attribute to the http request:

request.basic_auth('me','my-pw')

This appears to be the actual problem -- our apache server is not generally accessible, and requires a user login to access. So in order for a user to access redmine, they must log in to both the apache server and the redmine server. Mail fails to be delivered if basic_auth is not set to a valid user for the apache server. If basic_auth is set to a valid user for redmine which is not a valid user for the apache server, we get a 401 response (expected). If basic_auth is set to a valid apache user but the From: is not a valid redmine user (e.g. foo@example.com), we get a 422 error (expected).

In rdm-mailhandler.rb, self.post_form contains the following line:

  request.basic_auth url.user, url.password if url.user

When an email comes in, there is no authorized user; url.user is not set.
So no password is set for apache, and apache doesn't pass things on to redmine.
Even if url.user was set, the pw would be set to the redmine pw, not the apache one, although they might be the same.

If I force that line to:

request.basic_auth('valid-apache-user','valid-apache-user-pw')

The mail comes through, regardless of which apache user is used for the authorization.
Unfortunately, this requires modifying extra/mail_handler/rdm-mailhandler.rb and embedding the valid-apache-user and password in plaintext. The file permissions must be set to 610 in our case, but that will differ depending on how one has ruby and redmine set up.

Is there some way to set the request basic_auth to a key file similar to the redmine key-file param? That would at least allow for the file to be a little better protected (mode 600) and the pw to not be in plaintext.

Note that depending on what kind of authorization is set up for the apache server, the required request parameters might be different.

make that mode 640, not 610

With an unchanged rdm-mailhandler.rb, you should be able to specify a basic auth username and password in the URL to your Redmine. With your example from your very first post, this could look like this:

/usr/local/rvm/rubies/ruby-3.2.3/bin/ruby /home/redmine-test/redmine-test/extra/mail_handler/rdm-mailhandler.rb --url https://valid-apache-user:valid-apache-user-pw@localhost/redmine-test --key-file /home/redmine-test/redmine-test/config/mail_key --allow-override all --no-check-certificate

Unfortunately though, there is currently no way to hide this password from the process list. However required, you can "wrap" this invocation in another script which then contains the password and only needs to be readable by your mailserver user and which passes stdin along. You could then configure your postfix to invoke this script. Or you update your postfix config file which currently contains the URL (and thus will contain the password) to only be readable by root.

I'm not sure that will work. If the password is strong (contains certain special characters) it is rejected as invalid:
```
/usr/local/rvm/rubies/ruby-3.2.3/lib/ruby/3.2.0/uri/rfc3986_parser.rb:66:in `split': bad URI: "https://valid-apache-user:xxx@localhost/redmine-test/mail_handler" (URI::InvalidURIError)
from /usr/local/rvm/rubies/ruby-3.2.3/lib/ruby/3.2.0/uri/rfc3986_parser.rb:71:in `parse'
from /usr/local/rvm/rubies/ruby-3.2.3/lib/ruby/3.2.0/uri/common.rb:193:in `parse'
from /path-to-redmine-root/extra/mail_handler/rdm-mailhandler.rb:171:in `submit'
from /path-to-redmine-root/extra/mail_handler/rdm-mailhandler.rb:218:in `<main>'
```