Email notifications won't send after migrating to a new AWS instance with encrypted volumes

Added by shaneoh * 3 months ago

We've been running Redmine for many years for our company and love it - it's an absolutely critical system for us. We manage it ourselves in a simple AWS instance. We've upgraded many times over the years, as well as migrating to new instances for various reasons, but current details are:

- Redmine 4.1.1
- Ruby 2.6.5-p114
- Rails 5.2.4.2

All running on an Ubuntu 20.04.3 server.

As the title suggests I've struck a pretty strange issue. We've discovered the server's volume isn't encrypted, so migrated this. But now our email notifications won't send. The Redmine production log doesn't really help, it really just tells me that the execution is expired.

We use an app specific password, simply using Gmail to send these notifications. I tried generating a new one and also tried using the master password for the email account but I get the same issue. None of the other server logs have anything coming up when these failures happen. I get the same issue when I try to send a test email as well, along with the UI message "An error occurred while sending mail (execution expired)".

Has anyone else had anything similar or have any ideas about what could cause this?

Started PATCH "/issues/9833" for 52.52.52.52 at 2021-11-09 02:17:20 +0000
Processing by IssuesController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"xjNDkVDrW6jHwgnkhEwytherlh3y7VSW//mP5WRH39Akuqg0fgq1mzXOyxpGmqe/HZeW0AQAhInsIhbEqpSPsw==", "form_update_triggered_by"=>"", "issue"=>{"is_private"=>"0", "project_id"=>"28", "tracker_id"=>"5", "subject"=>"Move redmine.myserver.com to encrypted volumes", "description"=>"", "status_id"=>"2", "priority_id"=>"4", "assigned_to_id"=>"41", "parent_issue_id"=>"9799", "start_date"=>"2021-09-07", "due_date"=>"", "estimated_hours"=>"", "done_ratio"=>"0", "custom_field_values"=>{"1"=>""}, "tag_list"=>[""], "notes"=>"And yet another test email", "private_notes"=>"0", "lock_version"=>"8"}, "time_entry"=>{"hours"=>"", "activity_id"=>"13", "comments"=>""}, "suppress_mail"=>"0", "last_journal_id"=>"37453", "commit"=>"Submit", "prev_issue_id"=>"9890", "next_issue_id"=>"9825", "issue_position"=>"8", "issue_count"=>"84", "id"=>"9833"}
  Current user: username (id=41)
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: ca85727c-e5e2-4872-a587-c51feb03cc33) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x00007f7c15460670 @uri=#<URI::GID gid://redmine-app/User/41>>, #<GlobalID:0x00007f7c1546ba48 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 9fc3c49f-dcb6-448e-abc4-833a4e05ad6b) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x00007f7c1548dd78 @uri=#<URI::GID gid://redmine-app/User/8>>, #<GlobalID:0x00007f7c1548d440 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 05ec5cd9-e840-4056-9763-238a7990b347) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x00007f7c154a6530 @uri=#<URI::GID gid://redmine-app/User/40>>, #<GlobalID:0x00007f7c154a58b0 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: e49febfb-922d-4306-9fbd-e2dc1d41d4e6) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101b1476f8 @uri=#<URI::GID gid://redmine-app/User/55>>, #<GlobalID:0x000056101b1d3180 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 054ef354-3481-47cf-9fbb-3cb4d9fccdcb) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101bc17358 @uri=#<URI::GID gid://redmine-app/User/60>>, #<GlobalID:0x000056101bb828e8 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 0b19ed0a-a458-431d-983f-088bd9fa71ad) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101bf11d88 @uri=#<URI::GID gid://redmine-app/User/92>>, #<GlobalID:0x000056101bf97870 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 646bed30-3632-4ba6-8198-f0380af3ba6e) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101beb7c70 @uri=#<URI::GID gid://redmine-app/User/94>>, #<GlobalID:0x000056101beba1f0 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 77fc394f-5504-4fb8-bf6c-8f871bd98d6e) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101cbbee78 @uri=#<URI::GID gid://redmine-app/User/98>>, #<GlobalID:0x000056101cbdadf8 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 872212e5-225b-4d33-a32c-d10f0e1466d9) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101cd26fe0 @uri=#<URI::GID gid://redmine-app/User/101>>, #<GlobalID:0x000056101cd44310 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
Redirected to http://redmine.myserver.com/issues/9833?issue_count=84&issue_position=8&next_issue_id=9825&prev_issue_id=9890
Completed 302 Found in 159ms (ActiveRecord: 37.3ms)
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33] Performing ActionMailer::DeliveryJob (Job ID: ca85727c-e5e2-4872-a587-c51feb03cc33) from Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101e451fa8 @uri=#<URI::GID gid://redmine-app/User/41>>, #<GlobalID:0x000056101e450b58 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b] Performing ActionMailer::DeliveryJob (Job ID: 9fc3c49f-dcb6-448e-abc4-833a4e05ad6b) from Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000056101e493a98 @uri=#<URI::GID gid://redmine-app/User/8>>, #<GlobalID:0x000056101e4c3590 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b]   Rendering mailer/issue_edit.text.erb within layouts/mailer
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33]   Rendering mailer/issue_edit.text.erb within layouts/mailer
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b]   Rendered mailer/_issue.text.erb (15.6ms)
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b]   Rendered mailer/issue_edit.text.erb within layouts/mailer (16.2ms)
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b]   Rendering mailer/issue_edit.html.erb within layouts/mailer
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b]   Rendered mailer/_issue.html.erb (0.7ms)
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b]   Rendered mailer/issue_edit.html.erb within layouts/mailer (2.5ms)
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33]   Rendered mailer/_issue.text.erb (42.5ms)
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33]   Rendered mailer/issue_edit.text.erb within layouts/mailer (43.0ms)
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33]   Rendering mailer/issue_edit.html.erb within layouts/mailer
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33]   Rendered mailer/_issue.html.erb (3.0ms)
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33]   Rendered mailer/issue_edit.html.erb within layouts/mailer (6.7ms)
Started GET "/issues/9833" for 52.52.52.52 at 2021-11-09 02:17:21 +0000
Processing by IssuesController#show as HTML
  Parameters: {"id"=>"9833"}
  Current user: username (id=41)
  Rendering issues/show.html.erb within layouts/base
  Rendered issues/_action_menu.html.erb (2.8ms)
  Rendered plugins/redmineup_tags/app/views/issues/_tags.html.erb (1.1ms)
  Rendered issue_relations/_form.html.erb (1.0ms)
  Rendered issues/_relations.html.erb (1.4ms)
  Rendered issues/tabs/_history.html.erb (17.3ms)
  Rendered common/_tabs.html.erb (18.1ms)
  Rendered issues/_action_menu.html.erb (1.7ms)
  Rendered issues/_trackers_description.html.erb (0.4ms)
  Rendered issues/_form_custom_fields.html.erb (0.2ms)
  Rendered issues/_attributes.html.erb (16.5ms)
  Rendered plugins/redmineup_tags/app/views/issues/_tags_form.html.erb (0.4ms)
  Rendered issues/_form.html.erb (35.1ms)
  Rendered plugins/redmine_silencer/app/views/hooks/_silencer_suppress_mail.html.erb (0.2ms)
  Rendered attachments/_form.html.erb (1.4ms)
  Rendered issues/_edit.html.erb (44.5ms)
  Rendered issues/_action_menu_edit.html.erb (46.6ms)
  Rendered plugins/redmineup_tags/app/views/issues/_tags_sidebar.html.erb (8.2ms)
  Rendered issues/_sidebar.html.erb (11.7ms)
  Rendered watchers/_watchers.html.erb (3.1ms)
  Rendered issues/show.html.erb within layouts/base (115.4ms)
  Rendered plugins/redmineup_tags/app/views/tags/_additional_assets.html.erb (0.3ms)
  Rendered plugins/clipboard_image_paste/app/views/clipboard_image_paste/_headers.html.erb (0.2ms)
  Rendered plugins/redmineup_tags/app/views/tags/_select2_transformation_rules.html.erb (0.2ms)
  Rendered plugins/clipboard_image_paste/app/views/clipboard_image_paste/_add_form.html.erb (1.7ms)
Completed 200 OK in 204ms (Views: 112.9ms | ActiveRecord: 58.2ms)
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b] Email delivery error: execution expired
[ActiveJob] [ActionMailer::DeliveryJob] [9fc3c49f-dcb6-448e-abc4-833a4e05ad6b] Performed ActionMailer::DeliveryJob (Job ID: 9fc3c49f-dcb6-448e-abc4-833a4e05ad6b) from Async(mailers) in 30058.66ms
[ActiveJob] [ActionMailer::DeliveryJob] [05ec5cd9-e840-4056-9763-238a7990b347] Performing ActionMailer::DeliveryJob (Job ID: 05ec5cd9-e840-4056-9763-238a7990b347) from Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x00007f7c1540d290 @uri=#<URI::GID gid://redmine-app/User/40>>, #<GlobalID:0x00007f7c1540ca20 @uri=#<URI::GID gid://redmine-app/Journal/37454>>
[ActiveJob] [ActionMailer::DeliveryJob] [ca85727c-e5e2-4872-a587-c51feb03cc33] Email delivery error: execution expired