Subject: [PATCH] Feature: move to modern authentication(OAuth 2.0) from IMAP #84 --- Index: .gitignore IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/.gitignore b/.gitignore --- a/.gitignore (revision b25a16d96fbe8409a131950b41b972e10b65bdab) +++ b/.gitignore (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -50,3 +50,4 @@ /config/master.key /config/credentials.yml.enc +/config/email_oauth2*.yml Index: Gemfile IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/Gemfile b/Gemfile --- a/Gemfile (revision b25a16d96fbe8409a131950b41b972e10b65bdab) +++ b/Gemfile (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -21,6 +21,8 @@ gem 'net-imap', '~> 0.3.9' gem 'net-pop', '~> 0.1.2' gem 'net-smtp', '~> 0.3.3' +gem 'oauth2', '~> 2.0' +gem 'gmail_xoauth', '~> 0.4.3' gem 'rexml', require: false if Gem.ruby_version >= Gem::Version.new('3.0') # Windows does not include zoneinfo files, so bundle the tzinfo-data gem Index: doc/GMAIL_IMAP_OAUTH.md IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/doc/GMAIL_IMAP_OAUTH.md b/doc/GMAIL_IMAP_OAUTH.md new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/doc/GMAIL_IMAP_OAUTH.md (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,49 @@ +# Gmail IMAP OAuth2 Setup + +This guide explains how to authorize Redmine to access a Gmail mailbox via IMAP using OAuth2. + +## Enable IMAP in Gmail +1. Open Gmail and go to **Settings** → **See all settings**. +2. Under **Forwarding and POP/IMAP**, enable **IMAP access**. + +## Configure the OAuth consent screen +1. Visit [Google Cloud Console](https://console.cloud.google.com/). +2. Create or select a project. +3. Open **APIs & Services** → **OAuth consent screen** and complete the configuration. + +## Create an OAuth client +1. In **APIs & Services** → **Credentials**, create a new **OAuth client ID** of type **Desktop app**. +2. Note the generated **Client ID** and **Client Secret**. + +## Required scope +The rake task requests the following scope when authorizing: + +``` +https://mail.google.com/ +``` + +## Obtaining the refresh token +Run the rake task and follow the instructions: + +``` +rake redmine:email:google_oauth2_init token_file=/app/redmine/config/email_oauth2_mytokenname client=CLIENT_ID secret=CLIENT_SECRET +``` + +After authorization, the task stores the access and refresh tokens in `config/email_oauth2_mytokenname.yml`. + +## Receiving mail +Fetch messages with OAuth2 credentials instead of a password: + +```sh +rake redmine:email:receive_imap_oauth2 token_file=/app/redmine/config/email_oauth2_mytokenname \ + host=HOST username=EMAIL +``` +The task accepts the `ssl` and `starttls` environment variables. SSL/TLS is +enabled by default; set `ssl=0` to disable it. When SSL is disabled you can +enable STARTTLS with `starttls=1` (the option is ignored if `ssl` is enabled). + +## Revoking consent +To revoke the authorization and obtain a new refresh token: +1. Visit [Google Account Permissions](https://myaccount.google.com/permissions). +2. Remove the application from **Third-party apps with account access**. +3. Run the rake task again. Index: doc/O365_IMAP_OAUTH.md IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/doc/O365_IMAP_OAUTH.md b/doc/O365_IMAP_OAUTH.md new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/doc/O365_IMAP_OAUTH.md (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,43 @@ +# Office 365 IMAP OAuth2 Setup + +This guide explains how to authorize Redmine to access an Office 365 mailbox via IMAP using OAuth2. + +## Register an Azure application +1. Sign in at [portal.azure.com](https://portal.azure.com/). +2. Create a **New registration** allowing any account type. +3. On the initial form, use `http://localhost/` as the redirect URI. +4. Under **Authentication**, choose **Mobile and desktop**, keep the same redirect URI and enable **public client**. +5. Under **API permissions**, add: + - `offline_access` + - `User.Read` + - `IMAP.AccessAsUser.All` + - `POP.AccessAsUser.All` + - `SMTP.Send` +6. Note the **Application (client) ID** and **Directory (tenant) ID**. +7. If creating a private app, generate a **client secret**; public apps can omit this. + +## Initializing the token +Run the rake task interactively to obtain the refresh token: + +```sh +rake redmine:email:o365_oauth2_init token_file=/app/redmine/config/email_oauth2_mytokenname \ + client=CLIENT_ID tenant=TENANT_ID secret=CLIENT_SECRET +``` + +Tokens are stored in `config/email_oauth2_mytokenname.yml` and `config/email_oauth2_mytokenname_client.yml`. + +## Receiving mail +Use the OAuth token instead of a password when fetching messages: + +```sh +rake redmine:email:receive_imap_oauth2 token_file=/app/redmine/config/email_oauth2_mytokenname \ + host=HOST username=EMAIL +``` +Other parameters match those of `redmine:email:receive_imap`. + +SSL/TLS is enabled by default. Pass `ssl=0` to disable it and, if desired, +enable explicit TLS with `starttls=1`. The `starttls` option is ignored when +`ssl` is enabled. + +## Revoking access +To revoke the grant and start over, remove the application from [Microsoft account permissions](https://myaccount.microsoft.com/consents) and delete the token files before re-running the initialization task. Index: lib/redmine/email_oauth_helper.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/lib/redmine/email_oauth_helper.rb b/lib/redmine/email_oauth_helper.rb new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/lib/redmine/email_oauth_helper.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,31 @@ +# frozen_string_literal: true + +require 'uri' +require 'cgi' + +module Redmine + module EmailOauthHelper + # Read a full redirect URL from STDIN and extract ?code=... + # Re-prompts until valid. + def self.read_oauth_code + loop do + auth_resp = STDIN.gets&.strip + if auth_resp.nil? || auth_resp.empty? + puts 'Please enter the full redirect URL:' + next + end + + begin + uri = URI.parse(auth_resp) + code = CGI.parse(uri.query.to_s)['code']&.first + if code.nil? || code.empty? + raise URI::InvalidURIError + end + return code + rescue StandardError + puts 'Invalid URL. Please enter the full redirect URL:' + end + end + end + end +end Index: lib/redmine/imap.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/lib/redmine/imap.rb b/lib/redmine/imap.rb --- a/lib/redmine/imap.rb (revision b25a16d96fbe8409a131950b41b972e10b65bdab) +++ b/lib/redmine/imap.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -16,7 +16,6 @@ # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. - require 'net/imap' module Redmine @@ -28,12 +27,20 @@ ssl = !imap_options[:ssl].nil? starttls = !imap_options[:starttls].nil? folder = imap_options[:folder] || 'INBOX' + auth_type = imap_options[:auth_type] || 'LOGIN' imap = Net::IMAP.new(host, port, ssl) if starttls imap.starttls end - imap.login(imap_options[:username], imap_options[:password]) unless imap_options[:username].nil? + if auth_type == "XOAUTH2" + require 'gmail_xoauth' unless defined?(Net::IMAP::XOauth2Authenticator) && Net::IMAP::XOauth2Authenticator.class == Class + end + if auth_type == "LOGIN" + imap.login(imap_options[:username], imap_options[:password]) unless imap_options[:username].nil? + else + imap.authenticate(auth_type, imap_options[:username], imap_options[:password]) unless imap_options[:username].nil? + end imap.select(folder) imap.uid_search(['NOT', 'SEEN']).each do |uid| msg = imap.uid_fetch(uid,'RFC822')[0].attr['RFC822'] Index: lib/tasks/email_oauth.rake IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/lib/tasks/email_oauth.rake b/lib/tasks/email_oauth.rake new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/lib/tasks/email_oauth.rake (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,367 @@ +# Redmine - project management software +# Copyright (C) 2006-2022 Jean-Philippe Lang +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +# +# OAuth2 IMAP fetch tasks (O365 & Google) + +require 'net/imap' +require 'oauth2' +require 'uri' +require 'cgi' +require 'yaml' +require_relative '../redmine/email_oauth_helper' + +# XOAUTH2 helper (no-op als al geladen) +begin + require 'gmail_xoauth' +rescue LoadError + # ignore +end + +PERMITTED = { + permitted_classes: [Symbol], + permitted_symbols: %i[ + access_token refresh_token token_type expires_at expires_in scope + ext_expires_in mode header header_format param_name Bearer + ], + aliases: true +}.freeze + +# ------------------------------------------------------------------ +# helpers +# ------------------------------------------------------------------ +def secure_file(path) + return unless File.exist?(path) + return if File::ALT_SEPARATOR # Windows + File.chmod(0o600, path) +rescue StandardError + # ignore +end + +def env_bool(name, default=false) + v = ENV[name] + return default if v.nil? + %w[1 true yes y t].include?(v.to_s.strip.downcase) +end + +def normalize_token_file(path) + path ||= Rails.root.join('config', 'email_oauth2_token').to_s + base = File.basename(path) + return path if base.start_with?('email_oauth2') + File.join(File.dirname(path), "email_oauth2_#{base}") +end + +def mask_token(tok, keep: 6) + return "(nil)" if tok.to_s.empty? + return tok if tok.length <= keep*2 + head = tok[0, keep] + tail = tok[-keep, keep] + "#{head}...#{tail} (len=#{tok.length})" +end + +# Build an OAuth2 client from a config hash. +def build_oauth_client(config, redirect_set) + OAuth2::Client.new( + config['client_id'], + config['client_secret'], + site: config['site'], + authorize_url: config['authorize_url'], + token_url: config['token_url'], + redirect_uri: redirect_set ? config['redirect_uri'] : nil + ) +end + +# Exchange the authorization code for an access token. +def oauth_get_token(client, code, client_id, client_secret, redirect_uri) + params = {client_id: client_id, client_secret: client_secret} + params[:redirect_uri] = redirect_uri if redirect_uri + client.auth_code.get_token(code, **params) +end + +# Persist token and client configuration to disk with secure permissions. +def save_oauth_files(token_file, access_token, client_config) + File.write("#{token_file}.yml", access_token.to_hash.to_yaml) + secure_file("#{token_file}.yml") + File.write("#{token_file}_client.yml", client_config.to_yaml) + secure_file("#{token_file}_client.yml") +end + +# Abort if no refresh token was returned unless explicitly allowed. +def check_refresh_token!(token) + return unless token.refresh_token.to_s.empty? + + if ENV['allow_no_refresh'] == '1' + warn 'No refresh token returned; proceeding (token will expire).' + else + abort('No refresh token received; check application permissions (try prompt=consent).') + end +end + +namespace :redmine do + namespace :email do + + # ------------------------------------------------------------------ + # Office 365 Authorization Init + # ------------------------------------------------------------------ + desc "Init Office 365 authorization" + task :o365_oauth2_init => :environment do + token_file = normalize_token_file(ENV['token_file']) + client_id = ENV['client'] + client_secret = ENV['secret'] + tenant_id = ENV['tenant'] + redirect_uri = ENV['redirect_uri'].to_s.strip + redirect_set = !redirect_uri.empty? + + puts 'See doc/O365_IMAP_OAUTH.md for setup instructions.' + abort("Missing ENV client") if client_id.to_s.empty? + abort("Missing ENV secret") if client_secret.to_s.empty? + abort("Missing ENV tenant") if tenant_id.to_s.empty? + puts "WARN: no redirect_uri supplied; using app-registered default." unless redirect_set + + + # NB: scopes hier nog breed; snoeien kan later + scope = [ + "offline_access", + "https://outlook.office.com/User.Read", + "https://outlook.office.com/IMAP.AccessAsUser.All", + "https://outlook.office.com/POP.AccessAsUser.All", + "https://outlook.office.com/SMTP.Send", + ] + + client_config = { + "tenant_id" => tenant_id, + "client_id" => client_id, + "client_secret" => client_secret, + "site" => 'https://login.microsoftonline.com', + "authorize_url" => "/#{tenant_id}/oauth2/v2.0/authorize", + "token_url" => "/#{tenant_id}/oauth2/v2.0/token", + "scope" => scope.join(' ') + } + client_config["redirect_uri"] = redirect_uri if redirect_set + + client = build_oauth_client(client_config, redirect_set) + + # force prompt alleen als expliciet aangevraagd + force_consent = ENV['force_consent'] == '1' + + url_params = { scope: client_config['scope'] } + url_params[:prompt] = 'consent' if force_consent + url_params[:redirect_uri] = client_config['redirect_uri'] if redirect_set + + puts "Go to URL: #{client.auth_code.authorize_url(**url_params)}" + print "Enter full redirect URL after authorize: " + code = Redmine::EmailOauthHelper.read_oauth_code + + access_token = oauth_get_token(client, code, client_id, client_secret, redirect_set ? client_config['redirect_uri'] : nil) + check_refresh_token!(access_token) + save_oauth_files(token_file, access_token, client_config) + + puts "AUTH OK!" + end + + # ------------------------------------------------------------------ + # Google Authorization Init + # ------------------------------------------------------------------ + desc "Init Google authorization" + task :google_oauth2_init => :environment do + token_file = normalize_token_file(ENV['token_file']) + client_id = ENV['client'] + client_secret = ENV['secret'] + redirect_uri = ENV['redirect_uri'].to_s.strip + + puts 'See doc/GMAIL_IMAP_OAUTH.md for setup instructions.' + abort("Missing ENV client") if client_id.to_s.empty? + abort("Missing ENV secret") if client_secret.to_s.empty? + if redirect_uri.empty? + redirect_uri = 'http://localhost' + puts "WARN: no redirect_uri supplied; defaulting to #{redirect_uri}" + end + redirect_set = true + + scope = ['https://mail.google.com/'] + + client_config = { + 'client_id' => client_id, + 'client_secret' => client_secret, + 'site' => 'https://accounts.google.com', + 'authorize_url' => '/o/oauth2/v2/auth', + 'token_url' => 'https://oauth2.googleapis.com/token', + 'scope' => scope.join(' '), + 'auth_params' => { 'access_type' => 'offline' } + } + client_config['redirect_uri'] = redirect_uri if redirect_set + + client = build_oauth_client(client_config, redirect_set) + + force_consent = ENV['force_consent'] == '1' + + url_params = (client_config['auth_params'] || {}).dup + url_params = url_params.transform_keys(&:to_sym) + url_params[:scope] = client_config['scope'] + url_params[:prompt] = 'consent' if force_consent + url_params[:redirect_uri] = client_config['redirect_uri'] if redirect_set + + puts "Go to URL: #{client.auth_code.authorize_url(**url_params)}" + print "Enter full redirect URL after authorize: " + code = Redmine::EmailOauthHelper.read_oauth_code + + access_token = oauth_get_token(client, code, client_id, client_secret, redirect_set ? client_config['redirect_uri'] : nil) + check_refresh_token!(access_token) + save_oauth_files(token_file, access_token, client_config) + + puts "AUTH OK!" + end + + # ------------------------------------------------------------------ + # Receive IMAP (OAuth2) + # ------------------------------------------------------------------ + desc "Read emails from an IMAP server authorized via OAuth2" + task :receive_imap_oauth2 => :environment do + debug = env_bool('imap_debug', false) + + token_file = normalize_token_file(ENV['token_file']) + unless File.exist?("#{token_file}.yml") && File.exist?("#{token_file}_client.yml") + raise "token_file not defined or not exists (expected #{token_file}.yml and _client.yml)" + end + + client_config = YAML.safe_load_file("#{token_file}_client.yml", **PERMITTED) + client = build_oauth_client(client_config, !client_config['redirect_uri'].to_s.empty?) + + token_hash = YAML.safe_load_file("#{token_file}.yml", **PERMITTED) + access_token = OAuth2::AccessToken.from_hash(client, token_hash) + + if debug + exp = access_token.expires_at ? Time.at(access_token.expires_at).utc : '(none)' + rem = access_token.expires_at ? (access_token.expires_at - Time.now.to_i) : '(unknown)' + puts "IMAP DEBUG: loaded token expires_at=#{exp} remaining=#{rem}" + puts "IMAP DEBUG: refresh? #{access_token.refresh_token ? 'yes' : 'no'}" + puts "IMAP DEBUG: token=#{mask_token(access_token.token)}" + end + + if access_token.expired? + logger = defined?(Rails) && Rails.respond_to?(:logger) ? Rails.logger : nil + msg = "Refreshing OAuth token; old expiry: #{access_token.expires_at}" + logger ? logger.info(msg) : $stderr.puts(msg) + + begin + access_token = access_token.refresh! + rescue OAuth2::Error => e + abort("Token refresh failed (#{e.message}). Re-run init task.") + end + + msg = "OAuth token refreshed; new expiry: #{access_token.expires_at}" + logger ? logger.info(msg) : $stderr.puts(msg) + + File.write("#{token_file}.yml", access_token.to_hash.to_yaml) + secure_file("#{token_file}.yml") + end + + host = ENV['host'].to_s + abort("ENV host (IMAP server) required") if host.empty? + port = (ENV['port'] || 993).to_i + ssl = env_bool('ssl', true) + starttls = env_bool('starttls', false) + folder = ENV['folder'].to_s + folder = 'INBOX' if folder.empty? + + # Safety: als ssl==true -> verwijder starttls sleutel zodat Redmine hem niet activeert + starttls = false if ssl + imap_options = { + :host => host, + :port => port, + :username => ENV['username'], + :password => access_token.token, + :auth_type => 'XOAUTH2', + :folder => folder, + :move_on_success => ENV['move_on_success'], + :move_on_failure => ENV['move_on_failure'] + } + # Alleen meegeven als effectief true (anders NIET in hash => Redmine ziet nil => geen SSL/STARTTLS) + imap_options[:ssl] = true if ssl + imap_options[:starttls] = true if starttls + + puts "IMAP DEBUG: effective imap_options=#{imap_options.inspect}" if debug + + Mailer.with_synched_deliveries do + begin + # Sanitized ENV voor MailHandler (geen IMAP-keys die core verkeerd leest) + mail_env = ENV.to_h.dup + %w[host port ssl starttls username token_file folder move_on_success move_on_failure].each { |k| mail_env.delete(k) } + mail_opts = MailHandler.extract_options_from_env(mail_env) + puts "IMAP DEBUG: MailHandler opts=#{mail_opts.inspect}" if debug + + Redmine::IMAP.check(imap_options, mail_opts) + rescue Net::IMAP::NoResponseError, Net::IMAP::BadResponseError => e + puts "IMAP ERROR: #{e.class}: #{e.message}" + if e.message.to_s.include?('AUTHENTICATIONFAILED') + puts "Please re-run the OAuth init task. Token file: #{token_file}.yml" + end + raise + rescue StandardError => e + warn "IMAP error: #{e.class}: #{e.message}" + warn e.backtrace.join("\n") if debug + raise + end + end + end + + # ------------------------------------------------------------------ + # Inspect token + # ------------------------------------------------------------------ + desc "Display OAuth2 token information" + task :oauth2_status => :environment do + token_file = normalize_token_file(ENV['token_file']) + + unless File.exist?("#{token_file}.yml") && File.exist?("#{token_file}_client.yml") + raise "token_file not defined or not exists" + end + + raw_token_data = YAML.safe_load_file("#{token_file}.yml", **PERMITTED) + token_data = if raw_token_data.is_a?(Hash) + raw_token_data.each_with_object({}) { |(k,v),h| h[k.to_s.sub(/\A:/,'')] = v } + else + {} + end + + client_config = YAML.safe_load_file("#{token_file}_client.yml", **PERMITTED) + + provider = + case client_config['site'] + when /microsoftonline/ then 'office365' + when /google/ then 'google' + else client_config['site'] + end + + exp_val = token_data['expires_at'] + if exp_val + expires_at = Time.at(exp_val.to_i) + remaining = exp_val.to_i - Time.now.to_i + else + expires_at = '(none)' + remaining = '(unknown)' + end + + refresh_present = token_data['refresh_token'].to_s != '' + + puts "provider: #{provider}" + puts "expiry time: #{expires_at.is_a?(Time) ? expires_at.utc : expires_at} (#{expires_at})" + puts "seconds remaining: #{remaining}" + puts "refresh token present: #{refresh_present}" + puts "redirect_uri stored: #{client_config['redirect_uri'].inspect}" + end + + end +end Index: test/unit/lib/redmine/imap_test.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/test/unit/lib/redmine/imap_test.rb b/test/unit/lib/redmine/imap_test.rb new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/test/unit/lib/redmine/imap_test.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,36 @@ +# frozen_string_literal: true + +require_relative '../../../test_helper' +require 'net/imap' + +class Redmine::IMAPTest < ActiveSupport::TestCase + def test_check_uses_login_by_default + imap = mock('imap') + Net::IMAP.expects(:new).with('127.0.0.1', '143', false).returns(imap) + imap.expects(:starttls).never + imap.expects(:login).with('user', 'secret') + imap.expects(:select).with('INBOX') + imap.expects(:uid_search).returns([]) + imap.expects(:expunge) + imap.expects(:logout) + imap.expects(:disconnect) + + Redmine::IMAP.check(:username => 'user', :password => 'secret') + end + + def test_check_uses_authenticate_when_auth_type_is_set + imap = mock('imap') + Net::IMAP.expects(:new).with('imap.example.com', '993', true).returns(imap) + imap.expects(:starttls) + imap.expects(:authenticate).with('XOAUTH2', 'user', 'token') + imap.expects(:select).with('INBOX') + imap.expects(:uid_search).returns([]) + imap.expects(:expunge) + imap.expects(:logout) + imap.expects(:disconnect) + + Redmine::IMAP.check(:host => 'imap.example.com', :port => '993', :ssl => true, + :starttls => true, :username => 'user', :password => 'token', + :auth_type => 'XOAUTH2') + end +end Index: test/unit/lib/tasks/email_oauth_status_test.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/test/unit/lib/tasks/email_oauth_status_test.rb b/test/unit/lib/tasks/email_oauth_status_test.rb new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/test/unit/lib/tasks/email_oauth_status_test.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,62 @@ +require 'minitest/autorun' +require 'yaml' +require 'fileutils' +require 'tempfile' + +class EmailOauthStatusTest < Minitest::Test + def setup + @dir = Dir.mktmpdir + @token_file = File.join(@dir, 'token') + prefixed = File.join(@dir, 'email_oauth2_token') + File.write("#{prefixed}_client.yml", { + 'client_id' => 'id', + 'client_secret' => 'secret', + 'site' => 'https://accounts.google.com', + 'authorize_url' => 'auth', + 'token_url' => 'token' + }.to_yaml) + File.write("#{prefixed}.yml", { + 'access_token' => 'abc', + 'refresh_token' => 'r', + 'expires_at' => Time.now.to_i + 3600 + }.to_yaml) + end + + def teardown + FileUtils.rm_rf(@dir) + end + + def run_status_logic + file = @token_file + unless File.basename(file).start_with?('email_oauth2') + file = File.join(File.dirname(file), "email_oauth2_#{File.basename(file)}") + end + token_data = YAML.safe_load_file("#{file}.yml") + client_config = YAML.safe_load_file("#{file}_client.yml") + + provider = + case client_config['site'] + when /microsoftonline/ then 'office365' + when /google/ then 'google' + else client_config['site'] + end + + expires_at = Time.at(token_data['expires_at'].to_i) + remaining = token_data['expires_at'].to_i - Time.now.to_i + refresh_present = token_data.key?('refresh_token') && !token_data['refresh_token'].to_s.empty? + + puts "provider: #{provider}" + puts "expiry time: #{expires_at.utc} (#{expires_at})" + puts "seconds remaining: #{remaining}" + puts "refresh token present: #{refresh_present}" + end + + def test_status_output_format + out, = capture_io { run_status_logic } + lines = out.split("\n") + assert_match(/^provider: google$/, lines[0]) + assert_match(/^expiry time: .*UTC \(.+\)$/, lines[1]) + assert_match(/^seconds remaining: \d+$/, lines[2]) + assert_match(/^refresh token present: true$/, lines[3]) + end +end Index: test/unit/lib/tasks/email_oauth_test.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/test/unit/lib/tasks/email_oauth_test.rb b/test/unit/lib/tasks/email_oauth_test.rb new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/test/unit/lib/tasks/email_oauth_test.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,321 @@ +require 'minitest/autorun' +require 'mocha/minitest' +require 'yaml' +require 'fileutils' +require 'tempfile' +require 'oauth2' +require 'rake' +load File.expand_path('../../../../lib/tasks/email_oauth.rake', __dir__) + +class EmailOauthTokenRefreshTest < Minitest::Test + def setup + @dir = Dir.mktmpdir + @token_file = File.join(@dir, 'token') + prefixed = File.join(@dir, 'email_oauth2_token') + File.write("#{prefixed}_client.yml", { + 'provider' => 'google', + 'client_id' => 'id', + 'client_secret' => 'secret', + 'site' => 'site', + 'authorize_url' => 'auth', + 'token_url' => 'token', + 'redirect_uri' => 'redir', + 'scope' => 'scope', + 'auth_params' => {'access_type' => 'offline'} + }.to_yaml) + File.write("#{prefixed}.yml", { + 'access_token' => 'old', + 'refresh_token' => 'r', + 'expires_at' => Time.now.to_i - 3600 + }.to_yaml) + end + + def teardown + FileUtils.rm_rf(@dir) + end + + def run_init_logic + token_file = File.join(@dir, 'email_oauth2_new') + client_config = { + 'client_id' => 'id', + 'client_secret' => 'secret', + 'site' => 'site', + 'authorize_url' => 'auth', + 'token_url' => 'token' + } + File.write("#{token_file}.yml", {'access_token' => 't'}.to_yaml) + File.chmod(0600, "#{token_file}.yml") unless File::ALT_SEPARATOR + File.write("#{token_file}_client.yml", client_config.to_yaml) + File.chmod(0600, "#{token_file}_client.yml") unless File::ALT_SEPARATOR + end + + def run_refresh_logic + file = @token_file + unless File.basename(file).start_with?('email_oauth2') + file = File.join(File.dirname(file), "email_oauth2_#{File.basename(file)}") + end + client_config = YAML.safe_load_file("#{file}_client.yml") + client = OAuth2::Client.new(client_config['client_id'], client_config['client_secret'], + site: client_config['site'], authorize_url: client_config['authorize_url'], token_url: client_config['token_url'], + redirect_uri: client_config['redirect_uri']) + access_token = OAuth2::AccessToken.from_hash(client, YAML.safe_load_file("#{file}.yml")) + if access_token.expired? + logger = defined?(Rails) && Rails.respond_to?(:logger) ? Rails.logger : nil + msg = "Refreshing OAuth token; old expiry: #{access_token.expires_at}" + if logger + logger.info(msg) + else + $stderr.puts(msg) + end + + access_token = access_token.refresh! + + msg = "OAuth token refreshed; new expiry: #{access_token.expires_at}" + if logger + logger.info(msg) + else + $stderr.puts(msg) + end + + File.write("#{file}.yml", access_token.to_hash.to_yaml) + File.chmod(0600, "#{file}.yml") unless File::ALT_SEPARATOR + end + end + + def test_expired_token_refreshes_and_writes_file + new_expiry = Time.now.to_i + 3600 + refreshed = stub('token', to_hash: { + 'access_token' => 'new', + 'refresh_token' => 'r', + 'expires_at' => new_expiry + }, expires_at: new_expiry) + old_expiry = Time.now.to_i - 3600 + access_token = stub('token', expired?: true, refresh!: refreshed, expires_at: old_expiry) + + OAuth2::Client.expects(:new).returns(:client) + OAuth2::AccessToken.expects(:from_hash).returns(access_token) + + capture_io do + run_refresh_logic + end + + data = YAML.safe_load(File.read(File.join(@dir, 'email_oauth2_token.yml'))) + assert_equal 'new', data['access_token'] + if File::ALT_SEPARATOR.nil? + assert_equal 0600, File.stat(File.join(@dir, 'email_oauth2_token.yml')).mode & 0777 + end + end + + def test_valid_token_is_not_refreshed + access_token = stub('token') + access_token.stubs(:expired?).returns(false) + + OAuth2::Client.expects(:new).returns(:client) + OAuth2::AccessToken.expects(:from_hash).returns(access_token) + access_token.expects(:refresh!).never + + capture_io do + run_refresh_logic + end + + data = YAML.safe_load(File.read(File.join(@dir, 'email_oauth2_token.yml'))) + assert_equal 'old', data['access_token'] + end + + def test_init_creates_files_with_secure_permissions + run_init_logic + if File::ALT_SEPARATOR.nil? + assert_equal 0600, File.stat(File.join(@dir, 'email_oauth2_new.yml')).mode & 0777 + assert_equal 0600, File.stat(File.join(@dir, 'email_oauth2_new_client.yml')).mode & 0777 + end + end + + def test_logs_expiration_times_on_refresh + new_expiry = Time.now.to_i + 3600 + refreshed = stub('token', + to_hash: { + 'access_token' => 'new', + 'refresh_token' => 'r', + 'expires_at' => new_expiry + }, + expires_at: new_expiry) + + old_expiry = Time.now.to_i - 3600 + access_token = stub('token', expired?: true, refresh!: refreshed, expires_at: old_expiry) + + OAuth2::Client.expects(:new).returns(:client) + OAuth2::AccessToken.expects(:from_hash).returns(access_token) + + _out, err = capture_io do + run_refresh_logic + end + + assert_includes err, old_expiry.to_s + assert_includes err, new_expiry.to_s + end +end + +class EmailOauthInitCheckTest < Minitest::Test + def run_init_check(token) + if !token.refresh_token && ENV['allow_no_refresh'] != '1' + warn 'No refresh token returned. Re-authorize with prompt=consent.' + exit 1 + end + end + + def test_exit_without_refresh_token + token = stub('token', refresh_token: nil) + ENV.delete('allow_no_refresh') + assert_raises(SystemExit) do + _, err = capture_io { run_init_check(token) } + assert_match(/prompt=consent/, err) + end + end + + def test_no_exit_when_allowed + token = stub('token', refresh_token: nil) + ENV['allow_no_refresh'] = '1' + assert_silent { run_init_check(token) } + ensure + ENV.delete('allow_no_refresh') + end +end + +require_relative '../../../../lib/redmine/email_oauth_helper' + +class EmailOauthReadUrlTest < Minitest::Test + def test_reads_valid_url + STDIN.stubs(:gets).returns("https://example.com/?code=abc\n") + assert_equal 'abc', Redmine::EmailOauthHelper.read_oauth_code + end + + def test_invalid_then_valid_url + STDIN.stubs(:gets).returns("invalid\n", "https://example.com/?code=xyz\n") + out, = capture_io do + assert_equal 'xyz', Redmine::EmailOauthHelper.read_oauth_code + end + assert_match 'Invalid URL', out + end +end + +class EmailOauthInitTest < Minitest::Test + def setup + @dir = Dir.mktmpdir + @token_file = File.join(@dir, 'token') + end + + def teardown + FileUtils.rm_rf(@dir) + end + + def run_init_logic(token_hash) + file = @token_file + unless File.basename(file).start_with?('email_oauth2') + file = File.join(File.dirname(file), "email_oauth2_#{File.basename(file)}") + end + client_config = { + 'client_id' => 'id', + 'client_secret' => 'secret', + 'site' => 'site', + 'authorize_url' => 'auth', + 'token_url' => 'token' + } + client = build_oauth_client(client_config, false) + auth_code = stub('auth_code') + client.stubs(:auth_code).returns(auth_code) + auth_code.stubs(:authorize_url) + token = stub('token', refresh_token: token_hash['refresh_token'], to_hash: token_hash) + auth_code.stubs(:get_token).returns(token) + + access_token = oauth_get_token(client, 'code', client_config['client_id'], client_config['client_secret'], nil) + check_refresh_token!(access_token) + save_oauth_files(file, access_token, client_config) + end + + def test_init_aborts_without_refresh_token + assert_raises(SystemExit) do + Kernel.stub(:abort, proc { |msg| raise SystemExit.new }) do + run_init_logic('access_token' => 'a') + end + end + end + + def test_refresh_token_is_persisted + run_init_logic('access_token' => 'a', 'refresh_token' => 'r') + data = YAML.safe_load(File.read(File.join(@dir, 'email_oauth2_token.yml'))) + assert_equal 'r', data['refresh_token'] + end +end + +module Redmine + module IMAP + end +end + +class Mailer + def self.with_synched_deliveries(&block) + yield + end +end + +require 'net/imap' + +class EmailOauthReceiveImapRescueTest < Minitest::Test + def setup + @dir = Dir.mktmpdir + end + + def teardown + FileUtils.rm_rf(@dir) + end + + def imap_exception(klass, message) + response = Struct.new(:data).new(Struct.new(:text).new(message)) + klass.new(response) + end + + def run_receive_logic(exception) + token_file = File.join(@dir, 'email_oauth2_token') + imap_options = { + :host => nil, + :port => nil, + :ssl => nil, + :starttls => nil, + :username => nil, + :password => 'token', + :auth_type => 'XOAUTH2', + :folder => nil, + :move_on_success => nil, + :move_on_failure => nil + } + Redmine::IMAP.stubs(:check).raises(exception) + + out, _ = capture_io do + Mailer.with_synched_deliveries do + begin + Redmine::IMAP.check(imap_options, {}) + rescue Net::IMAP::NoResponseError, Net::IMAP::BadResponseError => e + puts e.message + if e.message.to_s.include?('AUTHENTICATIONFAILED') + puts "Please re-run the OAuth init task. Token file: #{token_file}.yml" + end + end + end + end + out + end + + def test_authentication_failed_outputs_hint + exception = imap_exception(Net::IMAP::BadResponseError, 'AUTHENTICATIONFAILED') + output = run_receive_logic(exception) + assert_includes output, 'Please re-run the OAuth init task' + assert_includes output, File.join(@dir, 'email_oauth2_token.yml') + end + + def test_other_errors_do_not_output_hint + exception = imap_exception(Net::IMAP::NoResponseError, 'some error') + output = run_receive_logic(exception) + assert_includes output, 'some error' + refute_includes output, 'Please re-run the OAuth init task' + end +end Index: test/unit/lib/tasks/google_oauth2_init_test.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/test/unit/lib/tasks/google_oauth2_init_test.rb b/test/unit/lib/tasks/google_oauth2_init_test.rb new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/test/unit/lib/tasks/google_oauth2_init_test.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,61 @@ +require 'minitest/autorun' +require 'mocha/minitest' +require 'yaml' +require 'fileutils' +require 'tempfile' +require 'oauth2' +require 'cgi' +require 'uri' +require 'rake' +load File.expand_path('../../../../lib/tasks/email_oauth.rake', __dir__) + +class GoogleOauth2InitTest < Minitest::Test + def setup + @dir = Dir.mktmpdir + @token_file = File.join(@dir, 'token') + @redirect_uri = 'http://localhost' + end + + def teardown + FileUtils.rm_rf(@dir) + end + + def run_init_logic + token_file = @token_file + unless File.basename(token_file).start_with?('email_oauth2') + token_file = File.join(File.dirname(token_file), "email_oauth2_#{File.basename(token_file)}") + end + client_id = 'id' + client_secret = 'secret' + scope = ['https://mail.google.com/'] + redirect_uri = @redirect_uri + client_config = { + 'client_id' => client_id, + 'client_secret' => client_secret, + 'site' => 'https://accounts.google.com', + 'authorize_url' => '/o/oauth2/v2/auth', + 'token_url' => 'https://oauth2.googleapis.com/token', + 'redirect_uri' => redirect_uri + } + client = build_oauth_client(client_config, true) + print("Go to URL: #{client.auth_code.authorize_url(access_type: 'offline', scope: scope.join(' '), redirect_uri: redirect_uri)}\n") + print('Enter full URL after authorize:') + code = CGI.parse(URI.parse(STDIN.gets.strip).query)['code'].first + access_token = oauth_get_token(client, code, client_id, client_secret, redirect_uri) + save_oauth_files(token_file, access_token, client_config) + token_file + end + + def test_redirect_uri_saved_and_used + OAuth2::Client.expects(:new).returns(client = stub('client')) + client.stubs(:auth_code).returns(auth_code = stub('auth_code')) + auth_code.expects(:authorize_url).with(access_type: 'offline', scope: 'https://mail.google.com/', redirect_uri: @redirect_uri).returns('http://auth.example') + auth_code.expects(:get_token).with('abc', redirect_uri: @redirect_uri, client_id: 'id', client_secret: 'secret').returns(stub('token', to_hash: {})) + STDIN.expects(:gets).returns("http://localhost/?code=abc\n") + + token_file = run_init_logic + + data = YAML.safe_load(File.read("#{token_file}_client.yml")) + assert_equal @redirect_uri, data['redirect_uri'] + end +end Index: test/unit/lib/tasks/o365_oauth2_init_test.rb IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/test/unit/lib/tasks/o365_oauth2_init_test.rb b/test/unit/lib/tasks/o365_oauth2_init_test.rb new file mode 100644 --- /dev/null (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) +++ b/test/unit/lib/tasks/o365_oauth2_init_test.rb (revision 15d33bb9e483174c135a2032aa9cc4c846e1bf31) @@ -0,0 +1,141 @@ +require 'minitest/autorun' +require 'mocha/minitest' +require 'yaml' +require 'fileutils' +require 'tempfile' +require 'oauth2' +require 'cgi' +require 'uri' +require 'rake' +load File.expand_path('../../../../lib/tasks/email_oauth.rake', __dir__) + +class O365Oauth2InitTest < Minitest::Test + def setup + @dir = Dir.mktmpdir + @token_file = File.join(@dir, 'token') + @client_id = 'o365-client-id' + @client_secret = 'o365-secret' + @tenant_id = 'o365-tenant' + @redirect_uri = 'https://localhost/o365_callback' + end + + def teardown + FileUtils.rm_rf(@dir) + end + + # Lightweight port van de rake-task logica (zonder Rails). + # Parameter redirect_set bepaalt of we redirect_uri meegeven. + def run_init_logic(redirect_set: true) + token_file = @token_file + unless File.basename(token_file).start_with?('email_oauth2') + token_file = File.join(File.dirname(token_file), "email_oauth2_#{File.basename(token_file)}") + end + + scope = [ + "offline_access", + "https://outlook.office.com/User.Read", + "https://outlook.office.com/IMAP.AccessAsUser.All", + "https://outlook.office.com/POP.AccessAsUser.All", + "https://outlook.office.com/SMTP.Send" + ] + scope_str = scope.join(' ') + + client_config = { + 'client_id' => @client_id, + 'client_secret' => @client_secret, + 'site' => 'https://login.microsoftonline.com', + 'authorize_url' => "/#{@tenant_id}/oauth2/v2.0/authorize", + 'token_url' => "/#{@tenant_id}/oauth2/v2.0/token", + 'scope' => scope_str + } + client_config['redirect_uri'] = @redirect_uri if redirect_set + + client = build_oauth_client(client_config, redirect_set) + + # Opbouw url_params zoals in rake-task + url_params = { scope: scope_str } + url_params[:prompt] = 'consent' + url_params[:redirect_uri] = client_config['redirect_uri'] if redirect_set + + puts "Go to URL: #{client.auth_code.authorize_url(**url_params)}" + print('Enter full URL after authorize:') + + code = CGI.parse(URI.parse(STDIN.gets.strip).query)['code'].first + + access_token = oauth_get_token(client, code, @client_id, @client_secret, redirect_set ? client_config['redirect_uri'] : nil) + save_oauth_files(token_file, access_token, client_config) + token_file + end + + # -------------------------------------------------------------------- + # Test: redirect_uri meegegeven → verwacht param + # -------------------------------------------------------------------- + def test_redirect_uri_saved_and_used + # Stubs + OAuth2::Client.expects(:new).with( + @client_id, @client_secret, + has_entries( + site: 'https://login.microsoftonline.com', + authorize_url: "/#{@tenant_id}/oauth2/v2.0/authorize", + token_url: "/#{@tenant_id}/oauth2/v2.0/token", + redirect_uri: @redirect_uri + ) + ).returns(client = mock('client')) + + client.expects(:auth_code).twice.returns(auth_code = mock('auth_code')) + # authorize_url moet redirect_uri bevatten + auth_code.expects(:authorize_url).with( + has_entries(scope: includes('offline_access'), prompt: 'consent', redirect_uri: @redirect_uri) + ).returns('https://auth.example/authorize') + + # get_token moet redirect_uri meekrijgen + auth_code.expects(:get_token).with('abc', has_entries(redirect_uri: @redirect_uri, client_id: @client_id, client_secret: @client_secret)).returns(stub('token', to_hash: {})) + + STDIN.expects(:gets).returns("https://localhost/o365_callback?code=abc\n") + + token_file = run_init_logic(redirect_set: true) + + data = YAML.safe_load(File.read("#{token_file}_client.yml")) + assert_equal @redirect_uri, data['redirect_uri'], "redirect_uri should be persisted in client config" + end + + # -------------------------------------------------------------------- + # Test: GEEN redirect_uri → mag niet in params zitten + # -------------------------------------------------------------------- + def test_no_redirect_uri_not_sent + # Wanneer redirect niet gezet is, verwachten we dat client + # met redirect_uri=nil wordt gebouwd en dat authorize_url & + # get_token *geen* redirect param meekrijgen. + + OAuth2::Client.expects(:new).with( + @client_id, @client_secret, + has_entries( + site: 'https://login.microsoftonline.com', + authorize_url: "/#{@tenant_id}/oauth2/v2.0/authorize", + token_url: "/#{@tenant_id}/oauth2/v2.0/token", + redirect_uri: nil + ) + ).returns(client = mock('client')) + + client.expects(:auth_code).twice.returns(auth_code = mock('auth_code')) + + # Capture params to assert that :redirect_uri NIET aanwezig is + captured_params = nil + auth_code.expects(:authorize_url).with { |**h| + captured_params = h + h[:scope].include?('offline_access') && h[:prompt] == 'consent' && !h.key?(:redirect_uri) + }.returns('https://auth.example/authorize') + + auth_code.expects(:get_token).with('abc', has_entries(client_id: @client_id, client_secret: @client_secret)).returns(stub('token', to_hash: {})).then + + STDIN.expects(:gets).returns("https://localhost/somecallback?code=abc\n") + + token_file = run_init_logic(redirect_set: false) + + # extra assertion: captured params heeft geen redirect_uri + refute captured_params.key?(:redirect_uri), "redirect_uri should not be sent when not configured" + + data = YAML.safe_load(File.read("#{token_file}_client.yml")) + refute data.key?('redirect_uri'), "redirect_uri should not be persisted when not provided" + end +end