Defect #44724
openIssue search with searchable custom fields is still very slow on MariaDB (fix of #43651 is never applied)
Description
Environment: Redmine 7.0.2, Rails 8.1.4, Ruby 4.0.9, MariaDB 11.8.8, adapter mysql2.
Data: ~112k issues, ~449k custom_values, ~6.3k enabled_modules, 9 searchable issue custom fields.
Problem
Searching text inside a project with many issues takes 36-131 s (almost all in ActiveRecord), so it times out behind a reverse proxy. Global search ("all projects") and small projects are fast (< 1.5 s).
Analysis
- The fix of #43651 (
JOIN_ORDER(issues, custom_values)insearch_result_ranks_and_ids) is only applied whenActiveRecord::Base.connection.supports_optimizer_hints?is true. Rails returns false for any MariaDB (!mariadb? && database_version >= "5.7.7"), so on MariaDB the hint is never added. Besides, MariaDB only supports new-style optimizer hints since 12.0. - In
search_scope, the following line producesissues.project_id IN ('123'), because Rails quotes numeric bind values as strings on MySQL adapters:scope = scope.where("#{searchable_options[:project_key]} IN (?)", projects.map(&:id))
With the quoted value, MariaDB 11.8 does not propagate the constant toprojects.id. The plan starts with a full scan ofenabled_modulesand repeats the issues/custom_values lookup for every project. - The same query with
IN (123), unquoted, starts withprojects(const) and runs in 0.4 s instead of 131 s.
Workaround applied locally (lib/plugins/acts_as_searchable/lib/acts_as_searchable.rb, search_scope):
scope = scope.where("#{searchable_options[:project_key]} IN (#{projects.map { |p| p.id.to_i }.join(",")})")
With this change the same search takes about 0.5 s and returns the same results.
Passing integer ids unquoted, or an equivalent fix for MariaDB, would avoid the regression on MariaDB, where the optimizer-hint approach cannot be used. I can provide the full EXPLAIN output if useful.
Related to #43651.
Files
Related issues
Updated by Go MAEDA 2 days ago
- Related to Defect #43651: Searching issues with searchable custom fields causes a performance regression on MySQL added
Updated by Go MAEDA 1 day ago
- File 0001-Fix-slow-issue-search-within-a-project-on-MariaDB.patch 0001-Fix-slow-issue-search-within-a-project-on-MariaDB.patch added
- Target version set to 6.1.7
Thank you for the detailed report and analysis.
I reproduced the issue on MariaDB 11.8.9 with a dataset of similar size (112,000 issues, 448,000 custom values, 700 projects, and 9 searchable issue custom fields). Searching within a project with 30,000 issues took about 1.9 seconds, and the query on custom_values used the plan you described.
I have attached a patch based on the workaround proposed by Francisco Torres García. Instead of embedding the IDs in the SQL string, the patch uses a hash condition:
scope = scope.where(searchable_options[:project_key] => projects.map(&:id))
Because Rails knows the column type for a hash condition, the IDs are passed as integers even on MySQL adapters. With this patch, the same search took about 0.2 seconds.
The change has no practical effect on PostgreSQL and SQLite, where the IDs were already passed as integers.