Patch #44116
closedEnable parallel test execution in GitHub CI
Description
This patch enables parallel tests by default.
- Enable parallel tests by default.
- Keep system tests single-process because they are not stable in parallel yet.
- The worker count can still be changed by setting
PARALLEL_WORKERS. - Adjust test code and configuration so tests run correctly and reliably in parallel.
- Fix several flaky tests exposed by parallel test execution.
Why enable Parallel Testing by default?¶
- Redmine's test workflow has gradually become slower. Fast and reliable test execution is essential for continuous development.
- As the test suite grows, Parallel Testing becomes essential for keeping test execution fast.
- Reliable parallel test execution requires test code and configuration that work correctly under parallel execution. To make this possible, we need to shift development to a workflow based on Parallel Testing.
- Rails has long enabled Parallel Testing by default.
Performance¶
| Before | After | |
|---|---|---|
| Test workflow runtime on GitHub Actions | 12m32s (average for 2026-04) | 5m29s (average of 5 runs) |
bin/rails test runtime on MacBook Air M4 (10 processors) |
2m29s | 1m07s |
Details¶
Enable parallel tests by default¶
- Isolate attachment and thumbnail directories per worker to avoid filesystem interference between parallel tests.
- Isolate generator test destination directories per worker to avoid cleanup conflicts between parallel tests.
Fix flaky IssueNestedSetConcurrencyTest¶
This stabilizes IssueNestedSetConcurrencyTest by initializing Active Record connections before running concurrent issue updates. This avoids random failures like the one below, which was exposed by parallel test execution.
Error:
IssueNestedSetConcurrencyTest#test_concurrent_subtask_removal:
RuntimeError: can't add a new key into hash during iteration
test/unit/issue_nested_set_concurrency_test.rb:90:in `block (3 levels) in test_concurrent_subtask_removal'
bin/rails test test/unit/issue_nested_set_concurrency_test.rb:76
Fix flaky tests caused by locale leakage¶
This stabilizes tests affected by two locale-related test isolation problems.
First, a locale set by one test can leak to a later test in the same worker. For example, TimeEntryTest#test_should_not_accept_closed_issue failed because the locale remained Italian:
Failure: TimeEntryTest#test_should_not_accept_closed_issue [test/unit/time_entry_test.rb:186]: --- expected +++ actual @@ -1 +1 @@ -"Cannot log time on a closed issue" +"Impossibile registrare il tempo su una segnalazione chiusa" bin/rails test test/unit/time_entry_test.rb:178
Second, resetting the locale through set_language_if_valid 'en' in teardown can populate Redmine::I18n's language lookup cache while valid_languages is stubbed. This can make later tests unable to resolve non-English locales. For example, Redmine::I18nTest#test_french_locale failed because French could not be resolved and the locale stayed English:
Failure: Redmine::I18nTest#test_french_locale [test/unit/lib/redmine/i18n_test.rb:256]: Expected: "French (Français)" Actual: "English" bin/rails test test/unit/lib/redmine/i18n_test.rb:254
To avoid these problems, this patch resets I18n.locale to I18n.default_locale before each test, and removes teardown code that reset the locale through set_language_if_valid 'en'.
These problems can happen even without parallel test execution because tests in the same process share locale-related global state. Parallel test execution can expose them more often.
Testing¶
- Confirmed that all tests pass at least 10 times on GitHub Actions and locally.
Files
Updated by Katsuya HIDAKA 23 days ago
You can also find the details, code diff, and test results for this patch at https://github.com/hidakatsuya/redmine/pull/43.
Updated by Go MAEDA 20 days ago
- Status changed from New to Resolved
I have committed the patches in r24698, r24699, and r24700. Thank you for the nice improvement.
Since .gitignore was updated in r24698 to ignore files matching db/*.sqlite3, I updated the svn:ignore property for the db directory in r24701 so that Subversion also ignores those files. However, I have not updated the .hgignore file used by Mercurial yet because I am currently proposing its removal in issue #44117.
I will close this issue once .hgignore is removed by #44117.
Updated by Katsuya HIDAKA 12 days ago
- File 0001-Enable-parallel-tests-only-in-test-workflow.patch 0001-Enable-parallel-tests-only-in-test-workflow.patch added
I found a problem.
By changing the default of parallelize to workers: :number_of_processors, plugin tests are now also run in parallel. This may unexpectedly break tests for plugins that do not support parallel testing. This was not intended.
I'm attaching a patch that reverts the default of parallelize to workers: 1 and runs tests in parallel only in Redmine's GitHub CI, so that plugin tests are not affected.
I confirmed all tests passed: https://github.com/hidakatsuya/redmine/actions/runs/26803439606
With this patch, the scope of this issue becomes:
- CI runs tests in parallel
- Fix tests to be stable under parallel execution
Updated by Go MAEDA 10 days ago
- Subject changed from Enable parallel test execution by default to Enable parallel test execution in GitHub CI
- Status changed from Reopened to Closed
Committed the patch posted in #note-6 in r24718.
To avoid failures in tests for some third-party plugins when parallel testing is enabled by default, parallel testing is now enabled by default only in GitHub CI. If you want to run tests in parallel in other environments, continue to specify the PARALLEL_WORKERS environment variable as before (for example, PARALLEL_WORKERS=8 bin/rails test).