Feature #32938
Rails 6: Zeitwerk support
Status: | Closed | Start date: | ||
---|---|---|---|---|
Priority: | Normal | Due date: | ||
Assignee: | % Done: | 0% | ||
Category: | Rails support | |||
Target version: | - | |||
Resolution: | Fixed |
Description
https://github.com/fxn/zeitwerk
DEPRECATION WARNING: Initialization autoloaded the constants ActiveRecord::Acts, ActiveRecord::Acts::Tree, Redmine::I18n, Redmine::Helpers, Redmine::Helpers::URL , Redmine::SafeAttributes, Redmine::SubclassFactory, CustomField, Redmine::Utils, Redmine::Configuration, Redmine::Scm::Adapters::CommandFailed, IssueRelation, Redmine::WikiFormatting::Macros, Redmine::Pagination, Redmine::SudoMode, ApplicationHelper , Redmine::WikiFormatting::Textile, Redmine::WikiFormatting::Textile::Formatter, Redmine::WikiFormatting::Textile::Helper, Redmine::WikiFormatting::HtmlParser , Redmine::WikiFormatting::Textile::HtmlParser, Redmine::WikiFormatting::Markdown, Redmine::WikiFormatting::Markdown::Formatter, Redmine::WikiFormatting::Markdown::HTML, Redmine::WikiFormatting::Markdown::Helper, Redmine::WikiFormatting::Markdown::HtmlParser, Redmine::Views::ApiTemplateHandler, and Setting. Being able to do this is deprecated. Autoloading during initialization is going to be an error condition in future versions of Rails. Reloading does not reboot the application, and therefore code executed during initialization does not run again. So, if you reload ActiveRecord::Acts, for example, the expected changes won't be reflected in that stale Module object. `config.autoloader` is set to `classic`. These autoloaded constants would have been unloaded if `config.autoloader` had been set to `:zeitwerk`. Please, check the "Autoloading and Reloading Constants" guide for solutions.
Related issues
Associated revisions
Extract Redmine::PluginLoader from Redmine::Plugin (#32938).
Patch by Takashi Kato.
History
#1
Updated by Marius BALTEANU over 2 years ago
- Related to Feature #29914: Migrate to Rails 6.1 with Zeitwerk autoloading added
#2
Updated by Go MAEDA over 2 years ago
- Category set to Rails support
#3
Updated by Enziin System about 2 years ago
+1 Thank you!
#4
Updated by Marius BALTEANU about 1 year ago
Pavel, do you have any working patch for this?
#5
Updated by Pavel Rosický about 1 year ago
sry, I don't. This is a pretty heavy change and Redmine has a lot of technical debt in this area. But I have a few hints based on my findings:
Zeitwerk depends on Rails conventions, especially the folder structure.
1/ initializers
/master/config/initializers/10-patches.rb
depends on
include Redmine::I18n
before lib/redmine.rb is loaded, so it triggers autoload. Split them into modules, require explicitly.
2/ there're multiple module definitions in a single file (many places), that don't match the filename and the expected folder structure.
3/ Ruby / Rails patches are global from the pre Ruby 2.0 era. They're patched non-deterministically in some random file
module ActiveRecord class Base def self.human_attribute_name(attr, options = {}) ... end end end
or a require chain like this
/master/lib/redmine/core_ext/string.rb
this issue can be avoided by using manual require at startup time, but you may consider using refinements instead
https://www.cloudbees.com/blog/ruby-refinements/
4/ require_dependency should be removed
/master/app/models/principal.rb#L221
here's a Rails guide and there's also an example of how to deal with STI
https://edgeguides.rubyonrails.org/autoloading_and_reloading_constants.html
5/ autoloads based on #constantize
/master/lib/redmine/wiki_formatting.rb#L42
6/ a folder structure violations, a few examples
/master/lib/diff.rb#L3
module RedmineDiff class Diff end end
should be
class Diff end
or
/master/lib/redmine/diff.rb
module Redmine class Diff end end
7/ module names should use camel-case, upper-case is reserved for constants
/master/lib/redmine/helpers/url.rb#L24
/master/lib/redmine/version.rb#L7
I think now it's pretty clear this is an extremely breaking change. Redmine is a part 1, but even if we fix all these issues, it's impossible to keep compatibility for plugins, but it's well, unavoidable...
Zeitwerk mode can be enabled by putting
config.load_defaults 6.1
into config/application.rb
and tested by
rails zeitwerk:check
Zeitwerk mode will become mandatory in Rails 7.
I think that decisions like module names and folder structure should be discussed with your core team. It's hard to accept a small PR without a confirmation it fixes anything and on the other hand, a big PR might include changes you won't be eventually willing to accept. I would recommend you to investigate these problems yourself, split them into smaller tasks and comment them with your expectations. The community can help you solve it this way, but this task is too big with many open questions.
#6
Updated by Pavel Rosický about 1 year ago
I had to remove some references due to your spam protection :)
#7
Updated by Go MAEDA about 1 year ago
- Duplicated by Defect #35004: DEPRECATION WARNING during startup added
#8
Updated by Go MAEDA about 1 year ago
- Target version set to 5.0.0
#9
Updated by Mizuki ISHIKAWA about 1 year ago
I haven't tried it, it seems that problem 7 can be avoided by writing config/initializers/inflations.rb.
https://github.com/fxn/zeitwerk#inflection
#10
Updated by Mizuki ISHIKAWA 8 months ago
- File 0001-Explicitly-require-a-class-called-in-initializers-th.patch
added
- File 0002-Split-multiple-classes-and-modules-that-existed-in-t.patch
added
- File 0003-Move-lib-redmine-core_ext-to-lib-core_ext.patch
added
- File 0004-Remove-require_dependency.patch
added
- File 0005-Fix-modules-where-folder-structure-does-not-meet-Zei.patch
added
- File 0006-Change-camelcase-rules-to-keep-special-module-or-cla.patch
added
- File 0007-Move-only-modules-and-classes-that-match-Zeitwerk-ru.patch
added
- File 0008-Change-autoload-mode-from-classic-to-zeitwerk.patch
added
I've created a draft patches to discuss Zeitwerk support.
Rails 7.0 is going to remove the old loading methods, so we need to think about how to support the new loading methods. https://weblog.rubyonrails.org/2021/9/15/Rails-7-0-alpha-1-released/ (Zeitwerk Exclusively section)
Description of changes. These patches have been split up as much as possible based on the description in #32938#note-5.
1/ initializers
/master/config/initializers/10-patches.rb
depends oninclude Redmine::I18n
before lib/redmine.rb is loaded, so it triggers autoload. Split them into modules, require explicitly.
0001-Explicitly-require-a-class-called-in-initializers-th.patch
There are two ways to work around this, either by enclosing it in Rails.application.reloader.to_prepare or by explicitly writing "require" before using it.
The simpler way is to enclose it in Rails.application.reloader.to_prepare, but the impact on the plugin is so great that we've decided to use the "require" method in 0001-Explicitly-require-a-class-called-in-initializers-th.patch.
Rails.application.reloader.to_prepare do # Autoload classes and modules needed at boot time here. end
2/ there're multiple module definitions in a single file (many places), that don't match the filename and the expected folder structure.
0002-Split-multiple-classes-and-modules-that-existed-in-t.patch
Previously, AnonymousUser was defined in app/models/user.rb, but since it is not correct according to Zeitwerk rules that the class name and file name do not match, I created a new app/models/anonymous_user.rb and split it.
We have also made similar changes to several other classes and modules.
3/ Ruby / Rails patches are global from the pre Ruby 2.0 era. They're patched non-deterministically in some random file
module ActiveRecord
class Base
def self.human_attribute_name(attr, options = {})
...
end
end
end
or a require chain like this
/master/lib/redmine/core_ext/string.rbthis issue can be avoided by using manual require at startup time, but you may consider using refinements instead
https://www.cloudbees.com/blog/ruby-refinements/
0003-Move-lib-redmine-core_ext-to-lib-core_ext.patch
It seemed difficult to fit this into the Zeitwerk rules, so I moved the directory.
4/ require_dependency should be removed
/master/app/models/principal.rb#L221here's a Rails guide and there's also an example of how to deal with STI
https://edgeguides.rubyonrails.org/autoloading_and_reloading_constants.html
0004-Remove-require_dependency.patch
5/ autoloads based on #constantize
/master/lib/redmine/wiki_formatting.rb#L42
This seems to have been resolved by other changes without a direct fix.
6/ a folder structure violations, a few examples
/master/lib/diff.rb#L3
...
0005-Fix-modules-where-folder-structure-does-not-meet-Zei.patch
7/ module names should use camel-case, upper-case is reserved for constants
/master/lib/redmine/helpers/url.rb#L24
/master/lib/redmine/version.rb#L7
0006-Change-camelcase-rules-to-keep-special-module-or-cla.patch
The effect of changing the class name and module name seemed to be significant, so I overwrote the movement of the camelize method without changing the name.
8 / other
0007-Move-only-modules-and-classes-that-match-Zeitwerk-ru.patch
0008-Change-autoload-mode-from-classic-to-zeitwerk.patch
When I tried to pass everything in the lib directory to autoload_paths, I found it very difficult to make the structure of the plugins and generators directories conform to Zeitwerk's rules. (Relation: 6/ a folder structure violations)
So I changed only the lib/redmine directory to lib/autoloads/redmine directory where I could apply autoloading, and set only the files under lib/autoloads directory as `config.autoload_paths += %W(#{config.root}/lib/autoloads)
`.
I already know that applying these patches will cause multiple problems, mainly related to development.
- probrem-1. The directory change from lib/redmine to lib/autoloads/redmine will cause many patches to fail to apply, and the changelog will be hard to read.
- probrem-2. I changed redmine.rb to "require" many files directly, but autoload doesn't work for files loaded here. Even if you rewrite ApplicationHelper during development, the changes will not be reflected until you restart `rails s`.
I'm sure there are many other problems with it, so feedback and new patches would be greatly appreciated.
#11
Updated by Marius BALTEANU 8 months ago
Thanks Mizuki for working on this. I have as weel some code, I will share it with you as soon I can.
#12
Updated by Marius BALTEANU 8 months ago
I've committed for now the second patch which is safe regardless the type of autoloader. Also, I've fixed some Rubocop offenses and I've updated the .rubocop_todo.yml
to match the new files.
I will continue with more fixes in order to decrease the number of changes required for zeitwerk
.
#13
Updated by Marius BALTEANU 8 months ago
For DateValidator
, I propose to move it from lib/redmine/core_ext/active_record.rb
to app/validators/date_validator.rb
. It will work on both classic
and zeitwerk
and I think it make more sense to stay in the app. Also, other Rails projects do this. Unfortunately, I didn't find a recommended by Rails team.
#14
Updated by Mizuki ISHIKAWA 8 months ago
Marius BALTEANU wrote:
For
DateValidator
, I propose to move it fromlib/redmine/core_ext/active_record.rb
toapp/validators/date_validator.rb
. It will work on bothclassic
andzeitwerk
and I think it make more sense to stay in the app. Also, other Rails projects do this. Unfortunately, I didn't find a recommended by Rails team.
I also think app/validators/date_validator.rb
would be better.
Thank you for your work on this issue.
#15
Updated by Marius BALTEANU 8 months ago
Thanks Mizuki! I've already committed the change.
In this moment, we have only one issue left that makesbin/rails zeitwerk:check
to fail, is lib/diff.rb
. This class is very confusing because we already have the Diff
class extracted from the unified_diff.rb
. I think it's better to rename the class or even better, to move it to lib/redmine/string_array_diff/
so in the end we will have:
lib/redmine/string_array_diff/diff.rb
lib/redmine/string_array_diff/diffable.rb
What do you think about this? Once we fix this, we can discuss the remaining deprecation warnings and hopefully we can close this issue.
Any feedback/help is appreciated.
#16
Updated by Mizuki ISHIKAWA 8 months ago
- File split-lib-diff.patch
added
Marius BALTEANU wrote:
What do you think about this?
I think the directory structure and naming is good.
In which file do you plan to include the Diffable for Array and String? Am I correct in interpreting the changes as in the attached patch?
1. Change the module structure to match the change in directory structure. (RedmineDiff::Diff => Redmine::StringArrayDiff::Diff, RedmineDiff::Diffable => Redmine::StringArrayDiff::Diffable)
2. Keep lib/diff.rb to maintain the behavior of expanding Arrays and Strings when require 'diff'
is executed in app/models/wiki_page.rb, etc.
#17
Updated by Marius BALTEANU 8 months ago
- File 0001-Refactor-lib-diff.rb.patch
added
Mizuki ISHIKAWA wrote:
Marius BALTEANU wrote:
What do you think about this?
I think the directory structure and naming is good.
In which file do you plan to include the Diffable for Array and String? Am I correct in interpreting the changes as in the attached patch?1. Change the module structure to match the change in directory structure. (RedmineDiff::Diff => Redmine::StringArrayDiff::Diff, RedmineDiff::Diffable => Redmine::StringArrayDiff::Diffable)
2. Keep lib/diff.rb to maintain the behavior of expanding Arrays and Strings whenrequire 'diff'
is executed in app/models/wiki_page.rb, etc.
I think we can move everything to lib/redmine/string_array_diff
and update the require in app/modeles/wiki_page.rb
and lib/redmine/helpers/diff.rb
as I did in attached patch. It works for you? In my CI, all tests pass.
#18
Updated by Mizuki ISHIKAWA 8 months ago
Marius BALTEANU wrote:
I think we can move everything to
lib/redmine/string_array_diff
and update the require inapp/modeles/wiki_page.rb
andlib/redmine/helpers/diff.rb
as I did in attached patch. It works for you? In my CI, all tests pass.
Thanks for attaching the patch for clarity.
It looks good and actually worked fine in my environment.
#19
Updated by Takashi Kato 8 months ago
- File alt-0001-Extract-Redmine-PluginLoader-from-Redmine-Plugin.patch
added
- File alt-0002-Move-preparation-code-to-redmine-preparation.rb.patch
added
- File alt-0003-Add-core-extensions-in-prepare-method.patch
added
- File alt-0004-Load-core-plugins-with-require.patch
added
- File alt-0005-Use-zeitwerk.patch
added
I made alternative patches to reduce require
at initialization time and to increase autoloading coverage.
- I cut out
Redmine::PluginLoader
fromRedmine::Plugin
and use PluginLoader at initialization time. Now, Zeitwerk autoloadsRedmine::Plugin
and other classes and modules in it. - I extracted the preparation code(building menu, setting permissions, queries) in
/lib/redmine.rb
toRedmine::Preparation
and it's now inRails.application.config. to_prepare
block. It reduces the use ofrequire
during initialization. - When setting the autoloading paths for Zeitwerk, I used the
namespace
option. It allows that Zeitwerk autoloads libraries under thelib/redmine
directory without moving them to another directory. - To simplify patching, I've left the core plugins and
RedCloth3
out of the autoload for now. I will fix them later.
#20
Updated by Takashi Kato 8 months ago
The above patches are appliable in current trunk(r21261).
#21
Updated by Marius BALTEANU 8 months ago
Takashi Kato wrote:
The above patches are appliable in current trunk(r21261).
Thanks Takashi for the patches, they look good to me. I've already committed some changes related to require
from patch 0004, I've renamed CoreExtensions
to CoreExt
(r21267) and moved lib/diff.rb
to lib/redmine/string_array_diff/diff
(r21262).
Applying them on top of the changes committed today fix all the warnings related to zeitwerk:
➜ redmine git:(feature/32938_zeitwerk) ✗ bin/rails zeitwerk:check Hold on, I am eager loading the application. W, [2021-10-28T01:34:34.662994 #61216] WARN -- : Creating scope :system. Overwriting existing method Enumeration.system. W, [2021-10-28T01:34:34.858254 #61216] WARN -- : Creating scope :sorted. Overwriting existing method Group.sorted. W, [2021-10-28T01:34:34.929581 #61216] WARN -- : Creating scope :sorted. Overwriting existing method User.sorted. All is good!
#22
Updated by Mizuki ISHIKAWA 8 months ago
Takashi Kato wrote:
The above patches are appliable in current trunk(r21261).
I've applied the patch #32938#note-19 and it looks great.
I think this patch supports Zeitwerk in a better way than the patch I posted.
I've tried to run some plugins, and they work well as long as the plugin side is adapted to Zeitwerk's rules.
#23
Updated by Marius BALTEANU 7 months ago
- Status changed from New to Closed
- Assignee set to Marius BALTEANU
- Target version deleted (
5.0.0) - Resolution set to Fixed
Patches committed, thanks everyone for working on this!
#24
Updated by Mizuki ISHIKAWA 7 months ago
Thank you for your hard work.
#25
Updated by Marius BALTEANU 7 months ago
- Related to Defect #36218: Plugin assets are not copied correctly in trunk r21289 added
#26
Updated by Go MAEDA 7 months ago
- Related to Defect #36245: ActiveSupport::Reloader.to_prepare not working in trunk 21287 added
#27
Updated by Go MAEDA 7 months ago
- Related to Defect #36287: rake redmine:plugins:assets fails added
#28
Updated by Go MAEDA 5 months ago
- Related to Defect #15958: AnonymousUser class not found by plugin added
#29
Updated by Go MAEDA 3 months ago
- Related to Defect #36835: Redmine::Plugin.assets_directory is not working after r21283 added
#31
Updated by Marius BALTEANU 3 months ago
- Status changed from Reopened to Closed