Some unit and functional tests miss fixtures and break when run alone
|Assignee:||Jean-Baptiste Barth||% Done:|
I develop many plugins these days, and had the surprise to see the test suite run ok on MacOSX and fail on one of my Linux systems (2 functional tests breaking). Same ruby version, gems, codebase, etc. It turns out it's because tests are not run in the same order on both platforms, and it seems some unit and functional tests depend on other tests and fail if the other test doesn't run before.
To be sure I prepared a test database and ran each test alone :
RAILS_ENV=test rake db:drop db:create db:migrate redmine:plugins db:schema:dump db:fixtures:load cp db/test.sqlite3 db/test.template.sqlite3 for t in test/**/*_test.rb; do cp db/test.template.sqlite3 db/test.sqlite3 RAILS_ENV=test ruby -Itest $t || echo $t |tee -a tmp/failing_alone.txt done
Here's the list I ended with, I excluded scm related tests because I'm not sure they actually have a problem (tests run against 2.1.0 stable version, results in trunk may differ but it's a looong thing to run) :
test/functional/groups_controller_test.rb test/functional/issue_categories_controller_test.rb test/functional/issue_statuses_controller_test.rb test/functional/mail_handler_controller_test.rb test/functional/queries_controller_test.rb test/functional/search_controller_test.rb test/functional/sessions_test.rb test/functional/welcome_controller_test.rb test/integration/api_test/issues_test.rb test/integration/api_test/users_test.rb test/integration/issues_test.rb test/unit/comment_test.rb test/unit/issue_category_test.rb test/unit/issue_relation_test.rb test/unit/issue_test.rb test/unit/journal_observer_test.rb test/unit/journal_test.rb test/unit/lib/redmine/menu_manager/menu_helper_test.rb test/unit/message_test.rb test/unit/project_test.rb test/unit/role_test.rb test/unit/time_entry_activity_test.rb test/unit/version_test.rb test/unit/watcher_test.rb
For most of them I think the fix may be easy, it's just a missing fixture, a coupling with a previous record or a missing setting. I'll try to see what's the problem in each case and commit fixes in trunk as I find them.
Fixed test/functional/mail_handler_controller_test.rb breaking when run alone (#12285)
#1 Updated by Jean-Baptiste Barth over 5 years ago
- Status changed from New to 7
- % Done changed from 0 to 50
OK it's almost only a problem of missing fixtures declarations, sorry for the many small commits. As I started this way I'll continue, unless somebody prefers one bigger commit (too bad I couldn't rebase/reorder locally, yay svn! ;))
Anyway, it might be easier to require
fixtures :all in most tests, but it's not possible for now since there are some YAML files in
test/fixtures/ that are not valid ActiveRecord fixtures. Any thought about this ?
#2 Updated by Jean-Baptiste Barth over 5 years ago
- Status changed from 7 to Resolved
- Target version set to Candidate for next major release
- % Done changed from 50 to 100
- Resolution set to Fixed
Changed my mind, that was 14 more commits for just fixtures declarations, so I just splitted it into unit/functional/integration, I hope it's fine. Jean-Philippe: I let you merge it when you want, but I'd be happy to have your opinion on "fixtures :all".