Defect #12285

Some unit and functional tests miss fixtures and break when run alone

Added by Jean-Baptiste Barth about 5 years ago. Updated almost 5 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Jean-Baptiste Barth% Done:

100%

Category:Code cleanup/refactoring
Target version:2.2.0
Resolution:Fixed Affected version:

Description

I develop many plugins these days, and had the surprise to see the test suite run ok on MacOSX and fail on one of my Linux systems (2 functional tests breaking). Same ruby version, gems, codebase, etc. It turns out it's because tests are not run in the same order on both platforms, and it seems some unit and functional tests depend on other tests and fail if the other test doesn't run before.

To be sure I prepared a test database and ran each test alone :

RAILS_ENV=test rake db:drop db:create db:migrate redmine:plugins db:schema:dump db:fixtures:load
cp db/test.sqlite3 db/test.template.sqlite3
for t in test/**/*_test.rb; do
  cp db/test.template.sqlite3 db/test.sqlite3
  RAILS_ENV=test ruby -Itest $t || echo $t |tee -a tmp/failing_alone.txt
done

Here's the list I ended with, I excluded scm related tests because I'm not sure they actually have a problem (tests run against 2.1.0 stable version, results in trunk may differ but it's a looong thing to run) :

test/functional/groups_controller_test.rb
test/functional/issue_categories_controller_test.rb
test/functional/issue_statuses_controller_test.rb
test/functional/mail_handler_controller_test.rb
test/functional/queries_controller_test.rb
test/functional/search_controller_test.rb
test/functional/sessions_test.rb
test/functional/welcome_controller_test.rb
test/integration/api_test/issues_test.rb
test/integration/api_test/users_test.rb
test/integration/issues_test.rb
test/unit/comment_test.rb
test/unit/issue_category_test.rb
test/unit/issue_relation_test.rb
test/unit/issue_test.rb
test/unit/journal_observer_test.rb
test/unit/journal_test.rb
test/unit/lib/redmine/menu_manager/menu_helper_test.rb
test/unit/message_test.rb
test/unit/project_test.rb
test/unit/role_test.rb
test/unit/time_entry_activity_test.rb
test/unit/version_test.rb
test/unit/watcher_test.rb

For most of them I think the fix may be easy, it's just a missing fixture, a coupling with a previous record or a missing setting. I'll try to see what's the problem in each case and commit fixes in trunk as I find them.

Associated revisions

Revision 10779
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/unit/watcher_test.rb breaking when run alone (#12285)

Revision 10780
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/unit/version_test.rb breaking when run alone (#12285)

Revision 10781
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/unit/issue_category_test.rb breaking when run alone (#12285)

Revision 10782
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/functional/groups_controller_test.rb breaking when run alone (#12285)

Revision 10783
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/functional/mail_handler_controller_test.rb breaking when run alone (#12285)

Revision 10784
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/functional/sessions_test.rb breaking when run alone (#12285)

Revision 10785
Added by Jean-Baptiste Barth about 5 years ago

Fixed test/functional/welcome_controller_test.rb breaking when run alone (#12285)

Revision 10786
Added by Jean-Baptiste Barth about 5 years ago

Fixed some more test/unit/*_test.rb breaking when run alone (#12285)

Revision 10787
Added by Jean-Baptiste Barth about 5 years ago

Fixed some more test/functional/*_test.rb breaking when run alone (#12285)

Revision 10788
Added by Jean-Baptiste Barth about 5 years ago

Fixed some more test/integration/*_test.rb breaking when run alone (#12285)

History

#1 Updated by Jean-Baptiste Barth about 5 years ago

  • Status changed from New to 7
  • % Done changed from 0 to 50

OK it's almost only a problem of missing fixtures declarations, sorry for the many small commits. As I started this way I'll continue, unless somebody prefers one bigger commit (too bad I couldn't rebase/reorder locally, yay svn! ;))

Anyway, it might be easier to require fixtures :all in most tests, but it's not possible for now since there are some YAML files in test/fixtures/ that are not valid ActiveRecord fixtures. Any thought about this ?

#2 Updated by Jean-Baptiste Barth about 5 years ago

  • Status changed from 7 to Resolved
  • Target version set to Candidate for next major release
  • % Done changed from 50 to 100
  • Resolution set to Fixed

Changed my mind, that was 14 more commits for just fixtures declarations, so I just splitted it into unit/functional/integration, I hope it's fine. Jean-Philippe: I let you merge it when you want, but I'd be happy to have your opinion on "fixtures :all".

#3 Updated by Jean-Philippe Lang almost 5 years ago

  • Status changed from Resolved to Closed
  • Target version changed from Candidate for next major release to 2.2.0

Thanks for the fix.

Jean-Baptiste Barth wrote:

"fixtures :all"

It would slow down the tests quite a bit so I'm not in favour of that.

#4 Updated by Jean-Philippe Lang almost 5 years ago

  • Subject changed from Some unit and functional tests break when run alone to Some unit and functional tests miss fixtures and break when run alone

#5 Updated by Go MAEDA almost 3 years ago

  • Related to Defect #7856: Change "Update" to "Update/Reply" on issue#show added

#6 Updated by Go MAEDA almost 3 years ago

  • Related to deleted (Defect #7856: Change "Update" to "Update/Reply" on issue#show)

Also available in: Atom PDF