Defect #8990

Git: Error when fetching changesets

Added by Dave Abrahams about 9 years ago. Updated over 8 years ago.

Status:ClosedStart date:2011-08-05
Priority:LowDue date:
Assignee:-% Done:

0%

Category:SCM
Target version:-
Resolution: Affected version:1.2.1

Description

This is what I found in my fetch-changesets log, which is being produced by the command

ruby script/runner "Repository.fetch_changesets" -e production >> /tmp/redmine-changeset-fetch.log 2>&1 & disown

warning: ignoring dangling symref upstream.
warning: ignoring dangling symref upstream.
warning: ignoring dangling symref upstream.
warning: ignoring dangling symref upstream.
/srv/redmine/vendor/rails/railties/lib/commands/runner.rb:48: /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `log': Mysql::Error: Duplicate entry '20-5c4e52e49648c196f629b8623b4aca7e4c0080e3' for key 'changesets_repos_rev': INSERT INTO `changesets` (`committed_on`, `comments`, `commit_date`, `scmid`, `user_id`, `revision`, `repository_id`, `committer`) VALUES('2011-08-04 10:00:11', 'Fix the bitwidth of the remaining fields.\n\ngit-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136884 91177308-0d34-0410-b5e6-96231b3b80d8', '2011-08-04 10:00:11', '5c4e52e49648c196f629b8623b4aca7e4c0080e3', NULL, '5c4e52e49648c196f629b8623b4aca7e4c0080e3', 20, 'Rafael Espindola <rafael.espindola@gmail.com>') (ActiveRecord::StatementInvalid)
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:324:in `execute'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:259:in `insert_sql'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:334:in `insert_sql'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:44:in `insert_without_query_dirty'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `insert'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/base.rb:2961:in `create_without_timestamps'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/timestamp.rb:53:in `create_without_callbacks'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/callbacks.rb:266:in `create'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/base.rb:2927:in `create_or_update_without_callbacks'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/callbacks.rb:250:in `create_or_update'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/base.rb:2577:in `save_without_validation'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/validations.rb:1090:in `save_without_dirty'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/dirty.rb:79:in `save_without_transactions'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:229:in `send'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:229:in `with_transaction_returning_status'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:136:in `transaction'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:182:in `transaction'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:228:in `with_transaction_returning_status'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:196:in `save'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:208:in `rollback_active_record_state!'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:196:in `save'
    from /srv/redmine/app/models/repository/git.rb:152:in `save_revision'
    from /srv/redmine/app/models/repository/git.rb:133:in `fetch_changesets'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:136:in `transaction'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:182:in `transaction'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/transactions.rb:188:in `transaction'
    from /srv/redmine/app/models/repository/git.rb:131:in `fetch_changesets'
    from /srv/redmine/lib/redmine/scm/adapters/git_adapter.rb:222:in `revisions'
    from /srv/redmine/lib/redmine/scm/adapters/git_adapter.rb:207:in `each_line'
    from /srv/redmine/lib/redmine/scm/adapters/git_adapter.rb:207:in `revisions'
    from /srv/redmine/lib/redmine/scm/adapters/abstract_adapter.rb:215:in `call'
    from /srv/redmine/lib/redmine/scm/adapters/abstract_adapter.rb:215:in `shellout'
    from /srv/redmine/lib/redmine/scm/adapters/abstract_adapter.rb:213:in `popen'
    from /srv/redmine/lib/redmine/scm/adapters/abstract_adapter.rb:213:in `shellout'
    from /srv/redmine/lib/redmine/scm/adapters/abstract_adapter.rb:194:in `shellout'
    from /srv/redmine/lib/redmine/scm/adapters/git_adapter.rb:368:in `scm_cmd'
    from /srv/redmine/lib/redmine/scm/adapters/git_adapter.rb:202:in `revisions'
    from /srv/redmine/app/models/repository/git.rb:129:in `fetch_changesets'
    from /srv/redmine/app/models/repository/git.rb:125:in `each'
    from /srv/redmine/app/models/repository/git.rb:125:in `fetch_changesets'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/associations/association_proxy.rb:215:in `send'
    from /srv/redmine/vendor/rails/activerecord/lib/active_record/associations/association_proxy.rb:215:in `method_missing'
    from /srv/redmine/app/models/repository.rb:243:in `fetch_changesets'
    from /srv/redmine/app/models/repository.rb:240:in `each'
    from /srv/redmine/app/models/repository.rb:240:in `fetch_changesets'
    from (eval):1
    from /usr/local/stow/ruby-enterprise-1.8.7-2010.02/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `eval'
    from /srv/redmine/vendor/rails/railties/lib/commands/runner.rb:48
    from /usr/local/stow/ruby-enterprise-1.8.7-2010.02/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
    from /usr/local/stow/ruby-enterprise-1.8.7-2010.02/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
    from script/runner:3

History

#1 Updated by Dave Abrahams about 9 years ago

Further info: as a result, I can look explicitly at the last revision on the "work" branch, but not at its diffs:

Processing RepositoriesController#revision (for 207.172.223.249 at 2011-08-05 04:37:53) [GET]
  Parameters: {"rev"=>"35f33dc9dfadc79e97d568ebaae51fcebbd285a0", "action"=>"revision", "id"=>"embt", "controller"=>"repositories"}
Rendering template within layouts/base
Rendering repositories/revision
Completed in 72ms (View: 61, DB: 3) | 200 OK [https://hub.boostpro.com/projects/embt/repository/revisions/35f33dc9dfadc79e97d568ebaae51fcebbd285a0]

Processing RepositoriesController#diff (for 207.172.223.249 at 2011-08-05 04:38:16) [GET]
  Parameters: {"rev"=>"35f33dc9dfadc79e97d568ebaae51fcebbd285a0", "action"=>"diff", "id"=>"embt", "controller"=>"repositories"}
Rendering template within layouts/base
Rendering common/error (404)
Completed in 105ms (View: 62, DB: 4) | 404 Not Found [https://hub.boostpro.com/projects/embt/repository/revisions/35f33dc9dfadc79e97d568ebaae51fcebbd285a0/diff]

#2 Updated by Toshi MARUYAMA about 9 years ago

  • Subject changed from Error when fetching changesets to Git: Error when fetching changesets
  • Status changed from New to Closed
  • Priority changed from High to Low
  • Resolution set to Invalid

It seems your database is corruption and git path and permission problem.
Please post to forum.

#3 Updated by Dave Abrahams about 9 years ago

Hours later, the diffs are showing up. I'll take this to the forum as requested, though.

#4 Updated by LluĂ­s Vilanova over 8 years ago

  • Status changed from Closed to Reopened

It seems like there actually is a problem, at least, on the interaction between redmine and mysql.

After pushing the contents of a git repository with a large history, first click on the repository tab on redmine starts the fetch of the changesets. Once that is apparently finished, mysql starts eating up a lot of I/O time (seen using iotop) and the aforementioned errors appear until mysql calms down a lot of time after that (I suppose it's doing something with the newly inserted data).

The error message shown in redmine's reporitory tab shows errors for different revision IDs every time you reload the page, which brings me to suspect these are made available little by little by mysql.

Hope this helps on pinning the exact source of the problem.

#5 Updated by Toshi MARUYAMA over 8 years ago

  • Status changed from Reopened to Closed

Please do not reopen by unrelated issues.
Git fetching revisions is refactored by #8857.

#6 Updated by Ethel Mambret over 8 years ago

Got same error as above on a working repository with redmine version 1.3.1 after some pushes on the git server.
Below the error:

An error occurred when trying to access the repository: Mysql::Error: Duplicate entry '3-0fcedca25abdf4160eeed7749a67800f5109b355' for key 'changesets_repos_rev': INSERT INTO `changesets` (`committed_on`, `comments`, `commit_date`, `scmid`, `user_id`, `revision`, `repository_id`, `committer`) VALUES('2012-04-02 17:45:15', 'added directory gammun_project containing old ASP website', '2012-04-02 17:45:15', '0fcedca25abdf4160eeed7749a67800f5109b355', NULL, '0fcedca25abdf4160eeed7749a67800f5109b355', 3, 'Ethel <>')

#7 Updated by Ethel Mambret over 8 years ago

  • Status changed from Closed to Reopened

#8 Updated by Toshi MARUYAMA over 8 years ago

As I described at #10470 note-13 ,
revision column and it is unique.

I changed transaction code at r9288 and r9290.
Please open new issues if still have this issue on Redmine 1.4.

#9 Updated by Toshi MARUYAMA over 8 years ago

  • Status changed from Reopened to Closed
  • Resolution deleted (Invalid)

Also available in: Atom PDF