Feature #1273

Need a way to re-sync repository history

Added by Anonymous over 6 years ago. Updated 4 months ago.

Status:NewStart date:2008-05-21
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:SCM
Target version:-
Resolution:

Description

I had to edit the regex used to pass changeset history (see #1253). This didn't associate the revisions/issues immediately, I had to delete the repository from the project and re-add it to get the associations updated. A way to force the re-sync would have saved doing this.

Trac also has this functionality which can be run from the command-line periodically because subversion (not sure about other back-ends) does allow you to edit log messages after a commit so a resync would pick these up.

  1. Manually through the command-line, similar to the fetch_changesets (which would enable scheduled daily resyncs)
  2. A re-sync history on the repositroy settings tab

Ideally both would be good.


Related issues

Related to Defect #3724: Mercurial repositories display revision ID instead of cha... Closed 2009-08-10
Related to Feature #4823: Don't evaluate commit-message "refs, closes, ..." when ad... New 2010-02-12
Related to Patch #7959: Patch for repository revision update (reload) New 2011-03-22
Related to Feature #5501: Git: Mercurial: Adding visual merge/branch history to rep... Closed 2010-05-11
Related to Defect #9897: Git: revisions and branches deleted in Git repository are... New
Related to Defect #10172: There is no lines on revision graph between revisions in ... New
Related to Feature #12853: Removing git branch result of database inconsistencies New
Related to Feature #13538: Role Based Access Control on Subversion commit log New
Duplicated by Feature #6024: Single-button repository reset Closed 2010-08-03
Duplicated by Feature #2667: subversion commit info modification (using hook) Closed 2009-02-04
Duplicated by Feature #5963: Reload SCM commit comments Closed 2010-07-26
Duplicated by Feature #3533: Editing subversion log message does not update 'Repositor... Closed 2009-06-24

History

#1 Updated by Oleg Marchuk over 5 years ago

Is there any progress?

#2 Updated by Anonymous over 5 years ago

I guess not, as it would be shown on the ticket, but I hope it can be addressed soon. Its a pain having to keep deleting the repository and then re-adding it to the project.

#3 Updated by Seth Voltz over 5 years ago

Trac also has this functionality which can be run from the command-line periodically because subversion (not sure about other back-ends) does allow you to edit log messages after a commit so a resync would pick these up.

  1. Manually through the command-line, similar to the fetch_changesets (which would enable scheduled daily resyncs)

I was looking for the same thing. Over at Jumpbox I found a solution which allows a cron job to periodically resync the repo.

cd /path/to/your/redmine
sudo ruby script/runner "Repository.fetch_changesets" -e production

#4 Updated by Anonymous over 5 years ago

Seth Voltz wrote:

I was looking for the same thing. Over at Jumpbox I found a solution which allows a cron job to periodically resync the repo.

We do this at the moment but that only fetches new changes sets. It doesn't resync ones that have already been fetched. We run fetch_changesets every 5 minutes at the moment. I'd like to generate a similar script that would be run nightly to do a full re-sync but finding time is always a pain and plus I know nothing about ruby. We'll see.

#5 Updated by Pablo Ruiz about 5 years ago

After looking at's redmine's code.. You can resync a repo by issuing the following SQL statements:

DELETE FROM changes WHERE changes.changeset_id IN (SELECT changesets.id FROM changesets WHERE changesets.repository_id = _ID_OF_YOUR_REPO);
DELETE FROM changesets_issues WHERE changesets_issues.changeset_id IN (SELECT changesets.id FROM changesets WHERE changesets.repository_id = _ID_OF_YOUR_REPO);
DELETE FROM changesets WHERE changesets.repository_id = _ID_OF_YOUR_REPO;

Now you can exec 'ruby script/runner "Repository.fetch_changesets" -e production' to force resync of repo data..

#6 Updated by Anonymous about 5 years ago

Pablo Ruiz wrote:

After looking at's redmine's code.. You can resync a repo by issuing the following SQL statements:

DELETE FROM changes WHERE changes.changeset_id IN (SELECT changesets.id FROM changesets WHERE changesets.repository_id = _ID_OF_YOUR_REPO);
DELETE FROM changesets_issues WHERE changesets_issues.changeset_id IN (SELECT changesets.id FROM changesets WHERE changesets.repository_id = _ID_OF_YOUR_REPO);
DELETE FROM changesets WHERE changesets.repository_id = _ID_OF_YOUR_REPO;

Now you can exec 'ruby script/runner "Repository.fetch_changesets" -e production' to force resync of repo data..

Thanks Pablo. Could I just do a delete all from changes, changesets_issues and changesets to resync all revisions of all projects? That is what we're actually looking for.

I may take a look at the fetch_changesets script and see if I can create a delete_changesets script that does this. I'm reluctant to start using SQL to directly access the redmine db incase

  • I break something
  • The schema changes in an update and I break something

Cheers

Russell

#7 Updated by Daniel Dehennin almost 2 years ago

I may take a look at the fetch_changesets script and see if I can create a delete_changesets script that does this. I'm reluctant to start using SQL to directly access the redmine db incase

  • I break something
  • The schema changes in an update and I break something

Hello, any news on a resync script?

I have an issue after an upgrade, some git commitID in the database are wrong, I put some logs in source:tags/1.4.4/lib/redmine/scm/adapters/git_adapter.rb#L198 to get the list of commitID causing errors and I wonder if I can only delete thoses.

We have 200 git projects (3.1Go) and wonder if a complete re-sync is not too much.

Regards

#8 Updated by Pavel Pivovarov over 1 year ago

Any improvements in this ticket?
Now I clean tables
"changes"
"changeset_parents"
"changesets"
"changesets_issues"
and do resync of repo data. It takes about hour to get job done, and after all there no revisions in repositories at all. I see all cleaned tables are filled with new data. What could I do with this? I have about 40 connected repositories (almost all are GIT) and remake all repos once a week it's a bad idea. Why "Repository.fetch_changesets" don't create revisions.

#9 Updated by Clemens R over 1 year ago

This is a very important feature. If our developers do change their comments no one gets notified!

+1

#10 Updated by Olivier Brisse over 1 year ago

Pavel Pivovarov wrote:

Any improvements in this ticket?
Now I clean tables
"changes"
"changeset_parents"
"changesets"
"changesets_issues"
and do resync of repo data. It takes about hour to get job done, and after all there no revisions in repositories at all. I see all cleaned tables are filled with new data. What could I do with this? I have about 40 connected repositories (almost all are GIT) and remake all repos once a week it's a bad idea. Why "Repository.fetch_changesets" don't create revisions.

That's how I do it:

repo = Repository.find(_REPO_ID_)
repo.send(:clear_changesets)
repo.extra_info["heads"] = []
repo.save

clear_changesets does all the SQL delete for you.

Seems to be work fine for me. extra_info["heads"] was an array of SHA before and nothing was happening.

#11 Updated by Daniel Felix over 1 year ago

Maybe this could be done within the administration. There could be some button, which just cleans the mentioned tables. The administrator could be started by the administrator.

#12 Updated by Harry Garrood over 1 year ago

A much more sensible way would be to have Redmine allow changing commit messages through the REST API. Then, post-commit hooks can be set up in version control so that only the single changed commit message is modified, instead of deleting and re-importing the whole history.

#13 Updated by Marcel Nadje over 1 year ago

We have set up a post-revprop-change-hook in subversion which

1. executes the following sql in the redmine database after the user has changed a commit message:
"DELETE FROM changesets WHERE repository_id IN (SELECT id FROM repositories WHERE project_id IN (SELECT id FROM projects WHERE identifier='#PROJECT_ID#') AND is_default = 1) AND revision>=#REV#;"

The sql statement deletes all rows in the changesets table from the default repository of the project with a revision >= the changed commit message in the subversion repositiory. So not all rows were deleted.

2. executes the HTTP GET command fetch_changesets to reload the deleted commit messages from the subversion repository.

I think a good feature is a HTTP GET command clear_changesets with the parameters project identifier, repositpry identifier and revision number. So redmine can do detetes in the right tables.
An alternative way is to extend the fetch_changesets command with an additional parameter revision number. If you call the existing command fetch_changesets with this additional parameter revision number then redmine is able to delete this single revision in the database an fetch the updated one from the scm repository.

#14 Updated by Daniel Dehennin over 1 year ago

Olivier Brisse wrote:

That's how I do it:

repo = Repository.find(_REPO_ID_)
repo.send(:clear_changesets)
repo.extra_info["heads"] = []
repo.save

clear_changesets does all the SQL delete for you.

Seems to be work fine for me. extra_info["heads"] was an array of SHA before and nothing was happening.

Hello, I'm not used to run redmine scripts and I wonder how you run this code.

Which library to include, etc.

Any hints?

Regards.

#15 Updated by Toshi MARUYAMA over 1 year ago

Daniel Dehennin wrote:

Hello, I'm not used to run redmine scripts and I wonder how you run this code.

On upper Redmine 2.0.0:

$ ruby script/rails console

#16 Updated by Daniel Dehennin over 1 year ago

Toshi MARUYAMA wrote:

Daniel Dehennin wrote:

Hello, I'm not used to run redmine scripts and I wonder how you run this code.

On upper Redmine 2.0.0:
[...]

Thank you very much, I have an issue with automatic spent time in commit messages, they are not cleared on :clear_changesets, so they cumulate each time I apply the procedure.

Regards.

#17 Updated by Daniel Dehennin over 1 year ago

Daniel Dehennin wrote:

Thank you very much, I have an issue with automatic spent time in commit messages, they are not cleared on :clear_changesets, so they cumulate each time I apply the procedure.

It duplicates comments in issues to.

Regards.

#18 Updated by Isaac Betesh 8 months ago

Building on Daniel Dehennin's answer, I dropped the following code into lib/tasks/reload_changesets.rb, so I can now run it as a rake task.

namespace :db do
  desc 'Clear all repositories so they can be reloaded'
  task :reload_changesets => :environment do
    Repository.all.each do |repo|
      repo.send(:clear_changesets)
      repo.extra_info["heads"] = []
      repo.save!
    end
  end
end

Note that I'm resetting all repositories here, but you might want to change that to take specific args, for instance using

RAILS_ENV=production REPO_IDS=10,11,12 rake db:reload_changesets

and changing "Repository.all" to "Repository.where(:id => ENV['REPO_IDS'])"

#19 Updated by Michael Kling 4 months ago

Thanks Isaac Betesh - your rake task solved it for me.

Just one small change: i had to save it under lib/tasks/reload_changesets.rake, otherwise rake couldn't find it.
(im on ruby 1.9.3, rails 3.2.17 and redmine 2.4.3)

Also available in: Atom PDF