Feature #6566

Convert the git adapter to rugged

Added by Felix Schäfer over 8 years ago. Updated about 6 years ago.

Status:NewStart date:2010-10-04
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:SCM
Target version:-
Resolution:

Description

There's been some talk about converting the git adapter to use git, but nothing concrete yet. I'll give it a try myself an post the link to the git branch here.

There have also been concerns about the speed of grit, but seeing it is used by github and implements stuff either as shell-outs or direct access depending on speed, I'd say it is fast enough for us. I will nonetheless try to provide a performance test to see how both adapters perform against a real-world git repository (probably against redmine.git ;-) ).

I've added the people whom I think might be interested in this matter as watchers, feel free to unwatch if this doesn't apply to you.


Related issues

Related to Redmine - Defect #7146: Git adapter lost commits before 7 days from database late... Closed 2010-12-21
Related to Redmine - Defect #6013: git tab,browsing, very slow -- even after first time Closed 2010-08-02
Related to Redmine - Defect #5357: Git: SCM revisions ordered by date/time (should be revers... New 2010-04-20

History

#1 Updated by Eric Davis over 8 years ago

When we added git branch support, there were a few patches that used grit. Might be useful to dig though that code and see if there is anything we can use.

#2 Updated by Felix Schäfer over 8 years ago

I was rather planning on making a "clean" adapter, but I'll sure have a look :-)

#3 Updated by michael turner over 8 years ago

This would be interesting if you can pull it off, although the few minutes I spent poking at Grit (and redmine.git) recently doesn't fill me with confidence. It went something like this...

$ irb
ruby-1.8.7-p249 > require 'grit'
 => true 

ruby-1.8.7-p249 > repo_path = '/home/michael/rails/redmine'
 => "/home/michael/rails/redmine" 

ruby-1.8.7-p249 > grit = Grit::Repo.new(repo_path)
 => #<Grit::Repo "/home/michael/rails/redmine/.git"> 

ruby-1.8.7-p249 > recent_list = grit.commits_since(start = 'master', since = '2010-10-10', extra_options = {}) 
SystemStackError: stack level too deep
## full traceback omitted ...

ruby-1.8.7-p249 > recent_list_alt =  `cd #{repo_path}; git log --since='2010-10-10' --format='%H'`.split
## succeeded. full array of results omitted ...

ruby-1.8.7-p249 > recent_list_alt.count
 => 81 

ruby-1.8.7-p249 > Grit.version 
 => "2.3.0" 

Digging a little deeper revealed that Grit attempts to load every commit in the repo, and then handle your constraints (since = DATE, etc.) by filtering the complete list in ruby before returning it to you. Apparently redmine.git is a little too large for that approach to work (not even taking performance into account).
Hopefully I'm just doing it wrong and you'll have better luck with it. :)

#4 Updated by Jean-Philippe Lang over 8 years ago

  • Target version deleted (Unplanned backlogs)

Removing it from the Unplanned features since it doesn't seem to be usable and is no longer updated (last release is 2 years old).

#5 Updated by John Dell over 8 years ago

Not saying it works, but last release was just yesterday (2.4.1) https://github.com/mojombo/grit/ so it certainly is being actively developed.

#6 Updated by Jean-Philippe Lang over 8 years ago

Right. I was looking at rubyforge :/

#7 Updated by Felix Schäfer over 8 years ago

  • Assignee deleted (Felix Schäfer)

#8 Updated by Jean-Philippe Lang over 6 years ago

  • Status changed from 7 to New

Assigned issue with no assignee back to New status.

#9 Updated by Gabriel Mazetto over 6 years ago

Please change this issue to "Covert the git adapter to rugged". Rugged is the oficial ruby library with ruby bindings for libgit2. You can find more about it here: https://github.com/libgit2/rugged

#10 Updated by Gabriel Mazetto over 6 years ago

I haven't found a way to make searches by date without having to walk over all commits, but found some useful ways of walking over the commit trees (to get for example from x to y hashes, that is probably what's going to be used to get latest commits:


1.9.3-p327 > require 'rugged'
 => true 

1.9.3-p327 > repo_path = '/path/to/redmine'
 => "/path/to/redmine" 

1.9.3-p327 > repo = Rugged::Repository.new(repo_path)
 => #<Rugged::Repository:0x007fa208972820>

1.9.3-p327 > walker = Rugged::Walker.new(repo)
 => #<Rugged::Walker:0x007f8092965d38>

1.9.3-p327 > walker.push("301d7e7cb13a09854a7c87aecedfcbfd27eb98d3")
1.9.3-p327 > walker.hide("c31f498ba6a21fd3e5ce7b9ba2f3b3cdc1b2e05b")

1.9.3-p327 > result = walker.to_a
 => [#<Rugged::Commit:0x007f8092977560>, #<Rugged::Commit:0x007f8092977538>, #<Rugged::Commit:0x007f8092977510>] 

or you can just iterate over for a more memory friendly operation:

1.9.3-p327 > walker.push("301d7e7cb13a09854a7c87aecedfcbfd27eb98d3")
1.9.3-p327 > walker.hide("c31f498ba6a21fd3e5ce7b9ba2f3b3cdc1b2e05b")

1.9.3-p327 > walker.each {|c| puts c.inspect}
#<Rugged::Commit:0x007f809295d2f0>
#<Rugged::Commit:0x007f809295d2a0>
#<Rugged::Commit:0x007f809295d250>

A commit object have the following methods:

=> [:message, :time, :committer, :author, :tree, :parents, :to_hash,..., :oid, :type,...]
1.9.3-p327 > c.to_hash
 => {:message=>"German translation updated by Daniel Felix (#10191)\n\ngit-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@10998 e93f8b46-1217-0410-a6f0-8f06a7374b81\n", :committer=>{:name=>"Toshi MARUYAMA", :email=>"marutosijp2@yahoo.co.jp", :time=>2012-12-14 14:47:46 UTC}, :author=>{:name=>"Toshi MARUYAMA", :email=>"marutosijp2@yahoo.co.jp", :time=>2012-12-14 14:47:46 UTC}, :tree=>#<Rugged::Tree:0x007f8091100130>, :parents=>[#<Rugged::Commit:0x007f80911006d0>]}

1.9.3-p327 > c.oid
 => "301d7e7cb13a09854a7c87aecedfcbfd27eb98d3" 

1.9.3-p327 > c.type
 => "commit" 

1.9.3-p327 > Rugged::Version
 => "0.16.0" 

#11 Updated by Toshi MARUYAMA over 6 years ago

  • Subject changed from Convert the git adapter to grit to Covert the git adapter to rugged

#12 Updated by Toshi MARUYAMA over 6 years ago

Gabriel Mazetto wrote:

Please change this issue to "Covert the git adapter to rugged".

Done.

#13 Updated by Jean-Philippe Lang over 6 years ago

  • Subject changed from Covert the git adapter to rugged to Convert the git adapter to rugged

Typo

#14 Updated by Gabriel Mazetto about 6 years ago

I decided to try making the conversion... and have some questions:
Do I have to use "AbstractAdapter"? It was designed to be used with command line in mind, which is not the case with rugged. Should I try to create a lean "AbstractAdapter" with libraries in mind? I saw another issue talking about libsvn for example, that could be the potential for this new abstract library, with only the "SCM actions" in mind.

Also available in: Atom PDF