Defect #5096

Redmine hangs up while browsing Git repository

Added by Vladimir Babchuk over 7 years ago. Updated almost 7 years ago.

Status:ClosedStart date:2010-03-16
Priority:NormalDue date:
Assignee:Toshi MARUYAMA% Done:

50%

Category:SCM
Target version:1.1.0
Resolution:Fixed Affected version:0.9.3

Description

git version 1.7.0.1
redmine version 0.9.3
rails are shipped with redmine (Rails 2.3.5)
ruby 1.8.7 (2009-12-24 patchlevel 248) [i386-freebsd8]
postgres (PostgreSQL) 8.2.15

other gems:
actionmailer (2.3.5, 2.3.4)
actionpack (2.3.5, 2.3.4)
activerecord (2.3.5, 2.3.4)
activeresource (2.3.5, 2.3.4)
activesupport (2.3.5, 2.3.4)
cgi_multipart_eof_fix (2.5.0)
daemons (1.0.10)
fastthread (1.0.7)
gem_plugin (0.2.3)
mongrel (1.1.5)
postgres (0.7.9.2008.01.28)
rack (1.1.0, 1.0.1)
rake (0.8.7)
sources (0.0.2)

While randomly browsing git repository whole site stops to respond.
While trying to stop mongrel I got this in production.log:

@Processing RepositoriesController#show (for 10.10.17.1 at 2010-03-16 15:01:43) [GET]
Parameters: {"action"=>"show", "id"=>"terminal-sdk", "controller"=>"repositories"}

Mongrel::TimeoutError (Mongrel timed out this thread: shutdown):
lib/redmine/scm/adapters/git_adapter.rb:92:in `lastrev'
lib/redmine/scm/adapters/git_adapter.rb:78
lib/redmine/scm/adapters/git_adapter.rb:66:in `each_line'
lib/redmine/scm/adapters/git_adapter.rb:66
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `call'
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `popen'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:177:in `shellout'
lib/redmine/scm/adapters/git_adapter.rb:65:in `entries'
app/models/repository.rb:63:in `entries'
app/controllers/repositories_controller.rb:72:in `show'
@

Also tried to use Webrick. No diference.

git-io-gets.diff Magnifier (1.49 KB) Toshi MARUYAMA, 2010-11-23 09:37


Related issues

Related to Redmine - Defect #7047: Git adapter very slow when a commit modifies a lot of files New 2010-12-04

Associated revisions

Revision 4624
Added by Toshi MARUYAMA almost 7 years ago

repository: git: change io.gets to io.readlines (#5404, #5096).

History

#1 Updated by Toshi MARUYAMA over 7 years ago

#2 Updated by Vladimir Babchuk over 7 years ago

Toshi Maruyama wrote:

Please see http://www.redmine.org/issues/4773#note-14 and try patch of #4773.

Patches didn't help.. (First patch was already in release, second - didn't help).
It seems that code hungs at call to shellout function:
Mongrel::TimeoutError (Mongrel timed out this thread: shutdown):
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:177:in `shellout'
lib/redmine/scm/adapters/git_adapter.rb:92:in `lastrev'
lib/redmine/scm/adapters/git_adapter.rb:78
lib/redmine/scm/adapters/git_adapter.rb:66:in `each_line'
lib/redmine/scm/adapters/git_adapter.rb:66
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `call'
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `popen'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:177:in `shellout'
lib/redmine/scm/adapters/git_adapter.rb:65:in `entries'
app/models/repository.rb:63:in `entries'
app/controllers/repositories_controller.rb:72:in `show'

#3 Updated by Toshi MARUYAMA over 7 years ago

Vladimir Babchuk wrote:

Patches didn't help.. (First patch was already in release, second - didn't help).

Sorry, patch is http://www.redmine.org/attachments/3272/git-fast-browse.patch .

#4 Updated by Vladimir Babchuk over 7 years ago

Toshi Maruyama wrote:

Vladimir Babchuk wrote:

Patches didn't help.. (First patch was already in release, second - didn't help).

Sorry, patch is http://www.redmine.org/attachments/3272/git-fast-browse.patch .

Thanks! That patch helped me.
But... it like a hack (now I can't see revision, comment and etc. in repo browser). So problem is still remained.
Any thoughts how to fix it right? I can tree it in free time, but i can't realize what is causes it.
If execute commands that executes shellout by hand - git exits instantly.
By default git uses less to page results, but I disabled it. What else can cause thread lockup?

#5 Updated by Toshi MARUYAMA over 7 years ago

Vladimir Babchuk wrote:

Thanks! That patch helped me.
But... it like a hack (now I can't see revision, comment and etc. in repo browser). So problem is still remained.
Any thoughts how to fix it right? I can tree it in free time, but i can't realize what is causes it.
If execute commands that executes shellout by hand - git exits instantly.
By default git uses less to page results, but I disabled it. What else can cause thread lockup?

http://www.redmine.org/issues/4773#note-12

Actually, what is really slow when browsing a git repository is to get information about last change on each file using git log -n 1. When removing this, browsing is pretty fast. Maybe we should disable this.

#6 Updated by Vladimir Babchuk over 7 years ago

Toshi Maruyama wrote:

Actually, what is really slow when browsing a git repository is to get information about last change on each file using git log -n 1. When removing this, browsing is pretty fast. Maybe we should disable this.

Yes, I see. But in my repository only near ~15 files and ~10 commits. It can't be parsed so long! I think - there are other cause of this problem.

#7 Updated by Mitch Pirtle over 7 years ago

I'm having the same performance problems related to git repositories. First the obligatory ENV schtuff:

  • Redmine 0.9.2.devel.3437 (MySQL)
  • ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux]
  • Rails 2.3.5
  • mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (x86_64) using readline 5.2

Info about repo:

  • git version 1.5.6.5
  • 400 commits
  • < 5,000 files

Troubling for me is how redmine completely blocks until this view is rendered, and we're talking several minutes.

#8 Updated by Felix Schäfer over 7 years ago

Mitch Pirtle wrote:

  • Redmine 0.9.2.devel.3437 (MySQL)

Which has known performance issue with git, I don't they have been backported to 0.9.x, but definitely are in 1.0.x and trunk.

Troubling for me is how redmine completely blocks until this view is rendered, and we're talking several minutes.

What rails server do you use? If it's a single webrick or mongrel, that happens because multi-threading/multiple incoming connections are handled by spinning up more instances, which neither of both does on itself.

#9 Updated by Mitch Pirtle over 7 years ago

Thanks for the explanation Felix.

This is a dev server running lighttpd, which probably explains the blocking.

Will update to trunk when I get the chance, as we could really use some relief.

#10 Updated by Toshi MARUYAMA about 7 years ago

Redmine git adapter use "io.gets" at source:tags/1.0.3/lib/redmine/scm/adapters/git_adapter.rb#L94 .
It seems there is same problem with #6860.
It maybe we should use "io.each_line" instead of "io.gets".

#11 Updated by Toshi MARUYAMA almost 7 years ago

I create a patch replacing "io.gets" and I attach it.

#12 Updated by Toshi MARUYAMA almost 7 years ago

  • Assignee set to Toshi MARUYAMA
  • Target version set to 1.1.0

#13 Updated by Toshi MARUYAMA almost 7 years ago

  • Status changed from New to Closed
  • Resolution set to Fixed

This issue's original problem seems to io.gets() problem.
I committed r4624 in svn trunk and r4627 in 1.1.
Git browsing performance issue is #7047.
I close this issue.

Also available in: Atom PDF