Project

General

Profile

Actions

Defect #5096

closed

Redmine hangs up while browsing Git repository

Added by Vladimir Babchuk about 14 years ago. Updated about 13 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Toshi MARUYAMA
Category:
SCM
Target version:
Start date:
2010-03-16
Due date:
% Done:

50%

Estimated time:
Resolution:
Fixed
Affected version:

Description

git version 1.7.0.1
redmine version 0.9.3
rails are shipped with redmine (Rails 2.3.5)
ruby 1.8.7 (2009-12-24 patchlevel 248) [i386-freebsd8]
postgres (PostgreSQL) 8.2.15

other gems:
actionmailer (2.3.5, 2.3.4)
actionpack (2.3.5, 2.3.4)
activerecord (2.3.5, 2.3.4)
activeresource (2.3.5, 2.3.4)
activesupport (2.3.5, 2.3.4)
cgi_multipart_eof_fix (2.5.0)
daemons (1.0.10)
fastthread (1.0.7)
gem_plugin (0.2.3)
mongrel (1.1.5)
postgres (0.7.9.2008.01.28)
rack (1.1.0, 1.0.1)
rake (0.8.7)
sources (0.0.2)

While randomly browsing git repository whole site stops to respond.
While trying to stop mongrel I got this in production.log:

@Processing RepositoriesController#show (for 10.10.17.1 at 2010-03-16 15:01:43) [GET]
Parameters: {"action"=>"show", "id"=>"terminal-sdk", "controller"=>"repositories"}

Mongrel::TimeoutError (Mongrel timed out this thread: shutdown):
lib/redmine/scm/adapters/git_adapter.rb:92:in `lastrev'
lib/redmine/scm/adapters/git_adapter.rb:78
lib/redmine/scm/adapters/git_adapter.rb:66:in `each_line'
lib/redmine/scm/adapters/git_adapter.rb:66
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `call'
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `popen'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:177:in `shellout'
lib/redmine/scm/adapters/git_adapter.rb:65:in `entries'
app/models/repository.rb:63:in `entries'
app/controllers/repositories_controller.rb:72:in `show'
@

Also tried to use Webrick. No diference.


Files

git-io-gets.diff (1.49 KB) git-io-gets.diff Toshi MARUYAMA, 2010-11-23 09:37

Related issues

Related to Redmine - Defect #7047: Git adapter very slow when a commit modifies a lot of filesNew2010-12-04

Actions
Actions #1

Updated by Toshi MARUYAMA about 14 years ago

Actions #2

Updated by Vladimir Babchuk about 14 years ago

Toshi Maruyama wrote:

Please see http://www.redmine.org/issues/4773#note-14 and try patch of #4773.

Patches didn't help.. (First patch was already in release, second - didn't help).
It seems that code hungs at call to shellout function:
Mongrel::TimeoutError (Mongrel timed out this thread: shutdown):
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:177:in `shellout'
lib/redmine/scm/adapters/git_adapter.rb:92:in `lastrev'
lib/redmine/scm/adapters/git_adapter.rb:78
lib/redmine/scm/adapters/git_adapter.rb:66:in `each_line'
lib/redmine/scm/adapters/git_adapter.rb:66
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `call'
lib/redmine/scm/adapters/abstract_adapter.rb:193:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `popen'
lib/redmine/scm/adapters/abstract_adapter.rb:191:in `shellout'
lib/redmine/scm/adapters/abstract_adapter.rb:177:in `shellout'
lib/redmine/scm/adapters/git_adapter.rb:65:in `entries'
app/models/repository.rb:63:in `entries'
app/controllers/repositories_controller.rb:72:in `show'

Actions #3

Updated by Toshi MARUYAMA about 14 years ago

Vladimir Babchuk wrote:

Patches didn't help.. (First patch was already in release, second - didn't help).

Sorry, patch is http://www.redmine.org/attachments/3272/git-fast-browse.patch .

Actions #4

Updated by Vladimir Babchuk about 14 years ago

Toshi Maruyama wrote:

Vladimir Babchuk wrote:

Patches didn't help.. (First patch was already in release, second - didn't help).

Sorry, patch is http://www.redmine.org/attachments/3272/git-fast-browse.patch .

Thanks! That patch helped me.
But... it like a hack (now I can't see revision, comment and etc. in repo browser). So problem is still remained.
Any thoughts how to fix it right? I can tree it in free time, but i can't realize what is causes it.
If execute commands that executes shellout by hand - git exits instantly.
By default git uses less to page results, but I disabled it. What else can cause thread lockup?

Actions #5

Updated by Toshi MARUYAMA about 14 years ago

Vladimir Babchuk wrote:

Thanks! That patch helped me.
But... it like a hack (now I can't see revision, comment and etc. in repo browser). So problem is still remained.
Any thoughts how to fix it right? I can tree it in free time, but i can't realize what is causes it.
If execute commands that executes shellout by hand - git exits instantly.
By default git uses less to page results, but I disabled it. What else can cause thread lockup?

http://www.redmine.org/issues/4773#note-12

Actually, what is really slow when browsing a git repository is to get information about last change on each file using git log -n 1. When removing this, browsing is pretty fast. Maybe we should disable this.

Actions #6

Updated by Vladimir Babchuk about 14 years ago

Toshi Maruyama wrote:

Actually, what is really slow when browsing a git repository is to get information about last change on each file using git log -n 1. When removing this, browsing is pretty fast. Maybe we should disable this.

Yes, I see. But in my repository only near ~15 files and ~10 commits. It can't be parsed so long! I think - there are other cause of this problem.

Actions #7

Updated by Mitch Pirtle over 13 years ago

I'm having the same performance problems related to git repositories. First the obligatory ENV schtuff:

  • Redmine 0.9.2.devel.3437 (MySQL)
  • ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux]
  • Rails 2.3.5
  • mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (x86_64) using readline 5.2

Info about repo:

  • git version 1.5.6.5
  • 400 commits
  • < 5,000 files

Troubling for me is how redmine completely blocks until this view is rendered, and we're talking several minutes.

Actions #8

Updated by Felix Schäfer over 13 years ago

Mitch Pirtle wrote:

  • Redmine 0.9.2.devel.3437 (MySQL)

Which has known performance issue with git, I don't they have been backported to 0.9.x, but definitely are in 1.0.x and trunk.

Troubling for me is how redmine completely blocks until this view is rendered, and we're talking several minutes.

What rails server do you use? If it's a single webrick or mongrel, that happens because multi-threading/multiple incoming connections are handled by spinning up more instances, which neither of both does on itself.

Actions #9

Updated by Mitch Pirtle over 13 years ago

Thanks for the explanation Felix.

This is a dev server running lighttpd, which probably explains the blocking.

Will update to trunk when I get the chance, as we could really use some relief.

Actions #10

Updated by Toshi MARUYAMA over 13 years ago

Redmine git adapter use "io.gets" at source:tags/1.0.3/lib/redmine/scm/adapters/git_adapter.rb#L94 .
It seems there is same problem with #6860.
It maybe we should use "io.each_line" instead of "io.gets".

Actions #11

Updated by Toshi MARUYAMA over 13 years ago

I create a patch replacing "io.gets" and I attach it.

Actions #12

Updated by Toshi MARUYAMA about 13 years ago

  • Assignee set to Toshi MARUYAMA
  • Target version set to 1.1.0
Actions #13

Updated by Toshi MARUYAMA about 13 years ago

  • Status changed from New to Closed
  • Resolution set to Fixed

This issue's original problem seems to io.gets() problem.
I committed r4624 in svn trunk and r4627 in 1.1.
Git browsing performance issue is #7047.
I close this issue.

Actions

Also available in: Atom PDF