Defect #7047
Git adapter very slow when a commit modifies a lot of files
| Status: | New | Start date: | 2010-12-04 | |
|---|---|---|---|---|
| Priority: | Low | Due date: | ||
| Assignee: | - | % Done: | 0% | |
| Category: | SCM | |||
| Target version: | - | |||
| Affected version: | Resolution: |
Description
I have a Git repo with thousands of files modified in some commits. Repository index action is very slow (>20s).
At first sight, it seems it's due to Redmine::Scm::Adapters::GitAdapter#revisions uses --raw option, which is not needed in all cases, at least not needed in Repository::Git#latest_changesets. It results in thousands of useless lines parsed after the shell-out.
I'll have a look at that. I admit it's not a common use case, hence the priority.
Related issues
Associated revisions
scm: git: implement find_changeset_by_name (#7047).
SQL "like" is slow.
scm: git: add flag of disable showing last commit in repository tree (#7047).
scm: git: split adapter entries() for browsing tree performance (#7047).
scm: git: override adapter entry() for browsing tree performance (#7047).
scm: git: call entries_git() in adapter entriese() for browsing tree performance (#7047).
scm: git: call entries_git() in adapter entry() for browsing tree performance (#7047).
cat/diff/blame link calls adapter entry().
Abstract adapter entry() calls adapter entries().
Git adapter entries() is very slow.
scm: git: use report_last_commit option in unit adapter test (#7047).
scm: git: override entries() in model for browsing tree performance (#7047).
scm: git: add the model method whether reporting last commit in repository tree (#7047).
scm: git: use the model value of whether reporting last commit in repository tree (#7047).
And remove entries_git() and the flag in adapter.
scm: git: change model report_last_commit() name to extra_report_last_commit() (#7047).
History
#1 Updated by Toshi MARUYAMA over 2 years ago
Please see http://www.redmine.org/issues/4773#note-14 and try patch http://www.redmine.org/attachments/3272/git-fast-browse.patch
#2 Updated by Toshi MARUYAMA over 2 years ago
Please see following links.
#3 Updated by Toshi MARUYAMA over 2 years ago
$ time wget https://www.chiliproject.org/projects/chiliproject/repository --2011-02-08 10:33:38-- https://www.chiliproject.org/projects/chiliproject/repository Resolving www.chiliproject.org... 184.171.175.157 Connecting to www.chiliproject.org|184.171.175.157|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 30894 (30K) [text/html] Saving to: `repository' 100%[================================================================================================>] 30,894 --.-K/s in 0.1s 2011-02-08 10:33:41 (213 KB/s) - `repository' saved [30894/30894] real 0m3.378s user 0m0.063s sys 0m0.023s
$ time wget https://www.chiliproject.org/projects/chiliproject/repository/revisions/master/show/db/migrate --2011-02-08 10:33:54-- https://www.chiliproject.org/projects/chiliproject/repository/revisions/master/show/db/migrate Resolving www.chiliproject.org... 184.171.175.157 Connecting to www.chiliproject.org|184.171.175.157|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 139043 (136K) [text/html] Saving to: `migrate' 100%[================================================================================================>] 139,043 216K/s in 0.6s 2011-02-08 10:34:20 (216 KB/s) - `migrate' saved [139043/139043] real 0m26.036s user 0m0.068s sys 0m0.024s
#4 Updated by Etienne Massip over 2 years ago
- Assignee changed from Jean-Baptiste Barth to Toshi MARUYAMA
#5 Updated by Toshi MARUYAMA over 2 years ago
The issue cannot be solved to no use of wrap git command such as Yuya's Mercurial extension.
source:trunk/lib/redmine/scm/adapters/mercurial/redminehelper.py
#6 Updated by Toshi MARUYAMA about 2 years ago
- File false_flag_report_last_commit.png added
- File true_flag_report_last_commit.png added
- Assignee deleted (
Toshi MARUYAMA)
I added flag to switch showing last commit at source:trunk/lib/redmine/scm/adapters/git_adapter.rb@4992#L25.
true¶
$ time wget http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate --2011-02-24 21:17:23-- http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate Resolving localhost... 127.0.0.1 Connecting to localhost|127.0.0.1|:3000... connected. HTTP request sent, awaiting response... 200 OK Length: 139135 (136K) [text/html] Saving to: `migrate' 100%[===================================================================================>] 139,135 --.-K/s in 0.009s 2011-02-24 21:18:11 (15.5 MB/s) - `migrate' saved [139135/139135] real 0m47.799s user 0m0.004s sys 0m0.013s
false¶
$ time wget http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate --2011-02-24 21:20:43-- http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate Resolving localhost... 127.0.0.1 Connecting to localhost|127.0.0.1|:3000... connected. HTTP request sent, awaiting response... 200 OK Length: 100345 (98K) [text/html] Saving to: `migrate' 100%[===================================================================================>] 100,345 --.-K/s in 0.007s 2011-02-24 21:20:45 (14.7 MB/s) - `migrate' saved [100345/100345] real 0m2.076s user 0m0.005s sys 0m0.007s
I don't have a plan to refactor git adapter.
#7 Updated by Toshi MARUYAMA over 1 year ago
Related issue:
https://www.chiliproject.org/issues/669
#8 Updated by Jean-Philippe Lang 4 months ago
- Status changed from 7 to New
Assigned issue with no assignee back to New status.