Feature #2799

Support for Bazaar's shared reposetories (created with init-repo)

Added by Thorbjørn Hermansen over 5 years ago. Updated almost 2 years ago.

Status:NewStart date:2009-02-21
Priority:NormalDue date:
Assignee:-% Done:

30%

Category:SCM
Target version:-
Resolution:

Description

Hi,

I don't know if this is a bug or a feature request, but I'm having a shared repository created with bzr init-repo --no-trees project_container. This repo has a branch called trunk, and directories called branches and tags. The branches and tags directories contains new bzr branches.

What I want to do is to have my repository in Redmine point to the project_container directory so that I browse all branches from Redmine. If I do that and run ruby script/runner "Repository.fetch_changesets" -e production I get these messages back:
bzr: ERROR: Not a branch: "[...]/project_container/.bzr/branch/".
bzr: ERROR: Not a branch: "[...]/project_container/.bzr/branch/".

Is it possible to support this?

To point the repository path in Redmine to project_container/trunk do indeed work, but then of course I can't browse branches which is in tags/branches.

20100907-ryxeo-bazaar_sharedrepo.tar.bz2 - Redmine 0.9.4.stable.3805 (3.47 KB) Eric Seigne, 2010-09-07 21:00

20100907b-ryxeo-bazaar_sharedrepo_for_redmine1.0.1.tar.bz2 - for redmine 1.0.1 stable (3.5 KB) Eric Seigne, 2010-09-07 21:56

bzr-discover-dir.py Magnifier - Discover whether a directory is bzr managed or not, and if so, how. (2.12 KB) Michael Trausch, 2011-01-14 22:14

bzr-discover-dir.py Magnifier - Second revision of script. (2.12 KB) Michael Trausch, 2011-01-16 02:25

redmine-1.1.3_patch-bzr-shared-repositories.bz2 (3.38 KB) Eric Seigne, 2011-05-01 16:18

redmine-1.2.bzr_sharedrepo.patch.bz2 (3.17 KB) Eric Seigne, 2011-11-20 22:12

redmine-1.4_patch-bzr-shared-repositories.bz2 (3.78 KB) Eric Seigne, 2012-05-08 16:35


Related issues

Related to Feature #4455: Mercurial overhaul Closed 2009-12-21
Related to Patch #4262: Bazaar: correct renames handling, other improvements New 2009-11-22
Related to Feature #4741: Improve Bazaar repositories error message in case to use ... New 2009-02-21
Related to Patch #2510: [SCM, Bazaar] Prefer revision's "author" field over "comm... New 2009-01-14
Related to Defect #2664: Mercurial: Repository path encoding of non UTF-8 characters Closed 2009-02-04
Related to Feature #3559: Allow viewing of sub-revision in Bazaar VCS revision history New 2009-06-29
Related to Defect #7524: Root repository directory not displayed with Bazaar on Ub... New 2011-02-02
Related to Defect #8030: Bazaar integration doesn't notice new commits to repository New 2011-03-31
Related to Defect #5357: Git: SCM revisions ordered by date/time (should be revers... New 2010-04-20
Related to Defect #7648: Git: Repository Statistics Incorrect New 2011-02-18
Related to Defect #7146: Git adapter lost commits before 7 days from database late... Closed 2010-12-21
Related to Feature #4273: SCM command availability automatic check in administratio... Closed 2009-11-23
Related to Defect #7984: Confusing revisions links on repository root New 2011-03-24
Related to Defect #12973: Bazaar: Incorrect issue-to-revision relation after "uncom... New

Associated revisions

Revision 5890
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: add adapter method to get .bzr/branch/branch.conf path from specified path (#2799, #4741, #8030).

Revision 5891
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: add unit adapter test to get .bzr/branch/branch.conf path from specified path (#2799, #4741, #8030).

Revision 5892
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: add adapter method to get "append_revisions_only" value from .bzr/branch/branch.conf (#2799, #4741, #8030).

Revision 5893
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: add unit adapter test to get "append_revisions_only" value from test repository .bzr/branch/branch.conf (#2799, #4741, #8030).

Revision 5894
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: use explicit return value in adapter branch_conf_path() (#2799, #4741, #8030).

Revision 5895
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: check not nil in adapter append_revisions_only() (#2799, #4741, #8030).

Revision 5899
Added by Toshi MARUYAMA over 3 years ago

scm: add exception of fetching revisions error in repository model (#5357, #2799, #4741, #8030).

Revision 5932
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: update test repository (#2799, #4741, #8030).

  • Shared repository with trees (format: 2a)
  • clone original branch to trunk
  • set append_revisions_only = true at trunk/.bzr/branch/branch.conf

Revision 5935
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: add unit adapter test that shared repository "append_revisions_only" is false (#2799, #4741, #8030).

adapter append_revisions_only() checks whether .bzr/branch/branch.conf exists.

Revision 5936
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: rename unit adapter test "test_append_revisions_only" to "test_append_revisions_only_true" (#2799, #4741, #8030).

Revision 5937
Added by Toshi MARUYAMA over 3 years ago

scm: bazaar: add unit adapter test that append_revisions_only is false in test shared repository branch (#2799, #4741, #8030).

History

#1 Updated by Nick Trew over 5 years ago

+1 for this.

Could Redmine be modified to list all directories within the "root directory"?

If one of these directories doesn't contain the .bzr directory, Redmine will display the "The entry or revision was not found in the repository." error message (which currently shows when attempting to add a shared repository).

#2 Updated by Andrew Burcin over 5 years ago

This would be very useful, especially for development with Bazaar using feature branching workflow.

I am willing to contribute to a bounty to make redmine browsing of Bazaar shared repositories work.

#3 Updated by Nikolai Bochev over 5 years ago

+1 for this also. Bazaar is gaining speed, just look at launchpad, and as of now there isn't an open source issue tracker supporting it's full feature set of workflows . I am also willing to contribute to a bounty not only for making shared repository browsing work within redmine, but a tighter integration with Bazaar in general.

#4 Updated by Attila Beregszaszi about 5 years ago

+1 on this from me as well. Any updates?

#5 Updated by Ben Oakes about 5 years ago

+1

#6 Updated by David Muir about 5 years ago

bzrtools has the branches command that lets you list branches in a location (`bzr branches`).
It will list branches regardless of there being a shared repository or not. The difficulty is more that depending on the number of folders, it could take quite a bit of time to search.
It probably wouldn't be a good idea to require the bzrtools plugin though, even though most who use bazaar probably have it installed. Will have a look at the source code for the command to see what it actually does. My guess is that it looks for .bzr folders, then runs bzr info to check and see if it's a branch or shared repository.

#7 Updated by Wei Li almost 5 years ago

+1

#8 Updated by Antti Kaijanmäki almost 5 years ago

I just found out about Redmine and it seems very interesting. I'm deploying it for testing for our little project and we need support for shared Bazaar repositories. I could work on this myself but as always I'm short on time and I've never even looked at ruby before.

Anyway, if someone begins developing this feature you can count me in for testing :)

+1

#9 Updated by sebastián scarano almost 5 years ago

yeap, count me too on this...

#10 Updated by sebastián scarano almost 5 years ago

somebody seems to have found some kind of workaround...

http://www.vxnick.com/2009/05/redmine-bazaar-repositories/

#12 Updated by Joe Julian over 4 years ago

I don't see how the last 2 comments apply to this.

I too would like to be able to have the proper development tree structure available.

#13 Updated by Joe Julian over 4 years ago

There are 2 ways to identify the shared repository:
  • bzr info will list 'shared repository: .'
  • .bzr/repository exists.

If the path points to a shared repo, some file tree should be built.

Branches can be identified by a similar combination of techniques:
  • bzr info will list the root repository as 'shared repository'
  • .bzr/branch exists.
Sub branches:
  • bzr info will list the root repository as 'shared repository' and the parent branch as 'parent branch'

As an example, let's examine the following repo:

$ mkdir /var/spool/bzr
$ bzr init-repo /var/spool/bzr
$ cd /var/spool/bzr
$ bzr info
Shared repository with trees (format: 2a)
Location:
  shared repository: .
$ ls -d .bzr/repository
.bzr/repository
$ bzr init fubar
$ cd fubar
$ bzr info
Repository tree (format: 2a)
Location:
  shared repository: /var/spool/bzr
  repository branch: .
$ ls -d .bzr/branch
.bzr/branch
$ bzr branch . foo
$ cd foo
$ bzr info
Repository tree (format: 2a)
Location:
  shared repository: /var/spool/bzr
  repository branch: .

Related branches:
  parent branch: /var/spool/bzr/fubar
$ ls -d .bzr/branch
.bzr/branch
$ cd /var/spool/bzr
$ bzr branch fubar bar
$ cd bar
$ bzr info
Repository tree (format: 2a)
Location:
  shared repository: /var/spool/bzr
  repository branch: .

Related branches:
  parent branch: /var/spool/bzr/fubar
$ ls -d .bzr/branch
.bzr/branch

In summary, if the path is a repo, scan the directory tree for directories containing .bzr/branch and run bzr info on those directories. Use that output to find out which branch is the parent and use that information to build the element tree.

I don't know how to do this in ruby, but hopefully someone can take this analysis and put something together.

#14 Updated by justin kelly about 4 years ago

+1

#15 Updated by Eric Seigne about 4 years ago

  • % Done changed from 0 to 20

hi all,
i've got a solution ! here is a bzr shared repository : https://redmine.ryxeo.com/projects/couveuse-code-source/repository

i could send a patch this night for testing users only ... someone interested ?

Éric

#16 Updated by Eric Seigne about 4 years ago

Files from our "Redmine 0.9.4.stable.3805" installation, i'll make an upgrade to lastest stable this night :)

#17 Updated by Eric Seigne about 4 years ago

Here is a tarball for redmine version 1.0.1.

#18 Updated by Nikolai Bochev almost 4 years ago

Any chance of this getting implemented ?

#20 Updated by Michael Trausch almost 4 years ago

It would seem that perhaps a better way to do this would be to have a small helper module, written in Python, that redmine can use to learn about the repository. I have written a simple script that could be used for just that. It is attached to this comment. If necessary, I will transfer copyright so that it can be maintained along with Redmine. I do apologize, but I do not possess the ability to add Ruby code to Redmine to use the output of this small program (but I suspect that someone could do that far easier than probing for all of the corner cases in the Ruby code).

The script uses the return status to indicate whether or not it found a bzr managed directory; return status is 1 (and stdout is empty) if the directory specified on the command line is not managed by bzr, return status is 0 (and stdout contains data for Redmine) if the directory specified on the command line is managed by bzr and is either a branch or a shared repository. Here is three sample runs from my system; the first on a directory that is neither a bzr branch nor shared repository, the second for a bzr shared repository (holding an svn-import of the Redmine code), and the third for a branch within that repository.

If this script's output should be different to make things easier, feel free to update it or let me know and I can update it. For that matter, if the script could be more useful by outputting more data, just specify what and how and I'll be happy to modify it to do that.

Hope this helps!

 $ ./bzr-discover-dir.py /etc
 $ echo $?
 1

 $ ./bzr-discover-dir.py /home/mbt/Projects/redmine/
 path: /home/mbt/Projects/redmine/
 type: shared-repository
 branches:
   /home/mbt/Projects/redmine/branches/0.6-stable/
   /home/mbt/Projects/redmine/branches/0.7-stable/
   /home/mbt/Projects/redmine/branches/0.8-stable/
   /home/mbt/Projects/redmine/branches/0.9-stable/
   /home/mbt/Projects/redmine/branches/1.0-stable/
   /home/mbt/Projects/redmine/branches/1.1-stable/
   /home/mbt/Projects/redmine/trunk/
 $ echo $?
 0

 $ ./bzr-discover-dir.py /home/mbt/Projects/redmine/trunk
 path: /home/mbt/Projects/redmine/trunk
 type: branch
 $ echo $?
 0

#21 Updated by Michael Trausch almost 4 years ago

Turns out that it would be useful to not filter out the file:// prefix, so I have modified the script that I've written to stop doing that. This way, it can then be used also for branches (and repositories) over the network (because bzrlib can handle that pretty transparently, as well). So now, instead of /path/to/file for local files, it will say file:///path/to/file (and it won't mangle network schemas like bzr+ssh:// or similar).

This is not a patch because:

  1. I do not know where it would belong in the source tree. It's essentially another program that the module would need to shellout to, but would be distributed with Redmine, as part of it.
  2. Even if I did, I do not (yet) know how to integrate the ability to use it into Redmine. I'm working on learning some Ruby and trying to figure out what's here, but I am not making any commitment to being able to produce a functional, test-covered implementation. If someone comes and does the tie-in work and such, my feelings won't be at all hurt.

However, assuming that I figure out how to accomplish this, I will post my work. Should anyone care to provide advice, I am hanging out on the IRC channel and also subscribed to this issue.

#22 Updated by Michael Trausch almost 4 years ago

Proceeding in my attempts to create a SCM module that does this cleanly, I am running into several issues. I am making my way through them (terribly slowly), but here's so far what I am looking for and lacking:

  • What functions are expected to be in the SCM adapter class that inherits from Redmine::Scm::Adapters::AbstractAdapter? It would seem that at least entries, revisions, diff, cat, annotate are expected, but the various classes seem to disagree.
  • What functions expected to appear in the SCM model class that inherits from the Repository model in app/models/repository.rb? Again, the different SCM models available do not seem to agree with each other so it's difficult for me to figure this out.
  • It seems that Redmine expects a revision to be a property that persists throughout an entire repository. Am I right on that? If so, that is an impediment to implementation because revisions in Bazaar are per-branch properties (that is, given two branches A and B in shared repository R_A, revision 3 in branch A may differ from revision 3 in branch B and will thus have different underlying revision identifiers). In order to make this a bit better, there are full revision identifiers (which are somewhat similar to git's notion of revision identifiers, but not precisely), but I don't know how that would fit into the model that Redmine appears to expect. (It might help to know that Bazaar's shared-repository concept exists solely to save on storage costs: two identical revisions in two different branches will point to the same on-disk data. It is technically possible to have completely unrelated branches in a shared repository, but that's not the case in practice, because there is little point in doing so.)
  • I am unclear on exactly how to build the information that the front-end requires. Reading through source code is not providing me the answers that I am seeking with any reasonable degree of efficiency, so some pointers on where to focus my attention would be greatly appreciated. Any information on assumptions or expectations of Redmine's repository management functionality would be helpful. Again, I am attempting to learn this from the sources, but I am finding it really, really difficult to read and truly comprehend the source code (mostly likely due to my unfamiliarity with Ruby, which I am attempting to learn as I go).

My biggest reason for working on this is to try to provide something that will be acceptable for integration into the mainline Redmine source and works in all situations that the bzr tool can handle itself, including branches and repositories that are not locally hosted on the system running Redmine. I don't know that I will be able to achieve that, though, so if someone else is working on this please don't stop doing so just because I'm attempting it.

#23 Updated by Michael Trausch almost 4 years ago

I am stuck on this. From what I can tell, this cannot be done efficiently, if at all, because Bazaar uses a different model from what Redmine expects.

Bazaar has a much simpler model than other VCSes for shared repositories: they only hold the data that is shared between the branches. However, Redmine seems to make the assumption that it can do things such as specify a branch name in the revision field passed to the VCS, and the VCS will sort it all out. Same goes for tags.

Here are the issues that I have run into so far:

  • It is not possible to efficiently get a list of all of the branches stored within a shared repository in Bazaar. The helper script that I have attached to this issue shows how it can best be done. However, this is not something that can be fixed in Redmine; the Bazaar VCS needs to cache the list of branches. Since branches cannot be added to the shared repository without Bazaar knowing about them (as far as I am aware), this should be a relatively easy bug to fix. Bazaar just needs to store the data in a catalog somewhere instead of deriving it every time it is needed.
  • Since Redmine passes both branch names and tags as revision identifiers, ambiguities are possible. People identify a revision by speaking of a (branch, revno) pair, tags are the same way, (branch, tag). Both revnos and tags can exist in multiple branches and point to different things. IOW, it only makes sense to use a revno and a tag in concert with a branch. What I do not see is how to do this and solve for all possible use cases. For that to be possible, Redmine would have to treat the branch as its own attribute of a repository, and make revisions (and tags) both attributes of branches. That would, though, complicate things because as can be seen in the Subversion SCM plugin, Redmine does not attempt to use branches there since a Subversion repository layout does not adhere to a strict layout for branches or tags. In short, one-size-fits-all causes problems for one or the other.
  • Because Redmine handles things the way it does, I do not see a means by which to identify a revision (passed in by the user) within the context of a branch (which they would be looking at). For that to work at all, Redmine would have to present to the SCM code a (branch, revno) pair (and the same goes for tags).

I am looking for a method by which this can be fixed, but it looks to me (admittedly, I am not an expert either in Ruby or the Redmine source code, so I could be way wrong here) that Redmine's concept of an SCM system has to be modified somewhat in order to make sense of things. This requires Redmine to store more data in its own database so that it is able to unambiguously pull up revisions from a Bazaar repository.

If Bazaar supported a repository-wide view of things, that would make things work just fine. However, all operations are performed on branches, not repositories, which is the biggest impediment. Until something changes, the current Bazaar plugin (which handles only a single branch) is the only way that things can work.

I would be welcome to suggestions or help. I'm in the IRC chat room, and am normally at my computer on and off throughout the day. This feature is critical to our adoption of the software, but I cannot proceed further with its implementation without at least some help and guidance.

#24 Updated by Eric Seigne over 3 years ago

Hello all,
here is a patch for redmine 1.1.3 with full bzr shared repositories support.

This patch is not clean, i'm sure but it works on our https://redmine.ryxeo.com so if someone want take it and make it better it's free.

On a new fresh redmine install
- tar xfvz redmine-1.1.3.tar.gz
- cd redmine-1.1.3
- bzip2 /tmp/redmine-1.1.3_patch-bzr-shared-repositories.bz2 (this patch)
- patch -p1 < /tmp/redmine-1.1.3_patch-bzr-shared-repositories

Éric

#25 Updated by André Bachmann over 3 years ago

Hi Eric,

is there a patch for Redmine 1.2.0?

#26 Updated by TridenT Job almost 3 years ago

Hi Eric, is there a patch for the latest 1.2.1 or 1.2.2 ?

#27 Updated by Eric Seigne almost 3 years ago

Hi all,
here is a patch for redmine 1.2.2, i've just updated my redmine :)

#28 Updated by TridenT Job almost 3 years ago

Eric Seigne wrote:

Hi all,
here is a patch for redmine 1.2.2, i've just updated my redmine :)

Thx for your very quick reply. I will test this today.

#29 Updated by Lluís Vilanova almost 3 years ago

Michael Trausch wrote:

I am stuck on this. From what I can tell, this cannot be done efficiently, if at all, because Bazaar uses a different model from what Redmine expects.

Why don't you just use the command bzr branches /path/to/shared/repo?

#30 Updated by Eric Seigne over 2 years ago

Here is a patch for redmine 1.4.1.

Éric

#31 Updated by Ugo Cupcic almost 2 years ago

Does anyone by anychance have a patched version of the bitnami vm proposed in the download section? (or the turnkey one)

#32 Updated by Lluís Vilanova almost 2 years ago

For the sake of future reference, and while it is not actual support for bazaar's features, latest Redmine versions support multiple repositories per project, which can alleviate the problem.

#33 Updated by TridenT Job almost 2 years ago

Lluís Vilanova wrote:

For the sake of future reference, and while it is not actual support for bazaar's features, latest Redmine versions support multiple repositories per project, which can alleviate the problem.

SO you suggest to add repository path to each branch ?

#34 Updated by Lluís Vilanova almost 2 years ago

TridenT Job wrote:

SO you suggest to add repository path to each branch ?

Sorry, I'm not quite sure I understand what you mean. With current Redmine, what you can do is have a separate repository for each "branch". So accessing code through "http://code.server/bzr/project" is like accessing the main branch, while you can have the branch "feature1" stored in another repository of the same project ("http://code.server/bzr/project.feature1").

It's not the same, but I'm using it right now as a work-around.

Also available in: Atom PDF