Need to understand find_project_repository inside of repositories_controller (potential paid work?)

Added by Michael Sanders about 1 month ago

Hi there,
Today, I have been working on getting a Redmine plugin working to add PlasticSCM as a valid SCM type, and be able to connect to our repository. This particular Redmine plugin, detailed and linked here (or just the direct Google Code link) is quite old. However, after an installation and configuration of the plugin, I was quite surprised that runs! It appears that maybe the scm adapters have not significantly changed in the last 9 years.

When plugging in our repository information, this error is shown when clicking on 'Repository' inside of the project: The entry or revision was not found in the repository.

Our environment runs in a custom docker image, and so I can't do any interesting VS Code ssh magic to make modifications or debug easily. Instead, I just added a million logger.info() methods, and found that on line 345 of repositories_controller.rb it expects a revision to be two things. Well, I know that the latter of the two things is a default branch string, and the string that the plugin is giving it is "/main" which is the default branch in Plastic. After that, it's hitting line 353 and erroring out.

Does anyone understand what is going on in that line? Why "/main" may not be acceptable for @rev (and what @rev is?)

Replies (15)

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

I did want to add, that if anyone has experience extending scm functionality in Redmine, I'd love to pay you to assist in getting this working. We can fork the original Redmine plugin on a public repository, and provide it to PlasticSCM customers.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Liane Hampe about 1 month ago

Hi Michael,

@rev is the revision name which seems not valid. To be valid the revision name needs to match this regex '%r{\A[a-f0-9]*\z}i' and pass the validation in repository context. If it does not, an exception is raised and rescued with rendering HTML status 404.

The default branch cannot be the problem when its named '/main'. I would guess that there is something with the 'params[:rev]' which either does not match the regex or is invalid w.r.t. the @repository itself.

Probably, revision names follow another pattern with Plastic?

Best Regards,
Liane

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

Liane,
Thanks so much for the help and thoughts here so far. I saw the regex a few lines above, and diagnosed it a little. From what I could find last night, it would only accept something along the lines of %r{r64}i because the regex was matching %r{ and }i literally in both PCRE and Python.

It seems like if the default branch being named '/main' is not the problem, then as you say, it may be running through the params[:rev] statement.

PlasticSCM doesn't seem to really capture the revision of a file or folder, other than referring to the changeset (group of revisions) that file was last modified within. Per PlasticSCM's own documentation, Working in a Plastic SCM workspace, a user often makes changes to several files at the same time, and enters a single checkin command to create new revisions of them all. Plastic SCM records this group of revisions as a changeset.

Above the method that returns default_branch to find_project_repository, it specifies:

        # used by branches and default_branch
        # return PlasticBranch for branch_name (update changeset)

The return of this method is as follows:

                unless name.blank? && name != default_branch
                  plastic_branch = PlasticBranch.new(name)
                  plastic_branch.revision = changeset
                  plastic_branch.scmid = id
                  return plastic_branch
                end

I'm thinking that either find_project_repository doesn't like /main, or plastic_branch.revision needs to be returned instead of plastic_branch. I'm going to try mucking with both.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

I hardcoded the plugin's default_branch method in its own adapter to return both 'main' without a forward-slash, and the revision id (which is the changeset number,) in this case, 1034.
The former produced the same 404, the latter, a 500. However, it did get farther, so I'm following that thread.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Liane Hampe about 1 month ago

Did you look into the log file? It should give you some more information about the 500 status.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

I did yeah. I'm looking into this now:

redmine-app      | I, [2022-08-09T13:29:52.053731 #1]  INFO -- : <plastic> pscm_cmd ret: #<IO:0x00007efce4c64458>
redmine-app      | I, [2022-08-09T13:29:52.060585 #1]  INFO -- : Completed 500 Internal Server Error in 2241ms (ActiveRecord: 7.7ms)
redmine-app      | F, [2022-08-09T13:29:52.061101 #1] FATAL -- :
redmine-app      | F, [2022-08-09T13:29:52.061144 #1] FATAL -- : NoMethodError (undefined method `is_selected=' for #<IO:(closed)>):
redmine-app      | F, [2022-08-09T13:29:52.061166 #1] FATAL -- :
redmine-app      | F, [2022-08-09T13:29:52.061186 #1] FATAL -- : plugins/redmine_plastic/lib/redmine/scm/adapters/plastic_adapter.rb:102:in `branches'
redmine-app      | plugins/redmine_plastic/app/models/repository/plastic.rb:80:in `branches'
redmine-app      | plugins/redmine_plastic/app/models/repository/plastic.rb:101:in `fetch_changesets'
redmine-app      | app/controllers/repositories_controller.rb:87:in `show'
redmine-app      | lib/redmine/sudo_mode.rb:61:in `sudo_mode'

The loginfo that states <plastic> beforehand is from me. 'ret' is what is being returned from a method that runs the command-line command for PlasticSCM (cm.exe, similar to svn's svn.exe or git's git.exe.)

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

I think it didn't like something in here, but I'm not acquainted enough with Ruby hashmaps to see anything immediately incorrect:

#make brs_hash
    scm_brs_hash = {}
    scm_brs.map do |br|
      scm_brs_hash[br] = {}
      scm_brs_hash[br][:last_changeset] = br.revision
      scm_brs_hash[br][:is_selected] = true if br.is_selected
    end

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

I don't think that having default_branch return the revision (changeset) instead of the branch was valid for this plugin. It's producing cm.exe querues like "cm find branch where name='1034'" which is obviously incorrect.

So, my thinking is that my prior assumption of that method returning default_branch to find_project_repository wasn't fully correct, and that a few other things are happening before it does that.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

Well, I have successfully setup a local development environment that mirrors our production environment. Now I just need to know what I'm looking at.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

Okay, I have no idea what's going on, and what Redmine is expecting. It's failing with the valid_name? check, but I can't find out much about what valid_name? does other than being a Ruby constant name. The default branch gets returned properly, which is what this code is requesting.

Does anyone have any ideas on what it should be expecting?

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

Ah, I see this:

def valid_name?(rev)
    return true if rev.nil?
    return true if REV_PARAM_RE.match?(rev)

    @repository ? @repository.valid_name?(rev) : true
  end

Reading about .nil? it sounds like the fact that it returns false is good, because it's effectively stating that the object exists and is not empty.
/main doesn't meet the criteria for the REV_PARAM_RE regex, but it seems like practically nothing does?

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

The REV_PARAM_RE regex will work with anything like 'hd235' or 'r5823' or 'xcghsds3663234' but not with '/main' or 'main', because it needs numbers.
So why does a check on the name of a branch need to conform to regex meant for revisions?

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

After thinking about this a little bit more, I think that what Redmine might be expecting is the ID for the default branch, instead of the branch's name, so I'm going to try that instead.

RE: Need to understand find_project_repository inside of repositories_controller (potential paid work?) - Added by Michael Sanders about 1 month ago

That was it! Changesets ended up getting loaded by the plugin, but it looks like they may have an issue being saved:

Completed 404 Not Found in 1233946ms (ActiveRecord: 104.3ms)

ActiveRecord::RecordNotFound (Couldn't find all Changesets with 'id': (all, {:conditions=>["scmid IN (?)", ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "50", "51", "52", "53", "54", "55", "56", "57", "58", "59", "60", "61", "62", "63", "64", "65", "66", "67", "68", "69", "70", "71", "72", "73", "74", "75", "76", "77", "78", "79", "80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "90", "91", "92", "93", "94", "95", "96", "97", "98", "99"]]}) [WHERE `changesets`.`repository_id` = ?] (found 0 results, but was looking for 2).):

plugins/redmine_plastic/app/models/repository/plastic.rb:175:in `save_revisions'
plugins/redmine_plastic/app/models/repository/plastic.rb:136:in `fetch_changesets'
app/controllers/repositories_controller.rb:87:in `show'
lib/redmine/sudo_mode.rb:61:in `sudo_mode'

(1-15/15)