Defect #7064

Mercurial adapter does not recognize non alphabetic nor numeric in UTF-8 copied files

Added by Toshi MARUYAMA almost 7 years ago. Updated almost 7 years ago.

Status:ClosedStart date:2010-12-07
Priority:NormalDue date:
Assignee:Toshi MARUYAMA% Done:

0%

Category:SCM
Target version:1.1.0
Resolution:Fixed Affected version:1.0.4

Description

Before applied patch

This is the original redmine image.

After applied patch

This is the image applied patch hg-urlescape.diff .

before.png (36 KB) Toshi MARUYAMA, 2010-12-07 13:47

after.png (37.2 KB) Toshi MARUYAMA, 2010-12-07 13:47

hg-urlescape.diff Magnifier (1.65 KB) Toshi MARUYAMA, 2010-12-07 13:47

settings.png (47.6 KB) Toshi MARUYAMA, 2010-12-10 09:17

hg-2010-12-10.diff Magnifier (27.7 KB) Toshi MARUYAMA, 2010-12-10 09:17

mercurial_repository.tar.gz (15.1 KB) Toshi MARUYAMA, 2010-12-10 09:17

annotate.png (25.4 KB) Toshi MARUYAMA, 2010-12-10 09:17

browse.png (114 KB) Toshi MARUYAMA, 2010-12-10 09:17

copied.png (53.6 KB) Toshi MARUYAMA, 2010-12-10 09:17

japanese-windows-repo-annotate.png (5.57 KB) Toshi MARUYAMA, 2010-12-23 15:14

japanese-windows-repo-browse.png (18.2 KB) Toshi MARUYAMA, 2010-12-23 15:14

japanese-windows-repo-setting.png (14.1 KB) Toshi MARUYAMA, 2010-12-23 15:14

r4632-utf-8.png (25.7 KB) Toshi MARUYAMA, 2011-01-06 13:16

r4633-utf-8.png (34.6 KB) Toshi MARUYAMA, 2011-01-06 13:16


Related issues

Related to Redmine - Defect #2664: Mercurial: Repository path encoding of non UTF-8 characters Closed 2009-02-04

Associated revisions

Revision 4633
Added by Toshi MARUYAMA almost 7 years ago

scm: mercurial: escape any filenames by urlescape (#2664, #7064).

In Mercurial, filename is not UTF-8 but byte-string.
So it should be escaped to UTF-8-safe string before parsing as XML document.

NOTE: Mercurial's {|obfuscate} filter cannot be used here, because it treats
filename as encoded by HGENCODING.

Contributed by Yuya Nishihara.

Revision 4634
Added by Toshi MARUYAMA almost 7 years ago

scm: mercurial: update test repository for following issues and fix tests.

Revision 4636
Added by Toshi MARUYAMA almost 7 years ago

scm: mercurial: add unit test for copied file (#7064).

History

#1 Updated by Yuya Nishihara almost 7 years ago

Hi, I like your patch in principle, and it should be {name|escape} for consistency.

But one thing, filenames in Mercurial are bytes, thus {name|escape} can contain invalid UTF-8 sequence for non-ASCII & non-UTF-8 filenames. It may confuse REXML parser.
The following entries have the same issue: file, file_mod, file_add, file_del and source .

See also #2664

#2 Updated by Toshi MARUYAMA almost 7 years ago

I merged issue-2664-0.9-stable-2010-04-11.patch of #2664 and latest-changesets-improvements-4.diff of #6159 excluding Subversion.
And I updated Mercurial test repository and add unit tests and functional tests.
And I attach images.

This patch fixes #2664, #5408, #3449 and #3567.

#3 Updated by Toshi MARUYAMA almost 7 years ago

  • % Done changed from 50 to 100

#4 Updated by Toshi MARUYAMA almost 7 years ago

I confirmed unit and functional tests pass for latin-1 encoding test repository on my Japanese(CP932) Windows Vista.

I post some images.

$ hg --version
Mercurial Distributed SCM (version 1.7.2)
$ RAILS_ENV=test ruby script/about
r:/Ruby186/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/rails/gem_dependency.rb:119:Warning: Gem::Dependency#version_requirements is deprecated and will be removed on  or after August 2010.  Use #requirement

About your application's environment
Ruby version              1.8.6 (i386-mingw32)
RubyGems version          1.3.7
Rack version              1.0
Rails version             2.3.5
Active Record version     2.3.5
Active Resource version   2.3.5
Action Mailer version     2.3.5
Active Support version    2.3.5
Application root          R:/work/hg-workdir/redmine-trunk-my-work
Environment               test
Database adapter          sqlite3
Database schema version   20101115000000
Loaded suite script/about
Started

Finished in 0.0 seconds.

0 tests, 0 assertions, 0 failures, 0 errors

#5 Updated by Toshi MARUYAMA almost 7 years ago

  • Assignee set to Toshi MARUYAMA
  • % Done changed from 100 to 0

#6 Updated by Toshi MARUYAMA almost 7 years ago

  • Subject changed from Mercurial adapter does not recognize non ASCII copied files to Mercurial adapter does not recognize non alphabetic nor numeric copied files

#7 Updated by Toshi MARUYAMA almost 7 years ago

  • Subject changed from Mercurial adapter does not recognize non alphabetic nor numeric copied files to Mercurial adapter does not recognize non alphabetic nor numeric in UTF-8 copied files

#8 Updated by Toshi MARUYAMA almost 7 years ago

I confirmed to fix copying UTF-8 named file.
I attach images.
Non UTF-8 named file is #2664 issue.
So, I close this issue.

Also available in: Atom PDF