Project

General

Profile

Actions

Defect #2664

closed

Mercurial: Repository path encoding of non UTF-8 characters

Added by Jérémie Delaitre about 15 years ago. Updated about 13 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Toshi MARUYAMA
Category:
SCM
Target version:
Start date:
2009-02-04
Due date:
% Done:

100%

Estimated time:
Resolution:
Fixed
Affected version:

Description

Environment

  • Server OS: Debian Lenny
  • Redmine: svn rev 2361 (same problem with 0.8.0)
  • Ruby: 1.8.6
  • RubyGems: 1.3.1
  • Rails: 2.1.2
  • PostgreSQL: 8.3.5
  • Mercurial: 1.0.1
  • System locale: en_us.UTF8
  • Database encoding: utf8
  • Database locale: fr_FR.UTF8 (same problem with en_us.UTF8)

Error

Running: ruby script/runner "Repository.fetch_changesets" -e production gives the following errors:

/home/redmine/redmine-0.8.0/vendor/rails/railties/lib/commands/runner.rb:47: /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:147:in `log':
 RuntimeError: ERROR        C22021  Minvalid byte sequence for encoding "UTF8": 0xe97365
    HThis error can also happen if the byte sequence does not match the encoding
 expected by the server, which is controlled by "client_encoding".        Fwchar.c        L1545
   Rreport_invalid_encoding: INSERT INTO "changes" ("changeset_id", "action", "revision", "branch", "from_path",
 "path", "from_revision") VALUES(781, E'A', NULL, NULL, NULL,
 E'/Quantity/doc/Présentation du projet.pdf', NULL) RETURNING "id" (ActiveRecord::StatementInvalid)
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:484:in `execute'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:929:in `select_raw'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:916:in `select'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all_without_query_cache'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:61:in `select_all'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:13:in `select_one'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:19:in `select_value'
        from /home/redmine/redmine-0.8.0/vendor/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:433:in `insert'
         ... 31 levels...
        from /home/redmine/redmine-0.8.0/vendor/rails/railties/lib/commands/runner.rb:47
        from /home/redmine/apps/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
        from /home/redmine/apps/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
        from script/runner:3

The error seems quite similar to #834, #917, and #1663 but the error is not appening on the same table. Here, the problem comes from the "changes" table while the already reported (and corrected) issues refer a problem on the "changesets" table.

The problem seems to comes from file path which are not converted to UTF-8 (as we can notice, there is a 'é' character in the file path).

I have tried different encoding in the repository tab settings without success.


Files

issue-2664-0.9-stable-2010-04-11.patch (10.7 KB) issue-2664-0.9-stable-2010-04-11.patch Yuya Nishihara, 2010-04-11 04:39
git-bzr.patch (1.37 KB) git-bzr.patch Toshi MARUYAMA, 2010-06-07 15:03
20110207-db.diff (649 Bytes) 20110207-db.diff Toshi MARUYAMA, 2011-02-07 10:39
20110207-git-cvs-fs.diff (2.09 KB) 20110207-git-cvs-fs.diff Toshi MARUYAMA, 2011-02-07 10:39
20110207-impl.diff (5.71 KB) 20110207-impl.diff Toshi MARUYAMA, 2011-02-07 10:39
hg-ruby-1.9.diff (3.06 KB) hg-ruby-1.9.diff Toshi MARUYAMA, 2011-02-07 12:15
for-trunk-r4893-20110220.patch (8.2 KB) for-trunk-r4893-20110220.patch Toshi MARUYAMA, 2011-02-20 15:59
ruby-1.9.2-japanese-windows.png (50.5 KB) ruby-1.9.2-japanese-windows.png Toshi MARUYAMA, 2011-03-04 13:07

Related issues

Related to Redmine - Defect #5251: Git: Repository path encoding of non UTF-8 charactersClosedToshi MARUYAMA2010-04-07

Actions
Related to Redmine - Defect #2274: Filesystem Repository path encoding of non UTF-8 charactersClosedToshi MARUYAMA2008-12-04

Actions
Related to Redmine - Defect #3462: CVS: Repository path encoding of non UTF-8 characters ClosedToshi MARUYAMA2009-06-08

Actions
Related to Redmine - Defect #7064: Mercurial adapter does not recognize non alphabetic nor numeric in UTF-8 copied filesClosedToshi MARUYAMA2010-12-07

Actions
Related to Redmine - Feature #2799: Support for Bazaar's shared reposetories (created with init-repo)New2009-02-21

Actions
Related to Redmine - Defect #6090: Most binary files become corrupted when downloading from CVS repository browser when Redmine is running on a Windows serverClosedToshi MARUYAMA2010-08-09

Actions
Related to Redmine - Feature #4050: Ruby 1.9 supportClosed2009-10-18

Actions
Related to Redmine - Feature #3396: Git: use --encoding=UTF-8 in "git log"ClosedToshi MARUYAMA2009-05-20

Actions
Related to Redmine - Defect #4773: Redmine+Git+PostgresSQL 8.4 fails with linux kernel tree (encoding)ClosedJean-Philippe Lang2010-02-09

Actions
Has duplicate Redmine - Defect #5408: Mercurial and chinese codeClosed2010-04-30

Actions
Has duplicate Redmine - Defect #3677: fetching changesets from Mercurial repository failsClosedToshi MARUYAMA2009-07-27

Actions
Has duplicate Redmine - Defect #8726: Redmine+Mercurial+PostgreSQL 9 falls with cyrrilic filenames in repositoryClosed2011-06-30

Actions
Actions

Also available in: Atom PDF