Defect #3462

CVS: Repository path encoding of non UTF-8 characters

Added by Rodrigo Rosenfeld Rosas over 8 years ago. Updated over 6 years ago.

Status:ClosedStart date:2009-06-08
Priority:LowDue date:
Assignee:Toshi MARUYAMA% Done:

0%

Category:SCM
Target version:1.2.0
Resolution:Fixed Affected version:

Description

We have some "ISO-8859-1" encoded filenames in our CVS repository. Repository.find(20).fetch_updates yields this error:

ActiveRecord::StatementInvalid (PGError: ERROR: invalid byte sequence for encoding "UTF8": 0xf3706961
HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".
: SELECT "changes".* FROM "changes" INNER JOIN changesets ON changes.changeset_id = changesets.id WHERE ("changes"."revision" = E'1.2' AND "changes"."path" = E'/xxxxx/AlterarSenha (c<F3>pia).png') AND (("changesets".repository_id = 20)) LIMIT 1):
...

The attached patch solved the problem for us.

Above is the relevant stack:
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:188:in `log'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:503:in `execute'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1000:in `select_raw'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:987:in `select'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all_without_query_cache'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/query_cache.rb:60:in `select_all'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/query_cache.rb:81:in `cache_sql'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/query_cache.rb:60:in `select_all'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:635:in `find_by_sql'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:1490:in `find_every'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:1452:in `find_initial'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:587:in `find'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:1812:in `find_by_path_and_revision'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/associations/association_collection.rb:370:in `send'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/associations/association_collection.rb:370:in `method_missing'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:2003:in `with_scope'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/associations/association_proxy.rb:202:in `send'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/associations/association_proxy.rb:202:in `with_scope'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/associations/association_collection.rb:366:in `method_missing'
/app/models/repository/cvs.rb:107:in `fetch_changesets'
/lib/redmine/scm/adapters/cvs_adapter.rb:192:in `revisions'
/lib/redmine/scm/adapters/cvs_adapter.rb:126:in `each_line'
/lib/redmine/scm/adapters/cvs_adapter.rb:126:in `revisions'
/lib/redmine/scm/adapters/abstract_adapter.rb:181:in `call'
/lib/redmine/scm/adapters/abstract_adapter.rb:181:in `shellout'
/lib/redmine/scm/adapters/abstract_adapter.rb:179:in `popen'
/lib/redmine/scm/adapters/abstract_adapter.rb:179:in `shellout'
/lib/redmine/scm/adapters/abstract_adapter.rb:165:in `shellout'
/lib/redmine/scm/adapters/cvs_adapter.rb:114:in `revisions'
/app/models/repository/cvs.rb:104:in `fetch_changesets'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:550:in `transaction'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/transactions.rb:129:in `transaction'
/opt/redmine/.gem/ruby/1.8/gems/activerecord-2.2.2/lib/active_record/transactions.rb:138:in `transaction'
/app/models/repository/cvs.rb:102:in `fetch_changesets'

cvs-path-encoding.patch Magnifier (471 Bytes) Rodrigo Rosenfeld Rosas, 2009-06-08 16:24


Related issues

Related to Redmine - Defect #2664: Mercurial: Repository path encoding of non UTF-8 characters Closed 2009-02-04

Associated revisions

Revision 5461
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: set path encoding instance value (#3462).

Revision 5462
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: convert encoding of command line arguments (#3462).

Revision 5463
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: convert encoding paths in revisions() of adapter (#3462).

Revision 5464
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: convert encoding paths in entries() of adapter (#3462).

Revision 5465
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: add path encoding select box at setting (#3462).

Revision 5520
Added by Toshi MARUYAMA over 6 years ago

scm: use i18n string at path encoding setting (#2274, #2664, #3462, #5251).

Revision 5521
Added by Toshi MARUYAMA over 6 years ago

scm: add Japanese translation "field_scm_path_encoding" (#2274, #2664, #3462, #5251).

Revision 5522
Added by Toshi MARUYAMA over 6 years ago

scm: update locales for path encoding setting (#2274, #2664, #3462, #5251).

Revision 5523
Added by Toshi MARUYAMA over 6 years ago

scm: use i18n string at path encoding setting note (#2274, #2664, #3462, #5251).

Revision 5524
Added by Toshi MARUYAMA over 6 years ago

scm: add Japanese string at path encoding setting note (#2274, #2664, #3462, #5251).

Revision 5525
Added by Toshi MARUYAMA over 6 years ago

scm: update locales for path encoding setting note (#2274, #2664, #3462, #5251).

Revision 5630
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: fix loss non ASCII paths if path_encoding is '' (#3462).

Revision 5863
Added by Toshi MARUYAMA over 6 years ago

scm: add "path_encoding" method in abstract adapter (#2274, #3462, #2664, #5251).

Revision 5866
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: override "path_encoding" method in adapter (#3462).

Revision 5867
Added by Toshi MARUYAMA over 6 years ago

scm: cvs: add unit adapter test of default path_encoding is UTF-8 (#3462).

History

#1 Updated by Toshi MARUYAMA almost 7 years ago

  • Assignee set to Toshi MARUYAMA
  • Priority changed from High to Normal

#2 Updated by Toshi MARUYAMA almost 7 years ago

  • Assignee deleted (Toshi MARUYAMA)
  • Priority changed from Normal to Low

Ruby 1.9 compatibility is very serious.

#3 Updated by Toshi MARUYAMA over 6 years ago

  • Assignee set to Toshi MARUYAMA

#4 Updated by Toshi MARUYAMA over 6 years ago

  • Subject changed from CVS path encoding problems to CVS: Repository path encoding of non UTF-8 characters

#5 Updated by Toshi MARUYAMA over 6 years ago

  • Target version set to 1.2.0

#6 Updated by Toshi MARUYAMA over 6 years ago

  • Status changed from New to Closed
  • Resolution set to Fixed

I finished implementing in r5465.
I can not prepare test repository for same reason with filesystem repository #2274.
If tar ball has Latin-1 path encoding files, I can't extract it on my Japanese Windows.

There are same limitations with git #5251.

Also available in: Atom PDF