Defect #7245

Wiki fails to find pages with cyrillic characters using postgresql

Added by Artem Malahov almost 7 years ago. Updated over 6 years ago.

Status:ClosedStart date:2011-01-07
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:I18n
Target version:1.1.1
Resolution:Fixed Affected version:1.1.0

Description

hello

I have updated my redmine from version 1.0.3 to version 1.0.5 (revision 4651) as write on Guide, but my wiki pages was deleted...

I looked postgres database, and found all my pages in it.

They also appear on the page table of contents.

But they dont work :(.
I traced that not displays (and does't create) only pages containing:
uppercase symbols
special chars (",' etc)

Now i rename all my pages to new right, but what to do in the future? or how can I solve the problem?

I write to forum.

My versions:

first ~ # ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [x86_64-linux]
first ~ # rails -v
Rails 2.3.5
first ~ # postgres --version
postgres (PostgreSQL) 8.4.4
first redmine.iso.vc # svn update
At revision 4651

rack 1.0.1
redmine 1.0.5 r4651

1.jpg - page /projects/iso/wiki (60.9 KB) Artem Malahov, 2011-01-07 12:15

2.jpg - pg_admin list of table wiki_pages (39.7 KB) Artem Malahov, 2011-01-07 12:15

3.jpg - page /projects/iso/wiki/index (37.2 KB) Artem Malahov, 2011-01-07 12:15

4.jpg - page /projects/iso/wiki/Этика_менеджмента (18.9 KB) Artem Malahov, 2011-01-07 12:15

Associated revisions

Revision 4680
Added by Jean-Philippe Lang almost 7 years ago

Fixed: wiki pages with cyrillic characters not found with postgresql (#7245).

History

#1 Updated by Jean-Philippe Lang almost 7 years ago

There is no database migration and no change in wiki controller/model in 1.0-stable, so I don't have a clue.

#2 Updated by Artem Malahov almost 7 years ago

Jean-Philippe Lang wrote:

There is no database migration and no change in wiki controller/model in 1.0-stable, so I don't have a clue.

May be problem in new version of i18n?

gem install i18n -v=0.4.2

#3 Updated by Jean-Philippe Lang almost 7 years ago

I don't think so.

#4 Updated by Artem Malahov almost 7 years ago

Jean-Philippe Lang wrote:

There is no database migration and no change in wiki controller/model in 1.0-stable, so I don't have a clue.

hello!

today i retest my redmine. I copy my backup and try to update it. That i see:

first /vhosts # cp -R redmine.iso.vc.bak redmine.iso.vc.bak.test
first /vhosts # cd redmine.iso.vc.bak.test
first redmine.iso.vc.bak.test # svn update > upd.txt
first redmine.iso.vc.bak.test # cat upd.txt | grep wiki
U    test/unit/wiki_test.rb
U    test/unit/lib/redmine/wiki_formatting/textile_formatter_test.rb
U    test/unit/lib/redmine/wiki_formatting/macros_test.rb
U    test/unit/lib/redmine/wiki_formatting.rb
U    test/unit/wiki_redirect_test.rb
U    test/unit/wiki_content_test.rb
U    test/unit/wiki_page_test.rb
U    test/functional/wikis_controller_test.rb
U    test/functional/wiki_controller_test.rb
U    app/models/wiki.rb
U    app/controllers/wiki_controller.rb
U    app/views/wiki/history.rhtml
U    app/views/wiki/diff.rhtml

I don't spec in svn/ruby, but i see that wiki is updated...

#5 Updated by Artem Malahov almost 7 years ago

And that i'am find in repository:
http://www.redmine.org/projects/redmine/repository/diff/trunk/app/models/wiki.rb?rev=4430&rev_to=3632

I think than this changes are bad.
May be problem in russian locale?

My lates version (1.0.3) revision is:

first redmine.iso.vc.bak.test # svn info
Path: .
URL: http://redmine.rubyforge.org/svn/trunk
Repository Root: http://redmine.rubyforge.org/svn
Repository UUID: e93f8b46-1217-0410-a6f0-8f06a7374b81
Revision: 4388
Node Kind: directory
Schedule: normal
Last Changed Author: jplang
Last Changed Rev: 4388
Last Changed Date: 2010-11-08 01:38:10 +0300 (Пнд, 08 Ноя 2010)

#6 Updated by Peng Wang almost 7 years ago

Yes, it took place on my installation too (from 1.0.2 to 1.1.0), editing wiki pages will result in failure

I18n::MissingInterpolationArgument (missing interpolation argument in "'%{page}' wiki页面已更\346\226\260" ({:id=>"Contacts"} given)):
  C:/Program Files/BitNami Redmine Stack/ruby/lib/ruby/gems/1.8/gems/i18n-0.4.2/lib/i18n/backend/base.rb:174:in `interpolate_without_deprecated_syntax'
  C:/Program Files/BitNami Redmine Stack/ruby/lib/ruby/gems/1.8/gems/i18n-0.4.2/lib/i18n/backend/base.rb:46:in `translate'
  C:/Program Files/BitNami Redmine Stack/ruby/lib/ruby/gems/1.8/gems/i18n-0.4.2/lib/i18n.rb:160:in `t'
  lib/redmine/i18n.rb:13:in `l'
  app/models/mailer.rb:198:in `wiki_content_updated'
  app/models/wiki_content_observer.rb:25:in `after_update'
  C:/Program Files/BitNami Redmine Stack/ruby/lib/ruby/1.8/observer.rb:185:in `notify_observers'
  C:/Program Files/BitNami Redmine Stack/ruby/lib/ruby/1.8/observer.rb:184:in `each'
  C:/Program Files/BitNami Redmine Stack/ruby/lib/ruby/1.8/observer.rb:184:in `notify_observers'
  app/controllers/wiki_controller.rb:123:in `update'

#7 Updated by Artem Malahov almost 7 years ago

After several attempts, I found an error:
problem in method .downcase - they don'n work with russian locale...

I am now looking for a solution...

#8 Updated by Artem Malahov almost 7 years ago

Artem Malahov wrote:

After several attempts, I found an error:
problem in method .downcase - they don'n work with russian locale...

I am now looking for a solution...

For me solution is:
  1. install gem unicode
    first models # gem install unicode
    
  2. replace code in /app/models/wiki.rb from
      # find the page with the given title
      def find_page(title, options = {})
        title = start_page if title.blank?
        title = Wiki.titleize(title).downcase
        page = pages.first(:conditions => ["LOWER(title) LIKE ?", title])
        if !page && !(options[:with_redirect] == false)
          # search for a redirect
          redirect = redirects.first(:conditions => ["LOWER(title) LIKE ?", title])
          page = find_page(redirect.redirects_to, :with_redirect => false) if redirect
        end
        page
      end
    

    to
    # find the page with the given title
      def find_page(title, options = {})
        require 'unicode'
        title = start_page if title.blank?
        #title = Wiki.titleize(title).downcase
        title = Unicode::downcase(Wiki.titleize(title))
        page = pages.first(:conditions => ["LOWER(title) LIKE ?", title])
        if !page && !(options[:with_redirect] == false)
          # search for a redirect
          redirect = redirects.first(:conditions => ["LOWER(title) LIKE ?", title])
          page = find_page(redirect.redirects_to, :with_redirect => false) if redirect
        end
        page
      end
    

#9 Updated by Jean-Philippe Lang almost 7 years ago

  • Target version set to 1.1.1

Another option would be to let the database to the job:

page = pages.first(:conditions => ["LOWER(title) LIKE LOWER(?)", title])

Tested OK with postgresql and cyrillic.

#10 Updated by Jean-Philippe Lang almost 7 years ago

  • Affected version (unused) changed from 1.0.5 to 1.1.0
  • Affected version changed from 1.0.5 to 1.1.0

Changing affected version since 1.0.5 stable is not affected.

#11 Updated by Jean-Philippe Lang almost 7 years ago

  • Status changed from New to Resolved

Fix committed in r4680 in trunk. Please, can you confirm the fix?

#12 Updated by Artem Malahov almost 7 years ago

Jean-Philippe Lang wrote:

Fix committed in r4680 in trunk. Please, can you confirm the fix?

yes. i update my revision and got a changes! Thank you!

#13 Updated by Jean-Philippe Lang almost 7 years ago

  • Subject changed from After update (1.0.5) wiki fail to Wiki fails to find pages with cyrillic characters using postgresql
  • Status changed from Resolved to Closed
  • Resolution set to Fixed

Merged in 1.1-stable in r4728.
Thanks for the feedback.

#14 Updated by Etienne Massip over 6 years ago

  • Category set to I18n

Also available in: Atom PDF