Defect #30441

Attachments with Unicode uppercase names are not shown in wiki pages

Added by Evgeny Seliverstov 4 months ago. Updated about 1 month ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Go MAEDA% Done:

0%

Category:Attachments
Target version:4.0.4
Resolution:Fixed Affected version:3.4.7

Description

Attachments with Unicode upper-case filenames (like TestТест.png aka Test\xd0\xa2\xd0\xb5\xd1\x81\xd1\x82.png) are not shown when included in wiki inline image macro.

Steps to reproduce:
1. Make a wiki page
2. Attach a file named TestТест.png
3. Write in a wiki page a macro !TestТест.png!

Expected:
- The image is shown as an attachment

Actual:
- The image is not shown because an attachment is not found
- The 404 error is in logs because Redmine tries to extract a generic resource, not an attachment

I have investigated this issue and it seems that the ApplicationHelper#parse_inline_attachments method downcases filenames and then those names are compared against the database in Attachment#latest_attach using String#casecmp. The casecmp method (do not confuse with a newer String#casecmp? method!) cannot deal with Unicode so the comparison fails. This bug can or can not be seen if an attachment name is downcased because casecmp can successfully compare downcased strings because their byte representation is the same.

The problem can be fixed easily just by avoiding calling String#downcase in ApplicationHelper#parse_inline_attachments.
Comparing ASCII filenames with any cases should still work without changes (because of casecmp).
Comparing Unicode filenames should work fine because we do not bother with changing filename at all.

I wrote a patch consisting of:
1. Test AttachmentTest#test_latest_should_get_with_unicode_downcase that reproduces a problem with an attachment search (passes before and after the fix).
2. Test ApplicationHelperTest#test_attached_images_with_textile_and_uppercase_non_ascii_filename that reproduces a problem with inline image macro (passes after the fix).
3. Small fix to ApplicationHelper#parse_inline_attachments that avoids calling downcase.

Related issues:
- #27780 can fix the problem just by calling String#casecmp? but it is ruby-2.4 only
- #20369 introduced the problem in 3.2.0

About system:

Environment:
  Redmine version                3.4.7.stable
  Ruby version                   2.4.4-p296 (2018-03-28) [x86_64-darwin17]
  Rails version                  4.2.11
  Environment                    production
  Database adapter               SQLite
SCM:
  Subversion                     1.10.0
  Mercurial                      4.8.2
  Git                            2.20.1
  Filesystem
Redmine plugins:
  no plugin installed

attachments-unicode-case.diff Magnifier - fix and tests (4.32 KB) Evgeny Seliverstov, 2019-01-16 13:47

test-30441.patch Magnifier (2.53 KB) Go MAEDA, 2019-04-12 06:27


Related issues

Related to Redmine - Defect #27780: Attachment sort doesn't work with Unicode New

Associated revisions

Revision 18057
Added by Go MAEDA about 1 month ago

Attachments with Unicode uppercase names are not shown in wiki pages (#30441).

Patch by Evgeny Seliverstov.

Revision 18058
Added by Go MAEDA about 1 month ago

Test for r18057 (#30441).

Patch by Go MAEDA.

Revision 18060
Added by Go MAEDA about 1 month ago

Merged r18057 and r18058 from trunk to 4.0-stable (#30441).

History

#1 Updated by Go MAEDA about 1 month ago

  • Target version set to Candidate for next minor release

#2 Updated by Go MAEDA about 1 month ago

  • Related to Defect #27780: Attachment sort doesn't work with Unicode added

#3 Updated by Go MAEDA about 1 month ago

Evgeny Seliverstov wrote:

The problem can be fixed easily just by avoiding calling String#downcase in ApplicationHelper#parse_inline_attachments.
Comparing ASCII filenames with any cases should still work without changes (because of casecmp).
Comparing Unicode filenames should work fine because we do not bother with changing filename at all.

I agree. The downcase method in ApplicationHelper#parse_inline_attachments is not necessary because the case-insensitive match is performed for filename variable in Attachment#latest_attach. I think the downcase method should have been removed when implementing #20369.

#4 Updated by Go MAEDA about 1 month ago

  • File test-30441.patchMagnifier added
  • Target version changed from Candidate for next minor release to 4.0.4

Rewrote the test.

#5 Updated by Go MAEDA about 1 month ago

  • Subject changed from Attachments with unicode uppercase names are not shown in wiki to Attachments with Unicode uppercase names are not shown in wiki pages
  • Status changed from New to Resolved
  • Assignee set to Go MAEDA
  • Resolution set to Fixed

Committed the patches. Thank you for reporting and fixing the issue.

#6 Updated by Go MAEDA about 1 month ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF