Defect #19529

downloaded files are truncated to 64Kb (or simply a 500 error) on second download

Added by Antoine Beaupré over 2 years ago. Updated almost 2 years ago.

Status:ClosedStart date:
Priority:HighDue date:
Assignee:-% Done:

0%

Category:SCM
Target version:-
Resolution:No feedback Affected version:2.5.1

Description

We have had a weird bug on our recently-upgraded Redmine server. It could be two bugs, but the symptoms seem so closely related that I feel they should be filed as one for now.

The problem is we can't download files twice from the "repository" tab reliably. For files larger than 64KB, they get truncated to 64KB the second time. Nothing really interesting shows up in the logs:

Started GET "/projects/civicrm-saas-traduction/repository/revisions/master/raw/l10n/fr_CA/civicrm.po" for X.X.X.X at 2015-03-30 17:10:14 -0400
Processing by RepositoriesController#raw as HTML
  Parameters: {"id"=>"civicrm-saas-traduction", "rev"=>"master", "path"=>"l10n/fr_CA/civicrm.po"}
  Current user: XXXXXXX (id=X)
  Rendered text template (0.0ms)
Sent data civicrm.po  (4.7ms)
Completed 200 OK in 807.8ms (Views: 4.5ms | ActiveRecord: 327.4ms)

Started GET "/projects/civicrm-saas-traduction/repository/revisions/master/raw/l10n/fr_CA/civicrm.po" for X,X,X,X at 2015-03-30 17:10:28 -0400
Processing by RepositoriesController#raw as */*
  Parameters: {"id"=>"civicrm-saas-traduction", "rev"=>"master", "path"=>"l10n/fr_CA/civicrm.po"}
  Current user: anonymous
  Rendered text template (0.0ms)
Sent data civicrm.po  (0.8ms)
Completed 200 OK in 33.1ms (Views: 0.7ms | ActiveRecord: 23.1ms)

The above is downloading the same "larger than 64KB" file twice. No error shows up in the production.log on the second try, but the file is truncated. Note that this works across user, and the phenomenon goes away after a while (say ~24h? need to be confirmed).

Now here's an example of a file smaller than 64KB:

Started GET "/projects/kt-sa-debian-upgrades/repository/revisions/master/entry/predict-os-2015-03-19.png" for 70.83.139.100 at 2015-03-30 17:10
:54 -0400
Processing by RepositoriesController#entry as HTML
  Parameters: {"id"=>"kt-sa-debian-upgrades", "rev"=>"master", "path"=>"predict-os-2015-03-19.png"}
  Current user: anarcat (id=3)
Creating scope :system. Overwriting existing method Enumeration.system.
Creating scope :sorted. Overwriting existing method User.sorted.
  Rendered text template (0.0ms)
Sent data predict-os-2015-03-19.png  (1.7ms)
Completed 200 OK in 592.4ms (Views: 1.4ms | ActiveRecord: 273.7ms)
Creating scope :sorted. Overwriting existing method Group.sorted.

Started GET "/projects/kt-sa-debian-upgrades/repository/revisions/master/entry/predict-os-2015-03-19.png" for 70.83.139.100 at 2015-03-30 17:11
:02 -0400
Processing by RepositoriesController#entry as */*
  Parameters: {"id"=>"kt-sa-debian-upgrades", "rev"=>"master", "path"=>"predict-os-2015-03-19.png"}
  Current user: anonymous
Completed 500 Internal Server Error in 35.9ms

ArgumentError (invalid byte sequence in UTF-8):
  lib/redmine/core_ext/string.rb:9:in `count'
  lib/redmine/core_ext/string.rb:9:in `is_binary_data?'
  app/controllers/repositories_controller.rb:199:in `is_entry_text_data?'
  app/controllers/repositories_controller.rb:175:in `entry_and_raw'
  app/controllers/repositories_controller.rb:161:in `entry'

The first attempt works, the second attempt completely fails with the above 500 error.

Environment:
  Redmine version                2.5.1.stable (2.5.1-2~bpo70+5 from Debian, upgraded from 1.4)
  Ruby version                   1.9.3-p194 (2012-04-20) [x86_64-linux]
  Rails version                  3.2.6
  Environment                    production
  Database adapter               MySQL (5.5.41-0+wheezy1)
SCM:
  Subversion                     1.6.17
  Mercurial                      2.2.2
  Cvs                            1.12.13
  Git                            1.9.1
  Filesystem
Redmine plugins:
  redmine_bootstrap_kit          0.2.0
  redmine_git_hosting            0.7.10
  redmine_openid_selector        0.0.1
  redmine_stealth                0.6.0

History

#1 Updated by Toshi MARUYAMA over 2 years ago

  • Status changed from New to Needs feedback

Antoine Beaupré wrote:

Ruby version 1.9.3-p194 (2012-04-20) [x86_64-linux]

Too old.

Database adapter MySQL (5.5.41-0+wheezy1)

You need to use mysql2 instead of mysql on Ruby >= 1.9.

#2 Updated by Toshi MARUYAMA over 2 years ago

ArgumentError (invalid byte sequence in UTF-8):

I think it is caused by one of your plugin because Redmine core uses "ASCII-8BIT".
source:tags/2.5.1/lib/redmine/scm/adapters/abstract_adapter.rb#L256

#3 Updated by Antoine Beaupré over 2 years ago

Toshi MARUYAMA wrote:

Antoine Beaupré wrote:

Ruby version 1.9.3-p194 (2012-04-20) [x86_64-linux]

Too old.

Hmm... that's strange, the redmine wiki mentions this is supportedm even in the current trunk:

http://www.redmine.org/projects/redmine/wiki/RedmineInstall

What would be a "not too old" ruby release?

Database adapter MySQL (5.5.41-0+wheezy1)

You need to use mysql2 instead of mysql on Ruby >= 1.9.

Any suggestions on how to do that?

Toshi MARUYAMA wrote:

ArgumentError (invalid byte sequence in UTF-8):

I think it is caused by one of your plugin because Redmine core uses "ASCII-8BIT".
source:tags/2.5.1/lib/redmine/scm/adapters/abstract_adapter.rb#L256

I see, i'll take a look at those. I don't see why it would affect image downloads, intermittently too...

#4 Updated by Toshi MARUYAMA over 2 years ago

Antoine Beaupré wrote:

Toshi MARUYAMA wrote:

Antoine Beaupré wrote:

Ruby version 1.9.3-p194 (2012-04-20) [x86_64-linux]

Too old.

Hmm... that's strange, the redmine wiki mentions this is supportedm even in the current trunk:

2012-04-20 is too old.

https://www.ruby-lang.org/en/news/2015/02/23/support-for-ruby-1-9-3-has-ended/

http://www.redmine.org/projects/redmine/wiki/RedmineInstall

What would be a "not too old" ruby release?

https://www.ruby-lang.org/en/downloads/

Database adapter MySQL (5.5.41-0+wheezy1)

You need to use mysql2 instead of mysql on Ruby >= 1.9.

Any suggestions on how to do that?

source:tags/2.6.5/config/database.yml.example#L1

#5 Updated by Antoine Beaupré over 2 years ago

Toshi MARUYAMA wrote:

Antoine Beaupré wrote:

Toshi MARUYAMA wrote:

Antoine Beaupré wrote:

Ruby version 1.9.3-p194 (2012-04-20) [x86_64-linux]

Too old.

Hmm... that's strange, the redmine wiki mentions this is supportedm even in the current trunk:

2012-04-20 is too old.

https://www.ruby-lang.org/en/news/2015/02/23/support-for-ruby-1-9-3-has-ended/

then why does the redmine wiki page shows 1.9.3 as a requirement? heck, for 2.6, 1.8.7 is a requirement!

seems to me this would need to be changed at the very least.

http://www.redmine.org/projects/redmine/wiki/RedmineInstall

What would be a "not too old" ruby release?

https://www.ruby-lang.org/en/downloads/

I think it would have been fair to assume I know where to find Ruby releases. My point here is more that Redmine's requirements need to be updated.

Are you saying that no one should be running Redmine under anything but the latest release (2.2.x at this time)?

Database adapter MySQL (5.5.41-0+wheezy1)

You need to use mysql2 instead of mysql on Ruby >= 1.9.

Any suggestions on how to do that?

source:tags/2.6.5/config/database.yml.example#L1

i see.

#6 Updated by Toshi MARUYAMA over 2 years ago

I said 1.9.3- p194 (2012-04-20) is too old.
ruby-1.9.3- p551 (2014-11-13) is last 1.9.3.
http://cache.ruby-lang.org/pub/ruby/1.9/

Antoine Beaupré wrote:

then why does the redmine wiki page shows 1.9.3 as a requirement?

https://github.com/rails/rails/blob/v4.2.1/rails.gemspec#L10

heck, for 2.6, 1.8.7 is a requirement!

https://github.com/rails/rails/blob/v3.2.21/rails.gemspec#L12

#7 Updated by Toshi MARUYAMA almost 2 years ago

  • Status changed from Needs feedback to Closed
  • Resolution set to No feedback

No feedback.

Also available in: Atom PDF