Defect #30411

Filesystem adapter does not show correct size for large files

Added by Felix Schäfer 6 days ago. Updated 6 days ago.

Status:NewStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:SCM
Target version:3.4.8
Resolution: Affected version:4.0.0

Description

We have a couple of large files (~9GB) which size doesn't show correctly.

I suspect source:/tags/4.0.0/lib/redmine/scm/adapters/filesystem_adapter.rb#L84, the conversion to a signed 32-bit Integer (https://apidock.com/ruby/Array/pack), to introduce errors for file sizes that are a little over 2GB.

Changing l and L to q and Q should solve the problem.


Related issues

Related to Redmine - Defect #12279: error with large files in filesystem repo Closed

History

#1 Updated by Go MAEDA 6 days ago

The expression "[File.size(target)].pack('l').unpack('L').first" was introduced in r1509.

According to the commit log, the purpose of the expression is to fix the issue that the adapter shows a negative value for large files under Win32. Maybe it returns -2147483648 for a file with 2GB size and -1 for a file with (4G - 1) bytes. The expression converts -2147483648 and -1 to 2147483648 and 4294967295.

However, if we change the 'l' and 'L' to 'q' and 'Q', -2147483648 and -1 are converted to 18446744071562067968 and 18446744073709551615 (correct values are 2147483648 and 4294967295). So, I think we have to fix this issue in a different way.

The following patch should fix the problem. It converts the value using pack and unpack only if File.size returns negative value.

Index: lib/redmine/scm/adapters/filesystem_adapter.rb
===================================================================
--- lib/redmine/scm/adapters/filesystem_adapter.rb    (revision 17791)
+++ lib/redmine/scm/adapters/filesystem_adapter.rb    (working copy)
@@ -76,12 +76,16 @@
                   not File.basename(e1).match(/^\.+$/) # avoid . and ..
               p1         = File.readable?(t1) ? relative_path : "" 
               utf_8_path = scm_iconv('UTF-8', @path_encoding, p1)
+              size       = File.directory?(t1) ? nil : File.size(t1)
+              # File#size returns a negative value under Win32 when
+              # the file size is >= 2GB and < 4GB
+              size = [size].pack('l').unpack('L').first if size.to_i < 0
               entries <<
                 Entry.new({ :name => scm_iconv('UTF-8', @path_encoding, File.basename(e1)),
                           # below : list unreadable files, but dont link them.
                           :path => utf_8_path,
                           :kind => (File.directory?(t1) ? 'dir' : 'file'),
-                          :size => (File.directory?(t1) ? nil : [File.size(t1)].pack('l').unpack('L').first),
+                          :size => size,
                           :lastrev =>
                               Revision.new({:time => (File.mtime(t1)) })
                         })

#2 Updated by Go MAEDA 6 days ago

  • Category set to SCM
  • Target version set to 3.4.8

#3 Updated by Go MAEDA 5 days ago

  • Related to Defect #12279: error with large files in filesystem repo added

Also available in: Atom PDF