Redmine

Bulk attachment downloads build the whole ZIP archive in memory before sending it, and each file is read entirely with File.binread before it is added. A single request can therefore temporarily require memory of nearly twice the size of the archive, which can be as large as the "Maximum total size for bulk download" setting allows (100 MB by default). When several users download large sets of attachments at the same time, the server may run out of memory. Wiki ZIP exports build their archives in the same way.

The attached patch generates these archives with zip_kit and streams them to the client. Each file is read in small chunks and written directly to the response, so the memory used per request no longer depends on the size of the archive. The response includes X-Accel-Buffering: yes so that nginx buffers it and releases the application process without waiting for slow clients.

The rubyzip gem, which Redmine currently uses to create ZIP archives, cannot stream them, because Zip::OutputStream requires a seekable output in order to rewrite local file headers. Since rubyzip is no longer needed at runtime, the patch moves it to the test group, where it is used to read the generated archives. The number of gems required in production does not change.

This patch changes the following behavior:

• Entries in bulk attachment downloads now use the creation time of each attachment in the user's time zone as their modification time, instead of the time of the download. Wiki ZIP exports already use the last update time of each page in the same way.
• The response no longer includes a Content-Length header, because the size of the archive is not known in advance, so browsers cannot show the progress of the download.