Feature #31553

Preview .webm as video instead of audio

Added by Martin von Wittich 2 months ago. Updated about 1 month ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Go MAEDA% Done:

0%

Category:Attachments
Target version:4.1.0
Resolution:Fixed

Description

When I attach a WebM video to a Redmine ticket, Redmine's ticket preview uses an <audio> tag instead of a <video> tag to display the video. This is unfortunate because it essentially renders the video preview unusable; the <audio> tag will only play the audio part of the video. Users currently have to download WebM videos to view them.

I've looked into the code and determined that Redmine uses the MiniMime library to determine an attachment's MIME type by looking at the extension, and MiniMime does indeed map webm to audio/webm. Of course, this isn't necessarily wrong as WebM is just a container format that can contain audio and/or video, so it's IMO not a MiniMime bug.

I would suggest to overwrite the webm mapping in Redmine with video/webm and always preview WebM files with a <video> tag, regardless whether it contains audio or video. This will waste a bit space when previewing audio files as browsers will display an empty area for the non-existent video part, but it's probably far easier than actually detecting whether the WebM file contains a video stream or not.

I've attached a short WebM video to illustrate the problem.

redmine.iserv.eu /var/www/redmine # sudo -u www-data ruby bin/about

sh: 1: svn: not found
sh: 1: cvs: not found
sh: 1: bzr: not found
Environment:
  Redmine version                4.0.3.stable
  Ruby version                   2.3.3-p222 (2016-11-21) [x86_64-linux-gnu]
  Rails version                  5.2.2.1
  Environment                    production
  Database adapter               PostgreSQL
  Mailer queue                   ActiveJob::QueueAdapters::AsyncAdapter
  Mailer delivery                sendmail
SCM:
  Mercurial                      4.0
  Git                            2.11.0
  Filesystem
Redmine plugins:
  redmine_agile                  1.4.10
  redmine_checklists             3.1.15
  redmine_custom_css             0.1.7
  redmine_local_avatars          1.0.3
  redmine_revision_branches      0.3.2
  redmineup_tags                 2.0.5

Example Domain.webm (34.8 KB) Martin von Wittich, 2019-06-11 12:34

Download.png (64.5 KB) Martin von Wittich, 2019-06-11 12:47

0001-Preview-.webm-as-video-instead-of-audio.patch Magnifier (698 Bytes) Seiei Miyagi, 2019-06-11 17:16

firefox-error.png (8.11 KB) Go MAEDA, 2019-06-13 10:19

Associated revisions

Revision 18261
Added by Go MAEDA about 1 month ago

Preview .webm as video instead of audio (#31553).

Patch by Seiei Miyagi.

History

#1 Updated by Martin von Wittich 2 months ago

OK, uploading that attachment to illustrate the issue didn't have the intended effect because redmine.org won't preview the attachment. Is redmine.org still running an older Redmine version...?

I've attached a screenshot how the attachment preview looks in our Redmine (using the Circle theme).

#2 Updated by Bernhard Rohloff 2 months ago

  • Status changed from New to Confirmed

I can confirm the behavior and would also prefer to have a preview for video files instead of an audio player. I think it's more common for people to attach videos to their tickets.

Martin von Wittich wrote:

...Is redmine.org still running an older Redmine version...?

Yes redmine.org runs a version which is far older than the latest release.

#4 Updated by Go MAEDA 2 months ago

  • Target version set to Candidate for next minor release

#5 Updated by Go MAEDA 2 months ago

After applying the patch, Firefox shows the following error instead of a video. The Japanese text in the screenshot says "No video with supported format and mime type found".

#6 Updated by Martin von Wittich 2 months ago

Go MAEDA wrote:

After applying the patch, Firefox shows the following error instead of a video. The Japanese text in the screenshot says "No video with supported format and mime type found".

Oops... I thought I had tested my WebM file both in Firefox and in Chrome, but you're right, I can reproduce the issue in Firefox. Sorry for that, I should've probably taken more care to choose an example file that works :)

I had created this file with the Chrome extension Screencastify. According to ffmpeg, it's encoded with h264:

Input #0, matroska,webm, from 'Example Domain.webm':
  Metadata:
    encoder         : w69b-mediawriter
  Duration: 00:00:01.85, start: 0.000000, bitrate: 154 kb/s
    Stream #0:0(eng): Video: h264 (Constrained Baseline), yuv420p(progressive), 746x720, SAR 1:1 DAR 373:360, 1k tbr, 1k tbn, 2k tbc (default)

But apparently, h264 isn't even an allowed video codec for the WebM container format, so Firefox isn't in the wrong here. Maybe the demo files provided by webmfiles.org are more suitable? https://www.webmfiles.org/demo-files/

I've attached http://dl5.webmfiles.org/elephants-dream.webm to a test ticket in our production Redmine and manually changed the <audio> tag to a <video> tag (I don't have a dev instance ready so I can't test the patch properly), and that one seems to work fine in Firefox and in Chrome.

#7 Updated by Go MAEDA about 1 month ago

  • Target version changed from Candidate for next minor release to 4.1.0

Setting the target version to 4.1.0.

#8 Updated by Bernhard Rohloff about 1 month ago

I've attached the patch to my dev environment and it works flawlessly with my test video made by a screen recorder. Very nice improvement!

#9 Updated by Go MAEDA about 1 month ago

  • Subject changed from Attachment preview plays WebM video with <audio> instead of <video> to Preview .webm as video instead of audio
  • Status changed from Confirmed to Closed
  • Assignee set to Go MAEDA
  • Resolution set to Fixed

Committed the patch. Thank you all for working on this issue.

#10 Updated by Go MAEDA about 1 month ago

  • Tracker changed from Defect to Feature

#11 Updated by Martin von Wittich about 1 month ago

Thanks for the fix!

Unfortunately a colleague just noticed that *.wmv is also affected; mini_mime maps it to audio/x-ms-wmv instead of video/x-ms-wmv:

wmv         audio/x-ms-wmv                                                            base64          

This time it seems more like a bug in mini_mime to me; after all, the V in WMV stands for video. The upstream MIME database that mini_mime is based on maps wmv both to audio/x-ms-wmv and video/x-ms-wmv, but mini_mime apparently only takes the first match and ignores the other one.

Should I file a separate Redmine issue for WMV, or should I file against mini_mime?

EDIT: I'm wondering whether mini_mime is still the right tool for the job, now that we've found multiple cases where its simplistic mapping of extensions to a single MIME type has proven problematic. How about using MIME::Types instead? It returns all MIME types for a given extension, and this makes it pretty easy to tell if a certain extension is capable of containing video or not:

irb(main):022:0* MIME::Types.type_for('test.wmv')
=> [#<MIME::Type: audio/x-ms-wmv>, #<MIME::Type: video/x-ms-wmv>]
irb(main):023:0> MIME::Types.type_for('test.webm')
=> [#<MIME::Type: audio/webm>, #<MIME::Type: video/webm>]
irb(main):027:0> MIME::Types.type_for('test.mp3')
=> [#<MIME::Type: audio/mpeg>]
irb(main):024:0> MIME::Types.type_for('test.wmv').find { |m| m.media_type == 'video' }
=> #<MIME::Type: video/x-ms-wmv>
irb(main):025:0> MIME::Types.type_for('test.webm').find { |m| m.media_type == 'video' }
=> #<MIME::Type: video/webm>
irb(main):026:0> MIME::Types.type_for('test.mp3').find { |m| m.media_type == 'video' }
=> nil

#12 Updated by Pavel Rosický about 1 month ago

I agree that in these cases a video format should be prefered.

MIME::Types instead? It returns all MIME types for a given extension, and this makes it pretty easy to tell if a certain extension is capable of containing video or not

you can't determine just according to an extension like webm if it contains a video or an audio because it could contain both. It's a matter of preference.

the main reason why redmine switched from mime-types were memory savings https://github.com/mime-types/ruby-mime-types/issues/123
mini_mime shares the same db as MIME::Types but it assumes only one return. Lookups return the highest priority, nonobsolete, registered match.

Martin von Wittich could you open an issue on https://github.com/discourse/mini_mime/issues and link this ticket? thanks

#13 Updated by Martin von Wittich about 1 month ago

Pavel Rosický wrote:

MIME::Types instead? It returns all MIME types for a given extension, and this makes it pretty easy to tell if a certain extension is capable of containing video or not

you can't determine just according to an extension like webm if it contains a video or an audio because it could contain both. It's a matter of preference.

Well, yes, but that's pretty much exactly what Redmine is trying to do here. It's true that you can only properly tell whether a given file contains a video stream if you actually look into the file, but for Redmine's purposes, just knowing whether a given file format could theoretically contain a video stream would be enough.

the main reason why redmine switched from mime-types were memory savings https://github.com/mime-types/ruby-mime-types/issues/123

OK, didn't know about that.

mini_mime shares the same db as MIME::Types but it assumes only one return. Lookups return the highest priority, nonobsolete, registered match.

Unfortunately, this very concept of mini_mime seems fundamentally broken to me. mini_mime assumes that MIME::Types' priority_compare will return the most fitting MIME type for a certain extension, ordered by some kind of priority - _but there is no concept of priority in MIME::Types@_. @priority_compare essentially just runs simplified on the MIME types and then compares them alphabetically. This means that audio will always win over video:

irb(main):005:0> MIME::Types.type_for('test.wmv').each { |m| pp m.simplified }; nil
"audio/x-ms-wmv" 
"video/x-ms-wmv" 
=> nil

So the MIME types returned by mini_mime are not the most fitting ones, as one would expect, but essentially those who come first in the alphabet. If there were such a thing as priorities in MIME::Types, this could work, but at the moment, I believe that mini_mime needs to be adapted so that it can return multiple MIME types. This would enable Redmine to properly determine whether a given file extension could in theory contain a video stream.

Martin von Wittich could you open an issue on https://github.com/discourse/mini_mime/issues and link this ticket? thanks

Done: https://github.com/discourse/mini_mime/issues/26

Also available in: Atom PDF