Redmine

I attached a patch for this issue.

The fix stores previews and extracted media in an attachment-specific directory and serves media through a dedicated endpoint.

Cache structure¶

The markdownized preview cache structure was changed from a single cache file based on digest_filesize.md to an attachment-specific directory based on the attachment id.
This change is required to support Pandoc's --extract-media option.

Previously, previews were stored as:

markdownized_previews/
└── <digest>_<filesize>.md

With image extraction enabled, the structure now becomes:

markdownized_previews/
└── 4/
    ├── preview.md
    └── media/
        ├── image1.png
        ├── image2.png
        └── image3.png

Pandoc is executed from the derived_cache directory and uses:

--extract-media=markdownized_previews/<attachment_id>/

and

chdir: basedir

This causes Pandoc to generate image references such as:

markdownized_previews/4/media/image1.png

Using an attachment-specific directory allows these generated paths to be mapped directly to the new route:

GET /attachments/markdownized_previews/4/media/image1.png

Using the previous digest_filesize cache file naming scheme would not provide a suitable target directory for extracted media files and would make it difficult to expose them through predictable URLs.

Another option would have been to post-process and rewrite the generated markdown output to replace image paths after the Pandoc conversion. However, this approach would rely on assumptions about Pandoc's generated output and media path conventions.

By using --extract-media together with an attachment-specific directory, the implementation can rely on Pandoc's generated references directly without modifying the markdown afterwards. This is more robust across different document formats (DOCX, ODT, PPTX), supports arbitrary Pandoc-generated media paths, and is less sensitive to future changes in Pandoc's output.

The cleanup logic was also updated to remove the complete preview directory, including any extracted media files, when attachments are removed or cached previews are cleared.

Configuration¶

Regarding the suggested markdownized_preview_max_media_count and markdownized_preview_max_media_total_size settings, I think the existing limits already provide sufficient protection.

markdownized_preview_max_source_size already limits the size of documents processed by Pandoc, including all embedded images. Since extracted media originates from the source document itself, its total size is naturally constrained by the configured source document size limit.

This patch¶

• extracts embedded images using --extract-media
• stores previews and extracted media in an attachment-specific directory
  • support arbitrary Pandoc-generated media paths, including structures such as Pictures/... (ODT), ppt/media/... (PPTX) and media/... (DOCX)
• adds a controller action and route for serving extracted media files
• applies the same permission checks as attachment downloads and thumbnails
• updates preview cleanup to remove extracted media files together with the generated preview
  • DOCX previews containing embedded images
  • ODT previews containing embedded images
  • Media file delivery through the new endpoint
  • Access control checks (authorized, unauthorized and forbidden access)
  • Missing files and invalid paths

The markdownized preview now correctly displays embedded media extracted by Pandoc while preserving Redmine's existing access control rules.

I attached an additional patch 0003 that implements a configurable placeholder mode for embedded images.

The patch introduces a new configuration setting:

markdownized_preview_images: render

• render (default)
  • extracts embedded images using Pandoc's --extract-media option
  • renders images in the generated preview

• placeholder
  • replaces embedded images with placeholders using a Pandoc Lua filter
  • preserves image positions in the document flow
  • does not extract media files
  • displays the image's alternative text when available, otherwise a generated placeholder such as 1.png

The implementation relies on a Pandoc Lua filter and does not perform post-processing of the generated markdown output.

This option provides a lightweight alternative for installations that prefer compact previews and reduced storage usage.
It may also be useful when a quick text-focused preview is preferred over rendering embedded images.

I have attached updated patches for current trunk. They replace the previous patches 0001-0003.

• 0001b-Fix-broken-images-in-markdownized-Office-document-pr.patch
  • Rebased on current trunk.
  • Stricter validation of the requested media path, with an additional test.
  • Only image files are served. Previously, the cached preview.md could also be requested.
  • preview_media is no longer accepted for API authentication, because the images are loaded by the browser with the normal session.
  • The test for deleting the preview cache is skipped when Pandoc is not available, like the other markdownized preview tests.
• 0002b-Add-images-to-Office-document-fixtures-44358.patch
  • Unchanged (the same files as in the previous 0002), only attached again to have a complete set: msword.docx and libreoffice-writer.odt with images.
• 0003b-Add-image-placeholder-mode-44358.patch
  • Rebased on current trunk.
  • configuration.yml.example now documents the value placeholder, which is the value the code checks. It previously documented replace.

The attachment tests pass with the default configuration.

I have attached a new set of patches with a different approach. They replace all previous patches.

The extracted images are now stored as regular attachments of the previewed attachment, instead of files in a cache directory.

Approach¶

• Images embedded in the document are saved as attachments, with the previewed attachment as container (has_many :markdownized_preview_images).
• The preview cache keeps its existing structure:

markdownized_previews/
└── <digest>_<filesize>.md

• A Pandoc Lua filter writes the embedded images to a temporary directory and names them markdownized_preview_image_<n>.<ext>. The Markdown references only this filename, so it contains no attachment id.
• The images are resolved by the existing InlineAttachmentsScrubber, like other inline images. No new route or controller action is needed, they are served by the normal attachment download.
• The images get their permissions from the container, so they are visible to the users who can see the attachment, also after the issue is made private or moved. They cannot be edited or deleted separately, like avatars in User.
• They are removed with the attachment by dependent: :destroy.
• Identical images are stored once on disk by the existing digest deduplication, also across issues. Each attachment still has its own row, so that the permissions stay separate.

How a preview is displayed¶

1. Cached .md exists and all images it references exist for this attachment?
   yes -> display it (normal case)
   no  -> continue

2. Run Pandoc without a lock, it writes the .md to the cache
   and the images to a temporary directory

3. Lock the attachment and check the images again
   complete -> keep them, another request was faster
   missing  -> replace them with the images from the temporary directory

4. Remove the temporary directory and display the preview

The cached preview is shared by all attachments with the same digest and size, for example the copy of an issue, but each attachment has its own images. This is why step 1 also checks the images, and not only the file.

Pandoc runs outside of the transaction, because the conversion can take a long time. Requests for the same attachment can run it at the same time, but the check in step 3 ensures that a page that is already displayed does not lose its images.

--extract-media reads referenced files¶

While working on this I found a problem with --extract-media, which my previous patches used. Pandoc fetches every resource that a document references. With Pandoc 3.7, a crafted ODT with xlink:href="file:///etc/hostname" caused Pandoc to read that file and write it to the media directory. With Markdown input, an http:// URL was fetched too (not tested with ODT). The previous patches only serve image files, but reading the file already happens during conversion.

These patches therefore do not use --extract-media. Pandoc runs with --sandbox, and the Lua filter only looks up images in the media bag of the document, so nothing is fetched. A test fixture covers this (libreoffice-writer-linked-image.odt).

Could someone with the required permissions please remove the previous patches from this issue?.

Pandoc versions¶

--sandbox is available since Pandoc 2.15, older versions exit with an error on this option. Images are therefore exported only with Pandoc 2.15 or later, for all formats. The new options are used together (--sandbox --quiet --lua-filter). --quiet is needed because some distribution packages print warnings for each conversion with --sandbox (Ubuntu 26.04, Pandoc 3.7.0.2: Could not load translations).

With older versions, the conversion runs as before, without the new options and without images. This means that the broken image references of this issue remain there.

Existing cached previews¶

Previews cached by Redmine 7.0 reference images that were never extracted. These patches do not replace them automatically: the check for missing images only looks for the new filenames, so an old preview stays as it is until the cache is cleared. Attachment.clear_markdownized_previews removes the cached previews and now also the preview images, and they are generated again when they are displayed.

This could be done once with a migration, for example:

class ClearMarkdownizedPreviews < ActiveRecord::Migration[8.1]
  def up
    Attachment.clear_markdownized_previews
  end
end

I did not include it, because I found no existing migration that touches files. A note in doc/UPGRADING would be an alternative, please let me know what is preferred.

The same applies after upgrading Pandoc from a version below 2.15: previews that are already cached stay as they are until the cache is cleared.

Patches¶

The binary fixtures are in a separate patch, so that the changes can be read in the issue. The tests need both patches, and the fixture patch requires git apply.

• 0001c-Store-images-of-markdownized-previews-as-attachments.patch: the change itself, with tests.
• 0002c-Add-images-to-Office-document-fixtures.patch: the test fixtures with images.

The attachment tests pass with Pandoc 2.15, 3.7.0.2 and 3.13 on Ruby 4.0 and PostgreSQL 18.
With Pandoc 2.14.2 they pass too, the 10 tests for images are skipped.
A PPTX with an image was also converted correctly with Pandoc 3.13. XLSX with images is not tested.

Review and feedback are welcome, especially on storing the images as attachments and on the handling of existing cached previews.