Project

General

Profile

Actions

Feature #32424

closed

CommonMark Markdown Text Formatting

Added by Jens Krämer over 4 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Category:
Text formatting
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Fixed

Description

This patch introduces a new text formatting named CommonMark Markdown (GitHub Flavored). It is based on CommonMarker and HTMLPipeline. The formatter was extracted from Planio where it will soon become the default for new accounts.

We built this instead of going with the existing RedCarpet Markdown implementation for a number of reasons:

  • From time to time users who are using the current Markdown formatter ask for a spec / formal list of all supported features. No such thing exists for RedCarpet. There is CommonMark but RedCarpet isn't going to support it in the short to medium term (see next point).
  • The future development of RedCarpet is uncertain. Few excerpts from a GitHub issue , a year ago:

Commonmark won't be supported anytime soon

and

A general message about the project for people skimming this thread: I'm sorry Redcarpet isn't really active anymore but my resources are pretty limited since I'm committed to other open source projects and still a student. Feel free to use any existing alternative ; this project isn't the great tool it used to be when it was maintained by Vicent.

  • With CommonMark evolving as a Markdown spec that is supported by many implementations and endorsed by organizations like Gitlab and GitHub (which both did the switch from RedCarpet to CommonMarker a while ago), it quickly becomes what users expect when they hear 'Markdown'.
  • Migrating existing Textile content is a bit easier since Pandoc has a dedicated Github Flavored Markdown writer module.
  • The HTML pipeline approach encourages splitting up the formatting process into it's different aspects (html generation, sanitizing, syntax highlighting etc) which allows for better testability and has potential for future re-use of components with other text formatters. Further, HTML pipeline filters work on actual DOM nodes, making tasks like adding classes to links etc much more straight forward and less prone to bugs than doing so with regular expressions.

Last but not least, this formatter solves a number of currently open issues regarding the RedCarpet based Markdown Formatter:

  • #19880 (Incorrect syntax for links in Markdown)
  • #20497 (Markdown formatting supporting HTML)
  • #20841 (Bare URLs in Markdown don't have "external" class)
  • #29172 (Markdown: External links broken)

The main reason why we want to introduce this as a third formatting option and not just replace the existing Markdown formatter is line endings - this formatter does not insert hard breaks (<br>) for simple newlines, but requires either a \ or two spaces at the end of the line to do so. Over time the new formatter could become the default and the RedCarpet based one might be renamed to Markdown (legacy) or similar.

English help files are included, and the patch makes sure these are delivered for any other languages as well until corresponding localized versions are available.


Files

0001-CommonMark-Markdown-text-formatter.patch (68.5 KB) 0001-CommonMark-Markdown-text-formatter.patch Jens Krämer, 2019-11-06 09:27
0002-attachments_helper-commonmark.patch (725 Bytes) 0002-attachments_helper-commonmark.patch Use common_mark format for rendering Markdown attachments Martin Cizek, 2019-12-08 23:42
0003-CommonMark-external_links_filter.rb.patch (808 Bytes) 0003-CommonMark-external_links_filter.rb.patch Fix InvalidURIError exception Martin Cizek, 2020-02-21 14:14
0004-CommonMark-formatter_test.rb.patch (2.09 KB) 0004-CommonMark-formatter_test.rb.patch Tests for #35036 - shared implementation with current Markdown formatter Martin Cizek, 2021-04-05 16:31
0005-CommonMark-unify-code-blocks-35104.patch (5.7 KB) 0005-CommonMark-unify-code-blocks-35104.patch See #35104 Martin Cizek, 2021-04-15 22:08
0006-CommonMark-remove-extensive-link-target-restriction.patch (5.8 KB) 0006-CommonMark-remove-extensive-link-target-restriction.patch Fixing the same issue like in #32766 Martin Cizek, 2021-04-17 19:28
0007-CommonMark-update-Sanitize.patch (3.72 KB) 0007-CommonMark-update-Sanitize.patch Update Sanitize & avoid deprecated keywords Martin Cizek, 2021-04-17 20:23
0008-attachments_controller_test.rb.patch (704 Bytes) 0008-attachments_controller_test.rb.patch Adjust assertion in markdown attachment test - insignificant whitespace Martin Cizek, 2021-04-27 21:43
0012-Fixes-Replace-class-var-allowlist-with-a-class-insta.patch (1.03 KB) 0012-Fixes-Replace-class-var-allowlist-with-a-class-insta.patch Marius BĂLTEANU, 2021-07-04 12:21
0011-Render-markdown-attachments-using-markdown-or-common.patch (2.71 KB) 0011-Render-markdown-attachments-using-markdown-or-common.patch Marius BĂLTEANU, 2021-07-04 12:21
0009-Fixes-Layout-HeredocIndentation-Use-2-spaces-for-ind.patch (1.72 KB) 0009-Fixes-Layout-HeredocIndentation-Use-2-spaces-for-ind.patch Marius BĂLTEANU, 2021-07-04 12:21
0010-Fixes-failing-test-on-CommonMark-by-striping-the-tra.patch (1.2 KB) 0010-Fixes-failing-test-on-CommonMark-by-striping-the-tra.patch Marius BĂLTEANU, 2021-07-04 12:21
0008-Replace-deprecated-Sanitize-keywords-32424.patch (3.63 KB) 0008-Replace-deprecated-Sanitize-keywords-32424.patch Marius BĂLTEANU, 2021-07-04 12:21
0006-Fixes-HTTP-500-error-when-invalid-URIs-are-provided-.patch (1.19 KB) 0006-Fixes-HTTP-500-error-when-invalid-URIs-are-provided-.patch Marius BĂLTEANU, 2021-07-04 12:21
0005-Mark-CommonMark-Markdown-GitHub-Flavored-as-experime.patch (967 Bytes) 0005-Mark-CommonMark-Markdown-GitHub-Flavored-as-experime.patch Marius BĂLTEANU, 2021-07-04 12:21
0007-Relax-allowed-protocols-in-links-by-denying-specific.patch (6.48 KB) 0007-Relax-allowed-protocols-in-links-by-denying-specific.patch Marius BĂLTEANU, 2021-07-04 12:21
0004-Pin-html-pipeline-to-2.13.2-32424.patch (739 Bytes) 0004-Pin-html-pipeline-to-2.13.2-32424.patch Marius BĂLTEANU, 2021-07-04 12:21
0003-Pin-sanitize-to-5.2-32424.patch (741 Bytes) 0003-Pin-sanitize-to-5.2-32424.patch Marius BĂLTEANU, 2021-07-04 12:21
0002-Pin-commonmarker-to-0.22-32424.patch (733 Bytes) 0002-Pin-commonmarker-to-0.22-32424.patch Marius BĂLTEANU, 2021-07-04 12:38
0001-Adds-CommonMark-Markdown-GitHub-Flavored-as-third-te.patch (74.5 KB) 0001-Adds-CommonMark-Markdown-GitHub-Flavored-as-third-te.patch Marius BĂLTEANU, 2021-07-04 12:38
demo_hardbreaks_setting.patch (3.56 KB) demo_hardbreaks_setting.patch Marius BĂLTEANU, 2021-07-27 01:29
configuration-setting-for-hardbreaks.patch (4.07 KB) configuration-setting-for-hardbreaks.patch Marius BĂLTEANU, 2021-07-28 00:19
0001-Add-setting-to-controll-the-hardbreaks-behaviour-fro.patch (4.57 KB) 0001-Add-setting-to-controll-the-hardbreaks-behaviour-fro.patch Marius BĂLTEANU, 2021-08-12 00:13
0001-Add-setting-in-admin-to-control-the-hardbreaks-behav.patch (3.89 KB) 0001-Add-setting-in-admin-to-control-the-hardbreaks-behav.patch Marius BĂLTEANU, 2021-09-04 13:21
task_list_view.png (34.6 KB) task_list_view.png Marius BĂLTEANU, 2022-01-22 10:43
task_list_edit.png (63.7 KB) task_list_edit.png Marius BĂLTEANU, 2022-01-22 10:43

Related issues

Related to Redmine - Defect #29172: Markdown: External links brokenClosed

Actions
Related to Redmine - Defect #20841: Bare URLs in Markdown don't have "external" classClosed

Actions
Related to Redmine - Feature #20497: Markdown formatting supporting HTMLClosed

Actions
Related to Redmine - Defect #19880: Incorrect syntax for links in MarkdownClosed

Actions
Related to Redmine - Defect #32563: Redmine 4 crashing with SEGFAULT under stress test when Markdown is usedClosed

Actions
Related to Redmine - Feature #33037: Support for Gitlab flavored Markdown Closed

Actions
Related to Redmine - Feature #32766: Remove the URI limitation from external markdown linksClosed

Actions
Related to Redmine - Feature #35742: Enable task list items for CommonMark text formattingClosedMarius BĂLTEANU

Actions
Related to Redmine - Feature #35747: Allow style attribute for HTML elements in CommonMark formatterClosed

Actions
Related to Redmine - Defect #35752: Warning message "nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri"Closed

Actions
Related to Redmine - Defect #35754: Redmine::WikiFormatting::CommonMark::FormatterTest#test_should_ignore_soft_breaks failsClosedMarius BĂLTEANU

Actions
Related to Redmine - Feature #35035: Refactor text formatting to HTML::PipelineNew

Actions
Related to Redmine - Defect #35765: Code with unsupported code language is not render when CommonMark Markdown is usedClosedMarius BĂLTEANU

Actions
Related to Redmine - Patch #35104: Code blocks - consistent rendering and retaining user-supplied language name in rendered HTMLClosedMarius BĂLTEANU

Actions
Related to Redmine - Feature #35889: Textile and Markdown attachment rendering should support third-party formattersNew

Actions
Related to Redmine - Defect #35892: Redmine::WikiFormatting::CommonMark::FormatterTest#test_footnotes fails with CommonMarker 0.23.2 ClosedMarius BĂLTEANU

Actions
Related to Redmine - Defect #21181: Markdown Formatter incorrectly parses listsClosed

Actions
Related to Redmine - Feature #20698: Add support for task list items in Textile formattingNew

Actions
Related to Redmine - Patch #36807: Remove CommonMark experimental flag and mark as deprecated the RedCarpet MarkdownClosedMarius BĂLTEANU

Actions
Has duplicate Redmine - Defect #22323: Markdown newline rendering brokenClosed

Actions
Actions

Also available in: Atom PDF