Feature #13051

Support any macro in (pdf) export for wiki's and issues

Added by Terence Mill over 5 years ago. Updated over 3 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Jean-Philippe Lang% Done:

0%

Category:PDF export
Target version:3.0.0
Resolution:Fixed

Description

This issue will be a container (link by reference other issues) for all the defect and feature issues existing,
which mostly pick just one part of the problem (macro).

At the moment the export function ignores macros or even there get stripped out completly (this is a workaround - no solution!).
Besides redmine core macros, plugin itself bring many macros into place which don't get rendered in pdf like in html view - that's a big disadvantage.
From the user side, the expectation is wysiwyg from the screen to any export format - no matter if html or pdf.

Yes, it' s a challeging task, but rich export is a strong feature in every system.

From my point of view its a general topic, and a solution in best can export pdf the same way the content area in the web page looks like. If pdf is problematic for this, maybe another export format (optimized for transport and print) is a better approach.

Related issues:

#9842 - {{toc}} is not replaced by table of content when exporting wiki page to pdf
#11697 - Problem with Wiki export to PDF / HTML / TXT — 'include' does not includes
#10689 - PDF export prints brackets on wiki links
#9783 - PDF Export ignores Child pages
#4992 - Add printer friendly version of pages
#275 - Wiki to Documents
#550 - Function to export whole wiki

Btw. regarding the last issues above there are already plugin's like redmine_pdf_documents that make use of the redmine core export, and beeing able to custom sort and create a booklet from selected wiki pages to put out on big pdf in own order and selected pages.

pdf_macro.patch Magnifier (206 Bytes) Paulo Neves, 2013-09-27 22:38

pdf-macro-trunk-r12194.diff Magnifier (782 Bytes) Toshi MARUYAMA, 2013-10-02 08:28

pdf_macro.patch Magnifier - Less bugs and more progressively more functionality (10.4 KB) Paulo Neves, 2013-10-17 12:17


Related issues

Related to Redmine - Defect #11697: Problem with Wiki export to PDF / HTML / TXT — 'include' ... New
Related to Redmine - Defect #10689: PDF export prints brackets on wiki links Closed
Related to Redmine - Defect #9783: PDF Export ignores Child pages Closed
Related to Redmine - Feature #4992: Add printer friendly version of pages New 2010-03-05
Related to Redmine - Feature #550: Function to export whole wiki Reopened
Related to Redmine - Defect #14412: Wiki > Textile text with hidden part not correctly export... New
Duplicated by Redmine - Defect #12964: Macros should be rendered to PDF Closed
Duplicated by Redmine - Defect #17970: Wiki PDF export doesn't render macros Closed
Duplicated by Redmine - Defect #18748: exporting wiki with child page not work correctly Closed

Associated revisions

Revision 13944
Added by Jean-Philippe Lang over 3 years ago

Adds support for macro and Redmine links in PDF export (#13051).

Revision 13946
Added by Jean-Philippe Lang over 3 years ago

Render issue description and journal notes with #textilizable (#13051).

History

#1 Updated by Toshi MARUYAMA over 5 years ago

  • Category set to PDF export

#2 Updated by Sergey Prosin over 5 years ago

That would be really useful functionality because it will allow us to use redmine wiki as project documentation source. Especially with redmine_pdf_documents plug-in available.

#3 Updated by Sergio Cambra over 5 years ago

But redmine_pdf_documents doesn't work in 2.x

#5 Updated by Sergio Cambra over 5 years ago

Thanks, README is misleading becuase is read there is no redmine 2.x port

#6 Updated by Massimo Rossello over 5 years ago

+1

#7 Updated by Terence Mill over 5 years ago

The redmine_pdf_wiki plugin uses wkhtml2pdf to export pdf and promises to resolve macros.
A big disadavantge is that it's only runny on passenger.

#8 Updated by David Majorel about 5 years ago

#14412 - Wiki > Textile text with hidden part not correctly exported to pdf should be listed in the related issues

#9 Updated by Toshi MARUYAMA about 5 years ago

  • Related to Defect #14412: Wiki > Textile text with hidden part not correctly exported to pdf added

#10 Updated by Paulo Neves about 5 years ago

I think I have found a way to call macros when a pdf export is requested.

The function responsible for converting Redmine's textile to html in the pdf exportation class is formatted_text. This function then calls the following function to convert the text from textile to html:

 Redmine::WikiFormatting.to_html(Setting.text_formatting, text)

I noticed that macros were not processed when the pdf export request was issued. Knowing that both the pdf parser and the browser view were rendered based on html, and that the browser was correctly rendering the hello world macro, it looked like different code paths were being followed. So I looked at the rendering of the View and found a method called textilizable. This method also converted the textile data into html, but did a bit more work, including processing the macros. Contained in it is also the Redmine::WikiFormatting.to_html method.

The problem of macros not always translating well to other usages, besides in a browser, will be dealt later.

Logically the only change to be made was to change WikiFormatting.to_html to textilizable and that is what I have done. I had some problems including the right libraries and when I was out of ideas why the errors were so strange, I looked for how textilizable was being used in the unit tests. Making the appropriate changes made everything work with very few changes.

To apply the patch (Redmine 2.3.2) simply type:

patch /path/to/redmine-2.3.2/lib/redmine/export/pdf.rb /path/to/pdf_macro.patch

Hope this helps.

#11 Updated by Toshi MARUYAMA about 5 years ago

  • Duplicated by Defect #12964: Macros should be rendered to PDF added

#12 Updated by Toshi MARUYAMA about 5 years ago

Note-11 patch is not unified diff.
This is converted patch.

#13 Updated by Yurii Monakov about 5 years ago

Patch from Note-12 causes error 500 on issue and Wiki PDF export (Redmine 2.3.1 and Redmine 2.3.3). Logs:

  • Issue export
    NoMethodError (undefined method `truncate' for #<Redmine::Export::PDF::ITCPDF:0xbfbe9f0>):
      app/helpers/application_helper.rb:697:in `block in parse_redmine_links'
      app/helpers/application_helper.rb:666:in `gsub!'
      app/helpers/application_helper.rb:666:in `parse_redmine_links'
      app/helpers/application_helper.rb:513:in `block (2 levels) in textilizable'
      app/helpers/application_helper.rb:512:in `each'
      app/helpers/application_helper.rb:512:in `block in textilizable'
      app/helpers/application_helper.rb:533:in `parse_non_pre_blocks'
      app/helpers/application_helper.rb:511:in `textilizable'
      lib/redmine/export/pdf.rb:118:in `formatted_text'
      lib/redmine/export/pdf.rb:144:in `RDMwriteHTMLCell'
      lib/redmine/export/pdf.rb:668:in `block in issue_to_pdf'
      lib/redmine/export/pdf.rb:660:in `issue_to_pdf'
      app/controllers/issues_controller.rb:124:in `block (2 levels) in show'
      app/controllers/issues_controller.rb:116:in `show'
    
  • Wiki export
    NoMethodError (undefined method `strip_tags' for #<Redmine::Export::PDF::ITCPDF:0xcaf7254>):
      app/helpers/application_helper.rb:821:in `block in parse_headings'
      app/helpers/application_helper.rb:819:in `gsub!'
      app/helpers/application_helper.rb:819:in `parse_headings'
      app/helpers/application_helper.rb:516:in `textilizable'
      lib/redmine/export/pdf.rb:118:in `formatted_text'
      lib/redmine/export/pdf.rb:144:in `RDMwriteHTMLCell'
      lib/redmine/export/pdf.rb:769:in `write_wiki_page'
      lib/redmine/export/pdf.rb:749:in `wiki_page_to_pdf'
      app/controllers/wiki_controller.rb:83:in `show'
    

#14 Updated by Paulo Neves about 5 years ago

Yes Yurii I found that problem and much more. it turned out that it was not straight forward. I have made significant progress to enable the PDF rendering to use textilizable but it is not there yet(wiki links are not parsed and some internal errors are still not solved) if you are in a hurry I can drop here the in progress patch, maybe you could finish it. I am sorry I have not given the follow up but I have been very busy.

#15 Updated by Paulo Neves about 5 years ago

Here goes the patch, as mentioned in my note before. Wiki links are not working and sometimes unescaped quoted also don't work. I will write about it further when I have time.

I also hope that now the patch is in the correct format.

#16 Updated by Paulo Neves almost 5 years ago

After some investigation I concluded that the problem with the crashes was due to text.html_safe in the textilizable method.

    if @parsed_headings.any?
      replace_toc(text, @parsed_headings)
    end

    text.html_safe

  end

For some reason the CGI.unescapeHTML crashes when the string coming out of textilizable is html safe and has quotes. It is not possible remove the html_safe method in textilizable because it would break most of the code relying in it. I have researched html_safe method and found that the way it is implemented in textilizable is dubious because there is user input being considered safe. I am not sure of additional security considerations, as I have tried injecting Javascript without success.

I found that the problem described above (CGI.unescapeHTML not failing) is really weird, because if I apply html_safe to the text and then perform a "cast" to force the SafeBuffer no error is triggered. On the other hand, even when calling CGI.unescapeHTML on an object returned by to_s method, an error is triggered. It should be noted that the to_s method simply returns self, so it is maybe expected that it's calling is redundant, and obviously would trigger the error just as a normal SafeBuffer would.

The solution I found was to keep everything unchanged in textilizable but performing a hard string cast, string interpolation in the formatted_text(text) method in pdf.rb

Finally I was able to reproduce and definitely confirm the error in the Rails console:

CGI.unescapeHTML('The experimental macro Hello Latex[&amp;quot;\&amp;quot;sdsd\&amp;quot;&amp;quot;'.html_safe)

I will now report it to the Rails mailing list to hear their opinion.

The last problem standing is the links to the Wikis embedded in the pdf. I don't know yet why they are not working but I will play with the Rails console to expedite the resolution of the problem. Afterwards I believe a patch is ready, and the tests should be generated.

#17 Updated by Piotr Oh over 4 years ago

Hi Paulo, is there any progress with your patch?

#18 Updated by Toshi MARUYAMA about 4 years ago

  • Duplicated by Defect #17970: Wiki PDF export doesn't render macros added

#19 Updated by Toshi MARUYAMA almost 4 years ago

https://github.com/redmine/redmine/pull/48

diff --git a/lib/redmine/export/pdf.rb b/lib/redmine/export/pdf.rb
index 00c6d13..67b0c8f 100644
--- a/lib/redmine/export/pdf.rb
+++ b/lib/redmine/export/pdf.rb
@@ -62,7 +62,9 @@ def formatted_text(text)
           html = Redmine::WikiFormatting.to_html(Setting.text_formatting, text)
           # Strip {{toc}} tags
           html.gsub!(/<p>\{\{([<>]?)toc\}\}<\/p>/i, '')
-          html
+      # Strip {{collapse()}} tags
+      html.gsub!(/\{\{collapse\((.*)?\)([^}]*)?\}\}/,'\1 \2')
+      html
         end

         def RDMCell(w ,h=0, txt='', border=0, ln=0, align='', fill=0, link='')
</class>

#20 Updated by Toshi MARUYAMA almost 4 years ago

  • Duplicated by Defect #18748: exporting wiki with child page not work correctly added

#21 Updated by Qing Li almost 4 years ago

+1

#22 Updated by Qing Li over 3 years ago

Please testing include macro, the bug have not been fixed, I am using 2.6.0 stable version.

#23 Updated by Jean-Philippe Lang over 3 years ago

  • Status changed from New to Closed
  • Assignee set to Jean-Philippe Lang
  • Target version set to 3.0.0
  • Resolution set to Fixed

Fixed in r13944, macros are now processed in the PDF export, as well as Redmine and wiki links.

Also available in: Atom PDF