Patch #1651
closed
Hack to make redmine use pygmentize instead of CodeRay
0%
Description
We're going to be using Redmine internally with lots of Python code involved. Thought I'd share this patch with you guys.
I never wrote in ruby before, and I used some of the code mentioned here: http://matt.tarbit.org/2008/02/05/extending-bluecloth-with-pygments
You need to test that pygmentize works on your system first, and if it does, this should work just fine. Unfortunately, syntax highlighting is not a plugin.
Files
Related issues
Updated by Calvin Cheng over 15 years ago
Youssef Abou-Kewik wrote:
We're going to be using Redmine internally with lots of Python code involved. Thought I'd share this patch with you guys.
I never wrote in ruby before, and I used some of the code mentioned here: http://matt.tarbit.org/2008/02/05/extending-bluecloth-with-pygmentsYou need to test that pygmentize works on your system first, and if it does, this should work just fine. Unfortunately, syntax highlighting is not a plugin.
Looks good. It's what I have been looking for!
Coderay has simply been just too limited. Pygments rock in comparison.
Will give this patch a shot. But it looks like I have to maintain this file thereafter since I track the trunk directly.
Updated by Youssef Abou-Kewik over 15 years ago
I'm attaching another update. This one should count fully on pygmentize instead of CodeRay (the previous patch relied on CodeRay to determine file type). However, this one relies on writing a temp file to /tmp where pygmentize can read it and determine its file type on its own. Also, files with UTF8 encoding should work now.
Updated by Anatolii Kucheruck over 15 years ago
Youssef Abou-Kewik wrote:
I'm attaching another update. This one should count fully on pygmentize instead of CodeRay (the previous patch relied on CodeRay to determine file type). However, this one relies on writing a temp file to /tmp where pygmentize can read it and determine its file type on its own. Also, files with UTF8 encoding should work now.
There are one "bug" in this patch: if pygmentize can't recognize file type (e.g. file has no extension or pygmentize has no such type in lexer's list), the error was occurred and empty string was returned, and we have no content.
I'm attaching athother one path with fix this (file type compulsorily set to "text" in described case).
Updated by Kornelius Kalnbach over 15 years ago
It is true that CodeRay's language support is still very limited, but development has started again. We're using Redmine for issue tracking :)
The latest CodeRay version supports some additional languages, but Python is still not included. Python support for CodeRay is scheduled for the next version.
pygments is a very good tool, I can only recommend it - but it would be nice to have a pure-Ruby solution. Ultraviolet is one with a lot of languages, but it has more dependecies and is slower than CodeRay.
Updated by Jean-Philippe Lang over 15 years ago
We're using Redmine for issue tracking :)
Excellent news :)
The latest CodeRay version supports some additional languages
I'll upgrade as soon as possible to this new release. Thanks!
Updated by Eric Davis about 15 years ago
Thank you for the patch. There are a few improvements needed though:
- Requires a UNIX environment. From what I non POSIX systems (Windows) don't support output redirection (e.g. The
>in `echo #{escape_shell_arg(content)} > /tmp/#{basename}`) - Requires several shell commands (
pygmentize,echo,rm) - Shell commands are assumed to be located in the system PATH, which may fail under environments like Apache Passenger
- Files are written and deleted without testing if they exist and are valid first. It wouldn't be too difficult to trick the regex in
escape_shell_argto remove a system file - Sending commands to the shell (
`command`) are run asynciolsly. Thesystem(command)should be used instead.
Updated by Youssef Abou-Kewik about 15 years ago
Well, the patch was for an immediate need that we had at our office. However, I think something like
http://code.google.com/p/syntaxhighlighter/
or
http://shjs.sourceforge.net/
Are actually better solutions than CodeRay or even my patch. I'll try to work something out using one of the solutions above and update this patch when I get some time.
Updated by Kornelius Kalnbach about 15 years ago
The question is: does it highlight Ruby as good as the current highlighter? Most syntax highlighters have very bad Ruby support. That was the original reason for me to write CodeRay.
Updated by Kornelius Kalnbach about 15 years ago
CodeRay development is slow at the moment. Maybe it would be best to make syntax highlighting a plugin (as the original post suggested), and leave the decision to the users. Ultraviolet for example supports a lot more languages and themes, but requires Oniguruma and other libraries, while solutions like JavaScript syntax highlighting minimize server load (I'm guessing here). But RedMine would still need a default syntax highlighter.
Updated by Casseen Star about 15 years ago
I found another patch to use pygments inside redmine: http://jakewharton.com/2008/11/12/using-pygments-in-redmine/
Updated by Eric Davis about 15 years ago
Kornelius Kalnbach wrote:
CodeRay development is slow at the moment. Maybe it would be best to make syntax highlighting a plugin (as the original post suggested), and leave the decision to the users.
I like that idea. Then alternative highlighers can be used and worked on separately.
Updated by Holger Just about 15 years ago
While the solution of Jake Wharton (see note #10) seems to be the cleanest version of the naive implementation, at least the current stable version of Debian does not include the required pygments patch. Therefore we still need a temporary file with the correct file extension to pass to the pygmentize program.
Because of the outstanding issues on the provided patch, I tried to improve it. My current version addresses some of the issues of Eric (#6). I now use the tempfile package (which needed to be extended for Ruby < 1.9 however). This should resolve most issues with unsafe file access. Some operations with weird filenames are still going to fail. But I think, those filenames should still be safe as tempfile is going to refuse to create certain dangerously named files.
Only pygmentize is required to be in the current path. The current version shipped with Debian Lenny (in package python-pygments) works fine. The patch however will probably still not work on Windows as the rewriting of the error stream of pygmentize will fail on that platform. However, this was not tested.
Note: Previous version of the patch perform unsafe file operations. Using specially crafted filenames in the repository it was possible to read, write and delete any file accessible by the user of the webserver. If you use one of the previously provided patches, you should update your installation as soon as possible!
Updated by Sebastien Metrot about 15 years ago
hi,
I tried the different patches to include Pygment sntax hilighting in redmine but none of them seemed to support it in the wiki. I hacked it a bit and came up with a solution that seems to work. I have attached my patch against revision 2666 of the redmine svn.
It was my first time hacking ruby and the solution seems very dirty to me (particularly the & that I have to transform into & by hand... anyone knows a better solution to unescape the wiki code?).
Updated by Jean-Baptiste Barth about 15 years ago
The ampersands becoming "%x%x" is a redcloth bug ; you could have a look at my second patch in #2985, it solves this problem by simply desactivating it in redcloth (be careful, there are other redcloth changes in this patch). I haven't took a deeper look at it to see why redcloth do that, and how to solve it properly sorry...
Updated by Youssef Abou-Kewik almost 15 years ago
- File syntax_highlight.diff syntax_highlight.diff added
OK, so this patch removes the dependency on pygmentize and no longer writes any temp files. We now use javascript highlighter which takes the load to the client side.
Updated by Jiongliang Zhang almost 15 years ago
There is a new way using pygments as highlight tool for repository files: http://www.redmine.org/boards/3/topics/6890
Updated by Go MAEDA over 7 years ago
- Related to Feature #3032: Use google Prettify for syntax highlighting instead of CodeRay added
Updated by Go MAEDA over 7 years ago
- Related to Feature #24681: Syntax highlighter: replace CodeRay with Rouge added
Updated by Go MAEDA over 5 years ago
- Category set to Text formatting
- Status changed from New to Closed
In the upcoming Redmine 4.0, such a hack is no longer necessary because the new syntax highlighter supports a large number of programming languages. Please see #24681 for details.