https://www.redmine.org/https://www.redmine.org/favicon.ico?16793021292016-10-10T03:59:33ZRedmineRedmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=739552016-10-10T03:59:33ZJiamin Wang
<ul></ul><p>error occurs in save_revision method in /app/models/git.rb or /app/models/subversion.rb</p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=739592016-10-10T07:02:07ZToshi MARUYAMA
<ul><li><strong>Category</strong> changed from <i>Database</i> to <i>SCM</i></li></ul> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=929082019-07-31T07:46:47ZTobias Fischer
<ul></ul><p>This is still an issue in Redmine 3.4.11</p>
<pre>
Environment:
Redmine version 3.4.11.stable.18254
Ruby version 2.2.1-p85 (2015-02-26) [x86_64-linux]
Rails version 4.2.11.1
Environment production
Database adapter Mysql2
SCM:
Subversion 1.6.17
Git 1.7.9.5
Filesystem
</pre>
<p>Error message in log:</p>
<pre>
Started GET "/projects/PROJECT/repository/REPO_ID" for 37.24.44.210 at 2019-07-31 09:43:30 +0200
Processing by RepositoriesController#show as HTML
Parameters: {"id"=>"PROJECT", "repository_id"=>"REPO_ID"}
Current user: USERNAME (id=5)
Completed 500 Internal Server Error in 145ms (ActiveRecord: 13.2ms)
ActiveRecord::StatementInvalid (Mysql2::Error: Incorrect string value: '\xF0\x9F\xA4\xA6' for column 'comments' at row 1: INSERT INTO `changesets` (`repository_id`, `revision`, `scmid`, `committer`, `commit_date`, `committed_on`, `comments`, `user_id`) VALUES (18, 'a5814b480017f7702d46d084da1aa1a7cb5d05b0', 'a5814b480017f7702d46d084da1aa1a7cb5d05b0', 'Tobias Fischer <tobias.fischer@domain.tld>', '2019-07-04', '2019-07-04 21:35:45', '[ci] gradle file bei vorherigem Commit vergessen</pre> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=929092019-07-31T07:51:31ZTobias Fischer
<ul></ul><p>Funny thing: Also the comment here above get's truncated at the position of the emoji in my log message. Emojis are a big issue for Redmine!</p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=931982019-08-17T16:30:30ZMarius BÄ‚LTEANU
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/31921">Feature #31921</a>: Changes to properly support 4 byte characters (emoji) when database is MySQL</i> added</li></ul> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=972842020-04-15T16:16:25ZToshi MARUYAMA
<ul><li><strong>File</strong> <a href="/attachments/25192">error.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25192/error.png">error.png</a> added</li><li><strong>File</strong> <a href="/attachments/25193">env.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25193/env.png">env.png</a> added</li></ul><p>I have added new test repository and tests because existing test repository encoding is 'ISO-8859-1'.<br /><a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/revisions/19705/entry/trunk/test/unit/repository_git_test.rb#L42">source:trunk/test/unit/repository_git_test.rb@19705#L42</a></p>
<p>On my CentOS 7 MySQL 5.5.62, utf8mb4 environment passes test.<br /><img src="https://www.redmine.org/attachments/download/25193/env.png" alt="" /></p>
<p>With this change,<br /><pre><code class="diff syntaxhl"><span class="gh">diff --git a/test/unit/repository_git_test.rb b/test/unit/repository_git_test.rb
</span><span class="gd">--- a/test/unit/repository_git_test.rb
</span><span class="gi">+++ b/test/unit/repository_git_test.rb
</span><span class="p">@@ -592,7 +592,8 @@</span> class RepositoryGitTest < ActiveSupport:
end
if File.directory?(REPOSITORY_UTF8_PATH) &&
<span class="gd">- !(Redmine::Database::mysql? && !is_mysql_utf8mb4)
</span><span class="gi">+ # !(Redmine::Database::mysql? && !is_mysql_utf8mb4)
+ true
</span> def test_utf8_emoji
repo = Repository::Git.create(
:project => @project,
</code></pre></p>
<p>NOT utf8mb4 environment causes error.</p>
<p><img src="https://www.redmine.org/attachments/download/25192/error.png" alt="" /></p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=972862020-04-15T17:02:14ZToshi MARUYAMA
<ul><li><strong>File</strong> <a href="/attachments/25194">subversion_repository.dump.gz</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25194/subversion_repository.dump.gz">subversion_repository.dump.gz</a> added</li><li><strong>File</strong> <a href="/attachments/25195">svn.diff</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25195/svn.diff">svn.diff</a> added</li></ul><p>This is Subversion test repository and test code.<br />Same results with git.</p>
<p>I give up to commit because it is very risky<br />if <a class="wiki-page" href="https://www.redmine.org/projects/redmine/wiki/Continuous_integration">CI server</a> is NOT utf8mb4, tests skip.</p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=972872020-04-15T17:08:31ZToshi MARUYAMA
<ul><li><strong>File</strong> <a href="/attachments/25196">mercurial.hg</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25196/mercurial.hg">mercurial.hg</a> added</li><li><strong>File</strong> <a href="/attachments/25197">mercurial.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25197/mercurial.png">mercurial.png</a> added</li></ul><p>This is Mercurial bundle file and its results.</p>
<p><img src="https://www.redmine.org/attachments/download/25197/mercurial.png" alt="" /></p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=972882020-04-15T17:15:26ZMarius BÄ‚LTEANU
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Normal</i></li></ul><p>Toshi, what do you think if we assign this to <a class="version" href="https://www.redmine.org/versions/152">4.2.0</a>? We already have a ticket there for utf8mb4 support on mysql (<a class="issue tracker-3 status-5 priority-4 priority-default closed" title="Patch: Add test for 4 byte characters (emoji) support (Closed)" href="https://www.redmine.org/issues/32054">#32054</a>).</p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=972892020-04-15T17:16:08ZToshi MARUYAMA
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Closed</i></li><li><strong>Resolution</strong> set to <i>Invalid</i></li></ul><p>Conclusion:</p>
<p>No error with correct server configuration and "encoding: utf8mb4" of database.yml.<br /><a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/revisions/19705/entry/trunk/config/database.yml.example#L12">source:trunk/config/database.yml.example@19705#L12</a></p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=972902020-04-15T17:18:37ZToshi MARUYAMA
<ul></ul><p>Marius BALTEANU wrote:</p>
<blockquote>
<p>Toshi, what do you think if we assign this to <a class="version" href="https://www.redmine.org/versions/152">4.2.0</a>? We already have a ticket there for utf8mb4 support on mysql, please see <a class="issue tracker-3 status-5 priority-4 priority-default closed" title="Patch: Add test for 4 byte characters (emoji) support (Closed)" href="https://www.redmine.org/issues/32054">#32054</a>.</p>
</blockquote>
<p>Good idea.</p> Redmine - Defect #24030: When SVN or Git repository has a commit comment include an emoji (4 bytes charactor), error occurshttps://www.redmine.org/issues/24030?journal_id=973772020-04-20T02:33:46ZToshi MARUYAMA
<ul><li><strong>File</strong> <a href="/attachments/25240">same-with-r19706.diff</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/25240/same-with-r19706.diff">same-with-r19706.diff</a> added</li></ul><p>This is additional patch.<br />"use same "Redmine::Database.mysql?" as with <a class="changeset" title="Fix a RuboCop offense Style/ColonMethodCall due to r19699 (#24030)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/19706">r19706</a>".</p>