Defect #4931

Migrating from trac is not possible, fails to allocate memory

Added by el cuco over 7 years ago. Updated over 7 years ago.

Status:ClosedStart date:2010-02-25
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:Importers
Target version:0.9.4
Resolution:Fixed Affected version:

Description

When trying to import a trac repository, the process will fail due to memory allocation errors (tested on Debian, with 512mb ram and a few GB of swap).

Migrating components............................
Migrating milestones.................
Migrating custom fields..
Migrating tickets............................(TRIMMED DOTS)...................................rake aborted!
failed to allocate memory

(See full trace by running task with --trace)

Eventually, this worked when I commented a few lines in the migration script:

turing:~/redmine-0.8# svn diff
Index: lib/tasks/migrate_from_trac.rake
===================================================================
--- lib/tasks/migrate_from_trac.rake    (revision 2903)
+++ lib/tasks/migrate_from_trac.rake    (working copy)
@@ -309,12 +309,12 @@
         text = text.gsub(/\[(\d+)\]/, 'r\1')
         # Ticket number re-writing
         text = text.gsub(/#(\d+)/) do |s|
-          if $1.length < 10
-            TICKET_MAP[$1.to_i] ||= $1
-            "\##{TICKET_MAP[$1.to_i] || $1}" 
-          else
-            s
-          end
+#           if $1.length < 10
+#             TICKET_MAP[$1.to_i] ||= $1
+#             "\##{TICKET_MAP[$1.to_i] || $1}" 
+#           else
+#             s
+#           end
         end
         # We would like to convert the Code highlighting too
         # This will go into the next line.

However, this is "not optimal" and I would like to have a better solution. See: http://www.redmine.org/boards/2/topics/8923

Associated revisions

Revision 3494
Added by Jean-Philippe Lang over 7 years ago

Removes a useless assignment in Trac importer (#4931).

History

#1 Updated by Jean-Philippe Lang over 7 years ago

  • Category set to Importers

#2 Updated by Jean-Philippe Lang over 7 years ago

How many tickets do you have in your Trac database?
Can you retry without the useless assignment:

Index: lib/tasks/migrate_from_trac.rake
===================================================================
--- lib/tasks/migrate_from_trac.rake    (revision 3487)
+++ lib/tasks/migrate_from_trac.rake    (working copy)
@@ -310,7 +310,7 @@
         # Ticket number re-writing
         text = text.gsub(/#(\d+)/) do |s|
           if $1.length < 10
-            TICKET_MAP[$1.to_i] ||= $1
+#            TICKET_MAP[$1.to_i] ||= $1
             "\##{TICKET_MAP[$1.to_i] || $1}" 
           else
             s

#3 Updated by el cuco over 7 years ago

Sorry for the late-update, I have been fighting Debian to get ROR running :)

I have about 1300 tickets, without many interactions (some do mention "ticket:1234" or something similar, but most do not). Your "patch" fixed the issue, now the imported does work.

Let me understand, the memory consumption was O(n^2) when n is the amount of tickets?

#4 Updated by Jean-Philippe Lang over 7 years ago

  • Status changed from New to Resolved
  • Target version set to 0.9.4
  • Resolution set to Fixed

Let me understand, the memory consumption was O(n^2) when n is the amount of tickets?

No, the ticket count was not the problem (1300 is ok).
I think you add many or large #\d+ patterns in some texts (wiki, ticket descriptions...) that were not ticket ids. And they were filling the TICKET_MAP hash.

Thanks for the feedback.

#5 Updated by el cuco over 7 years ago

Yes, each ticket title contains 2-3 numbers... Now lets work on a backport of this to Debian :)

#6 Updated by el cuco over 7 years ago

I have contacted the Debian developers, in order to fix the Debian package for the upcomming release (so Redmine is part of the official distribution). They asked if this bug will be part of 0.9.4 (which apparently will happen, as you tagged it properly).

Anyway, if you are interested, the bug is found here: http://bugs.debian.org/576682 (this includes another bug in Debian + a patch, feel free to comment on this if you want).

#7 Updated by Jean-Philippe Lang over 7 years ago

  • Status changed from Resolved to Closed

Yes, it will be part of 0.9.4. It was merged in 0.9-stable in r3643.

Also available in: Atom PDF