Patch #14567

migrate_from_trac.rake does not convert timestamps in Trac database version 23

Added by Craig Rodrigues over 5 years ago. Updated about 5 years ago.

Status:NewStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:Importers
Target version:-

Description

In Trac API 0.12, the representation of timestamps was changed from seconds since the epoch
to microseconds since the epoch:
http://trac.edgewall.org/wiki/TracDev/ApiChanges/0.12#Timestampstorageindatabase

See this patch that changed it in Trac:

http://trac.edgewall.org/attachment/ticket/6466/6466-microsecond-times-r8770.patch

If you try to run the existing migrate_from_trac.rake script, the year will be thousands
of years in the future, and the importer will populate the date in redmine is 0000000000.
This causes many things to break.
The problem still exists with the trunk version of the migrate_from_trac.rake script.

In #882 , the user encountered the problem, but went down the wrong path with the fix,
because he didn't understand why timestamps were being imported incorrectly.

I am submitting a better fix, which detects the database version of Trac. If the database
version is > 22, then every time a timestamp from Trac is encountered, we divide by 1 million.

This patch is against trunk.
I have used this patch successfully and it fixes all my problems importing from Trac.

I think many people are hitting these issues when doing Trac -> Redmine migrations. Here are some example tickets:

#5764
#8690
#12477

migrate_from_trac.rake.patch.txt Magnifier (5.31 KB) Craig Rodrigues, 2013-07-29 16:41


Related issues

Related to Redmine - Patch #5764: migrate_from_trac does not support trac 0.12 New 2010-06-27
Related to Redmine - Defect #8690: User can't be displayed Needs feedback 2011-06-27
Related to Redmine - Defect #12477: Internal server error after migrate from Trac 0.12.3 ro R... Needs feedback
Related to Redmine - Defect #14590: migrate_from_trac.rake does not import Trac users, uses t... Closed
Related to Redmine - Patch #14592: migrate_from_trac.rake does not properly parse First Name... Closed
Related to Redmine - Patch #14593: migrate_from_trac.rake, print out validation errors when ... New
Related to Redmine - Defect #14844: migrate_from_trac is not run in database transaction New
Related to Redmine - Defect #6868: migrate_from_trac and trac 0.12 Reopened 2010-11-11
Related to Redmine - Defect #10738: trac import issue New
Related to Redmine - Defect #20943: migrate_from_trac.rake dont work Resolved
Duplicated by Redmine - Defect #14845: Error in user controller Closed
Duplicated by Redmine - Defect #17898: Exception in Migration from Trac to Redmine (incl. solution) Closed

History

#1 Updated by Toshi MARUYAMA over 5 years ago

  • Description updated (diff)

#2 Updated by Toshi MARUYAMA over 5 years ago

  • Description updated (diff)

#3 Updated by Toshi MARUYAMA over 5 years ago

  • Related to Patch #5764: migrate_from_trac does not support trac 0.12 added

#4 Updated by Toshi MARUYAMA over 5 years ago

#5 Updated by Toshi MARUYAMA over 5 years ago

  • Related to Defect #12477: Internal server error after migrate from Trac 0.12.3 ro Redmine 2 added

#6 Updated by Toshi MARUYAMA over 5 years ago

  • Related to Defect #14590: migrate_from_trac.rake does not import Trac users, uses too short password added

#7 Updated by Toshi MARUYAMA over 5 years ago

  • Related to Patch #14592: migrate_from_trac.rake does not properly parse First Name and Last Name added

#8 Updated by Toshi MARUYAMA over 5 years ago

  • Related to Patch #14593: migrate_from_trac.rake, print out validation errors when doing save added

#9 Updated by Etienne Massip over 5 years ago

#10 Updated by Etienne Massip over 5 years ago

#11 Updated by Etienne Massip over 5 years ago

#12 Updated by Craig Rodrigues about 5 years ago

I have a copy of migrate_from_trac.rake in a Github fork, where I am preserving the patches:

https://github.com/rodrigc/redmine/blob/master/lib/tasks/migrate_from_trac.rake

Hopefully my patches can be incorporated into Redmine, so that I don't have to keep my own
private copy. This will benefit others who are trying to migrate to Redmine from Trac.

#13 Updated by Craig Rodrigues about 5 years ago

I used these patches to migrate about 3000 Trac tickets to Redmine. See:

FreeNAS project migration from Trac to Redmine

#14 Updated by Marc Schlaich about 5 years ago

Craig, could you please have a look at #14844? Would be great if this could be fixed in this patch. Shouldn't be too hard but I have no Ruby experience else I would do it on my own.

#15 Updated by Craig Rodrigues about 5 years ago

Marc,

I'll see what I can do, but can't promise anything. My main reasons for fixing the
migrate_from_trac.rake script to do the following:

FreeNAS project migration from Trac to Redmine

However, since I have finished that, I can't devote a lot more time to Remdmine, but I will try if I have time.

I would encourage you to take a whack at learning Ruby and trying to fix this script.
I didn't know Ruby before fixing migrate_from_trac.rake but I learned enough to get going.
The code in Redmine is very clean, so it is easy to pick up.

The code in Redmine which accesses the database is part of the ActiveRecord class in Rails. ActiveRecord
has an API for doing transactions:

http://api.rubyonrails.org/classes/ActiveRecord/Transactions.html

Give it a shot and see if you can get something working, and I can review it.

#16 Updated by Toshi MARUYAMA about 5 years ago

  • Related to Defect #14844: migrate_from_trac is not run in database transaction added

#17 Updated by Toshi MARUYAMA about 5 years ago

  • Related to Defect #6868: migrate_from_trac and trac 0.12 added

#18 Updated by Toshi MARUYAMA about 5 years ago

#19 Updated by Toshi MARUYAMA about 4 years ago

  • Duplicated by Defect #17898: Exception in Migration from Trac to Redmine (incl. solution) added

#20 Updated by Toshi MARUYAMA about 3 years ago

  • Related to Defect #20943: migrate_from_trac.rake dont work added

Also available in: Atom PDF