Defect #5771: Problem when importing git repository - Redmine

Defect #5771

Problem when importing git repository

Added by Pavel Valodzka almost 3 years ago. Updated almost 3 years ago.

Status:

Closed

Start date:

2010-06-28

Priority:

Normal

Due date:

Assignee:

Eric Davis

% Done:

100%

Category:

Database

Target version:

1.0.1

Affected version:

1.0.0-RC

Resolution:

Fixed

Description

When importing git repository I got following error:

ActiveRecord::StatementInvalid (PGError: ERROR:  value too long for type character varying(255)
: INSERT INTO "changes" ("changeset_id", "action", "branch", "path", "revision", "from_path", "from_revision") VALUES(885, E'D', NULL, E'"src/main/resources/...lot of text..."', NULL, NULL, N
ULL) RETURNING "id")

  lib/redmine/scm/adapters/abstract_adapter.rb:300:in `save'
  lib/redmine/scm/adapters/abstract_adapter.rb:299:in `each'
  lib/redmine/scm/adapters/abstract_adapter.rb:299:in `save'
  lib/redmine/scm/adapters/abstract_adapter.rb:289:in `save'
  app/models/repository/git.rb:65:in `fetch_changesets'
  app/models/repository/git.rb:65:in `each'
  app/models/repository/git.rb:65:in `fetch_changesets'
  app/controllers/repositories_controller.rb:70:in `show'

Change in abstract_adapter.rb that fix it:

Change.create(:changeset => changeset,
              :action => file[:action],
              :path => file[:path][0...255])

5771-Allow-arbitrary-length-filenames-in-repositories-5771.patch - Patch in GIT format (51.2 KB) Holger Just, 2010-07-05 23:03

0001-Force-the-default-value-of-path-to-be-set-on-the-Cha.patch (1.41 KB) Holger Just, 2010-07-06 23:01

Related issues

Associated revisions

Revision 3828
Added by Eric Davis almost 3 years ago

Extend changes.path and changes.from_path to support longer paths. #5771

Long paths to SCM files would be truncated or wouldn't insert into some
databases. This extends those fields to support arbitrary length filenames.

Contributed by Holger Just

Revision 3830
Added by Eric Davis almost 3 years ago

Force the default value of path to be set on the Change model class. #5771

This is needed because MySQL does not support default values on text columns
(Error introduced in r3828)

Contributed by Holger Just

Revision 3860
Added by Eric Davis almost 3 years ago

Force-clear the default value for migration from :string to :text. #5846 #5771

This is a special case for MySQL in Win32.

Contributed by Holger Just.

History

#1 Updated by Felix Schäfer almost 3 years ago

Category set to Database
Assignee set to Holger Just

First one for you Holger ;-) Though I suspect this isn't a pg-only thing.

Thanks for the report Pavel!

#2 Updated by Holger Just almost 3 years ago

I looks like the current schema allows only path names (inside a repository) with a maximum length of 255 chars all combined.

The column type in the migration is :string, which translates to a varchar(255) in most databases. So we either live with this or change the column type to :text which would allow arbitrary length paths (which I would prefer).

#3 Updated by Holger Just almost 3 years ago

On Sqlite3, this is not reproducible as it happilly stuffs over 1000 chars long strings into a varchar(255)... Now setting up a local PostgreSQL DB for testing...

@Pavel: Which version of PostgreSQL are you using?

#4 Updated by Pavel Valodzka almost 3 years ago

8.4.2

#5 Updated by Holger Just almost 3 years ago

File 5771-Allow-arbitrary-length-filenames-in-repositories-5771.patch added

I could reproduce the error on Postgres 8.4. The attached patch fixes this behavior by changing the datatype of :path and :from_path from text (aka. varchar(255)) to text. Also included into the patch is an update of the test git repository with a new commit for a really long file.

Eric, please check the patch and commit at will.

#6 Updated by Eric Davis almost 3 years ago

Status changed from New to Closed
Assignee changed from Holger Just to Eric Davis
Target version set to 1.0.0 (RC)
% Done changed from 0 to 100
Resolution set to Fixed

Committed in r3828.

#7 Updated by Eric Davis almost 3 years ago

Status changed from Closed to Reopened

Someone reported a bug with this on MySQL

http://twitter.com/chrismckee/status/17862499316

#8 Updated by Chris McKee almost 3 years ago

Problem is occurring in \apps\redmine\db\migrate\20100705164950_change_changes_path_length_limit.rb
Output on migrate as follows:

C:\BitNami-RedmineStack\apps\redmine>rake db:migrate RAILS_ENV="production" --tr
ace > log.txt
rake aborted!
An error has occurred, all later migrations canceled:

Mysql::Error: BLOB/TEXT column 'path' can't have a default value: ALTER TABLE `c
hanges` CHANGE `path` `path` text DEFAULT '' NOT NULL
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/connection_adapters/abstract_adapter.rb:219:in `log'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/connection_adapters/mysql_adapter.rb:323:in `execute'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/connection_adapters/mysql_adapter.rb:511:in `change_column'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:352:in `send'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:352:in `method_missing'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:328:in `say_with_time'
C:/BitNami-RedmineStack/ruby/lib/ruby/1.8/benchmark.rb:293:in `measure'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:328:in `say_with_time'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:348:in `method_missing'
./db/migrate//20100705164950_change_changes_path_length_limit.rb:3:in `up_withou
t_benchmarks'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:282:in `send'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:282:in `migrate'
C:/BitNami-RedmineStack/ruby/lib/ruby/1.8/benchmark.rb:293:in `measure'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:282:in `migrate'
C:1:in `__send__'
C:1:in `migrate'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:486:in `migrate'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:562:in `call'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:562:in `ddl_transaction'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:485:in `migrate'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:472:in `each'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:472:in `migrate'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:400:in `up'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/activ
e_record/migration.rb:383:in `migrate'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/tasks/databa
ses.rake:116
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:i
n `call'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:i
n `execute'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:i
n `each'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:i
n `execute'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:i
n `invoke_with_call_chain'
C:/BitNami-RedmineStack/ruby/lib/ruby/1.8/monitor.rb:242:in `synchronize'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:i
n `invoke_with_call_chain'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:i
n `invoke'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:
in `invoke_task'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:
in `top_level'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:
in `each'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:
in `top_level'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:
in `standard_exception_handling'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:
in `top_level'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:
in `run'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:
in `standard_exception_handling'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:
in `run'
C:/BitNami-RedmineStack/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31
C:/BitNami-RedmineStack/ruby/bin/rake:19:in `load'
C:/BitNami-RedmineStack/ruby/bin/rake:19

Removing this file allows trunk to migrate successfully.

#9 Updated by Chris McKee almost 3 years ago

Forgot to add; running on Windows Bitnami Redmine stack (Redmine trunk) and MySQL5

#10 Updated by Felix Schäfer almost 3 years ago

It seems TEXT columns can't have a default either: BLOB and TEXT columns cannot have DEFAULT values, though there's no mention of the NOT NULL thing (which might be an incidence of the no default, I'm no db expert in any way).

#11 Updated by Holger Just almost 3 years ago

File 0001-Force-the-default-value-of-path-to-be-set-on-the-Cha.patch added

I created an amendment patch to the first one. It removes the default value from the column and instead sets it in a before_save hook in the model. This should fix the issue.

It is also available on Github.

#12 Updated by Eric Davis almost 3 years ago

Status changed from Reopened to Closed

I've changed the migration in r3830. It's odd because both my desktop and CI server showed that migration working with MySQL, SQLite, and Postgres.

#13 Updated by Kirill Ponomarev almost 3 years ago

I updated Redmine to r3859 and got error when migrate:

(in C:/redmine trunk)
** Invoke db:migrate (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:migrate
==  ChangeChangesPathLengthLimit: migrating ===================================
-- change_column(:changes, :path, :text, {:null=>false})
rake aborted!
An error has occurred, all later migrations canceled:

Mysql::Error: BLOB/TEXT column 'path' can't have a default value: ALTER TABLE `c
hanges` CHANGE `path` `path` text DEFAULT '' NOT NULL
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/connection_a
dapters/abstract_adapter.rb:219:in `log'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/connection_a
dapters/mysql_adapter.rb:323:in `execute'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/connection_a
dapters/mysql_adapter.rb:511:in `change_column'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:352:in `send'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:352:in `method_missing'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:328:in `say_with_time'
c:/ruby/lib/ruby/1.8/benchmark.rb:293:in `measure'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:328:in `say_with_time'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:348:in `method_missing'
./db/migrate//20100705164950_change_changes_path_length_limit.rb:3:in `up_withou
t_benchmarks'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:282:in `send'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:282:in `migrate'
c:/ruby/lib/ruby/1.8/benchmark.rb:293:in `measure'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:282:in `migrate'
c:1:in `__send__'
c:1:in `migrate'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:486:in `migrate'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:562:in `call'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:562:in `ddl_transaction'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:485:in `migrate'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:472:in `each'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:472:in `migrate'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:400:in `up'
c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.5/lib/active_record/migration.rb
:383:in `migrate'
c:/ruby/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/tasks/databases.rake:116
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_c
hain'
c:/ruby/lib/ruby/1.8/monitor.rb:242:in `synchronize'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_c
hain'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exceptio
n_handling'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exceptio
n_handling'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31
c:/ruby/bin/rake:19:in `load'
c:/ruby/bin/rake:19

#14 Updated by Felix Schäfer almost 3 years ago

Kirill Ponomarev wrote:

I updated Redmine to r3859 and got error when migrate:

You probably are on MySQL on a 32bit windows? We are already aware of this, a fix should get pushed soon (you can get the fix here if you are in a hurry).

#15 Updated by Eric Davis almost 3 years ago

Target version changed from 1.0.0 (RC) to 1.0.1
Affected version set to 1.0.0-RC

Updated the migration in r3860 and r3861.

#16 Updated by Peter Lieverdink almost 3 years ago

For the record, this is not a problem on Windows only. MariaDB 5.1 (and thus MySQL 5.1 and newer) on Linux have the same issue.

A quick fix if you're running 1.0.0 from git is:

git cherry-pick a628b0f186cf4d182ce5cee1a497ad42c5246406

Also available in: Atom PDF

Redmine

Issues

Custom queries