Project

General

Profile

Actions

Defect #3295

closed

Mysql database getting crashed(Internal Error)

Added by Mannan Mohamed Abubackar over 16 years ago. Updated over 16 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Database
Target version:
-
Start date:
2009-05-05
Due date:
% Done:

0%

Estimated time:
Resolution:
Invalid
Affected version:

Description

Dear All,
First, I started using redmine v 0.8.0 since last 4 months. Most of the time i received Internal error whenever i click any project after login to redmine. Sometimes, If i try click again on the specific project,then, it works fine. But, Its happening every day. Then, i read the redmine forum, Mr. Jean-Philippe Lang suggest to update settings.yml file under redmine directory. So last week, i upgraded with latest redmine v0.8.3 and everything was goes fine yesterday, but all of the sudden I get an internal error everytime whenever i click on any specific projects. On the background, i received INNODB DB error from mysql logs as summarized below,

InnoDB: Error: page n:o stored in the page read in is 2753267632, should be 1106!
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 1106.
InnoDB: You may have to recover from a backup.
090505 9:14:49 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex f691ce33a41b87b0964eb1e0f80d0ab641b7dadef7daa7f1f98c36adbabf1e4d5ee4f48ae4e181cee561dc5b992024398dd6df19363faba7d1ac2e12d442a7
207f4240692729c5afe1e9bb08c944b2a0eb4f354aadb42178ee651f07b29939753f9d5a2ecbd76be96965e16dc76a1
len 16384; hex f691ce33a41b87b0964eb1e0f80d0ab641b7dadef7daa7f1f98c36adbabf1e4d5ee4f48ae4e181cee561dc5b992024398dd6df19363faba7d1ac2e12d442a7
% L 84U - J V A [ y bQ G * M E k PF , 9 k R a ,$ E 9 X f v [~ ?F c: _" w n 7
J\GLi zb \' v ! e : A q {O2R N H d o0Hos0) 1 L t+ s$io | cd 0C fM l J$ 3 % U , KZo wif w &: K
+ &k FR* H 4C ,]<a# $D ) j `? ?Q Gqq = ] : d k F C 3~ T _ % \g& b zv
c 5 $v % p^ U T * ]$ - { $ G: l Z 58AV NM.\ ? 4 z 'UO c .`0 r{6L k . O7 b X c 5 U [
yp0 H R x 1 &3 R$ X ;H H R q ;O * [{$ ? e G | K ls - a t(R
* :1e E i 2R " * S Aro \ 7u
w [Q t 3 J F A D r b o j B ?H o?* * X*R *6 -# 7 :NQ )@< w 2 ' X m ) 1 +Jv*e W$P z :
O<6I E X x%x + S # m j$ Rw z v _ w d R= H }K C \z G oEc wA1wmj U , &e 7 n N a: : | |9 9 \ o { }
p}~5 3 .' W > v ? 4 [ a bo Ie jz7 ? 3 9 M / n# & / G o 9T 2 [ Tw? r|? { N
(~r? ~ fr uD z h> *Q t 7Cw ?P L + =R = J } /u JR +| V L kj ~ x5 | 5m _<
?( 4 oh& } ' 8 { ^ ga v - *I W % # : _ x K = RA sn& ? #u h 0 s( ^ 3HO D 9
$ 4 5 SG w & O TqPJ _( } o& W} u e p{ L n & afr]< oL \ L *|1 f | 3 fa d M 4 sC A R Z 4/ \
^a > w P U Op w G [ OwcS) = (m W I Po p q $# c | 'D <]n <Ln BdHaf C? 1 Gus | ^ S L?" N i v 4} { ysev U@_s > i xn n ?> <=L o iR ] x V 7 g { T y
/8 =Z1O - Y {< - ` r ? / x J . { &tlG2 N Y H \2 UX\a \ K &7 } 6c gF qBV1
f j 2]_ ~ L { 7 < ~ 0 x t Q] ZH O ; ! ) \ q _ nn ) D o t \~; J r p $gz@ ( For 0 < m@, P
6dg, C 3 vh- # fS6l i Yjf; 3% & s ]0& ge * ` M KF f: Kvv774 <- 3 " G 7y z~7 ? a Q .qm al o qO Jt
0 " N tg / O6 1 4 o 'Ow / G; _n&? g l S Sw ^ c 9a U < >zE| _ J ' u i + p
l D | e D N n ] d xi VF ;InnoDB: End of page dump
090505 9:14:49 InnoDB: Page checksum 1874453672, prior-to-4.0.14-form checksum 4171692658
InnoDB: stored checksum 4136750643, prior-to-4.0.14-form stored checksum 1775741609
InnoDB: Page lsn 1102568158 4158302193, low 4 bytes of lsn at page end 1447491018
InnoDB: Page number (if stored to page already) 2753267632,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 4102743265
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 1106.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
InnoDB: Ending processing because of a corrupt database page.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
090505 9:14:49 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
090505 9:14:49 InnoDB: Started; log sequence number 0 73461063
090505 9:14:49 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.67-0ubuntu6-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
InnoDB: Error: page n:o stored in the page read in is 2753267632, should be 1106!
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 1106.
InnoDB: You may have to recover from a backup.
090505 9:14:59 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex f691ce33a41b87b0964eb1e0f80d0ab641b7dadef7daa7f1f98c36adbabf1e4d5ee4f48ae4e181cee561dc5b992024398dd6df19363faba7d1ac2e12d442a7
207f4240692729c5afe1e9bb08c944b2a0eb4f354aadb42178ee651f07b29939753f9d5a2ecbd76be96965e16dc76a10ab803d4f1aef27ac022b2a0e483e36db6aae8a840b6283
51b4de78aa87beacf

Redmine production error logs:
Processing ProjectsController#show (for X.X.X.X at 2009-05-05 10:11:30) [GET]
Session ID: 50089cce93d100d2cc42f1aab8d12c98
Parameters: {"action"=>"show", "id"=>"XYZ-projects", "controller"=>"projects"}

ActiveRecord::StatementInvalid (Mysql::Error: Lost connection to MySQL server during query: SELECT count(DISTINCT `issues`.id) AS count_all, tracker_id AS tracker_id FROM `issues` LEFT OUTER JOIN `projects` ON `projects`.id = `issues`.project_id LEFT OUTER JOIN `issue_statuses` ON `issue_statuses`.id = `issues`.status_id LEFT OUTER JOIN `trackers` ON `trackers`.id = `issues`.tracker_id WHERE ((projects.id = 35 OR projects.parent_id = 35)) AND (projects.status=1) GROUP BY tracker_id ):
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:147:in `log'
/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:302:in `execute'
/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:537:in `select'
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all_without_query_cache'
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:59:in `select_all'
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:80:in `cache_sql'
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:59:in `select_all'
/vendor/rails/activerecord/lib/active_record/calculations.rb:237:in `execute_grouped_calculation'
/vendor/rails/activerecord/lib/active_record/calculations.rb:124:in `calculate'
/vendor/rails/activerecord/lib/active_record/calculations.rb:122:in `catch'
/vendor/rails/activerecord/lib/active_record/calculations.rb:122:in `calculate'
/vendor/rails/activerecord/lib/active_record/calculations.rb:48:in `count'
/app/controllers/projects_controller.rb:100:in `show'
/app/models/issue.rb:262:in `visible_by'
/vendor/rails/activerecord/lib/active_record/base.rb:1857:in `with_scope'
/app/models/issue.rb:261:in `visible_by'
/app/controllers/projects_controller.rb:96:in `show'
/vendor/rails/actionpack/lib/action_controller/base.rb:1166:in `send'
/vendor/rails/actionpack/lib/action_controller/base.rb:1166:in `perform_action_without_filters'
/vendor/rails/actionpack/lib/action_controller/filters.rb:579:in `call_filters'
/vendor/rails/actionpack/lib/action_controller/filters.rb:572:in `perform_action_without_benchmark'
/vendor/rails/actionpack/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
/vendor/rails/actionpack/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/vendor/rails/actionpack/lib/action_controller/rescue.rb:201:in `perform_action_without_caching'
/vendor/rails/actionpack/lib/action_controller/caching/sql_cache.rb:13:in `passenger_orig_perform_action'
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in `cache'
/vendor/rails/activerecord/lib/active_record/query_cache.rb:8:in `cache'
/vendor/rails/actionpack/lib/action_controller/caching/sql_cache.rb:12:in `passenger_orig_perform_action'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/request_handler.rb:53:in `perform_action'
/vendor/rails/actionpack/lib/action_controller/base.rb:529:in `send'
/vendor/rails/actionpack/lib/action_controller/base.rb:529:in `process_without_filters'
/vendor/rails/actionpack/lib/action_controller/filters.rb:568:in `process_without_session_management_support'
/vendor/rails/actionpack/lib/action_controller/session_management.rb:130:in `process'
/vendor/rails/actionpack/lib/action_controller/base.rb:389:in `process'
/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:149:in `handle_request'
/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:107:in `dispatch'
/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:104:in `synchronize'
/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:104:in `dispatch'
/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:120:in `dispatch_cgi'
/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:35:in `dispatch'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/request_handler.rb:38:in `process_request'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_request_handler.rb:165:in `main_loop'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb:321:in `start_request_handler'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb:282:in `handle_spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb:163:in `safe_fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb:161:in `fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb:161:in `safe_fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb:280:in `handle_spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb:163:in `safe_fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb:161:in `fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb:161:in `safe_fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb:279:in `handle_spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:317:in `__send__'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:317:in `main_loop'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:168:in `start_synchronously'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:135:in `start'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:112:in `fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:112:in `start'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb:179:in `start'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb:222:in `spawn_rails_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb:217:in `synchronize'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb:217:in `spawn_rails_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb:126:in `spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb:251:in `handle_spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:317:in `__send__'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:317:in `main_loop'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb:168:in `start_synchronously'
/usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn-server:46

Rendering /home/infra/www/redmine-0.8.3/public/500.html (500 Internal Server Error) ==================================================================================

OS: UBUNTU 8.10
Mem: 4GB
Database: Mysql v 5.0.67
Rails 2.2.2
Apache 2.2.9
Ruby 1.8.7 ==================================================================================

Guys, Your urgent help is highly appreciated to solve this issue. Please don't ignore and consider this issue for me, its very critical now.


Files

Error.JPG (33.8 KB) Error.JPG Internal Error Mannan Mohamed Abubackar, 2009-05-05 09:28
Actions

Also available in: Atom PDF