Question : How to fix corruption of the innodb tables?

Hello,

MySQL server keeps crushing with InnoDB engine turned on.

For example:

factorys_web.moderatorships                        OK
factorys_web.monitorships                          OK
factorys_web.posts                                 OK
factorys_web.schema_migrations                     OK
factorys_web.topics                                OK
factorys_web.users                                 OK
mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... '

mysql> describe users;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    79
Current database: factorys_web

+-------------------+--------------+------+-----+---------+----------------+
| Field             | Type         | Null | Key | Default | Extra          |
+-------------------+--------------+------+-----+---------+----------------+
| id                | int(11)      | NO   | PRI | NULL    | auto_increment |
| email             | varchar(255) | YES  |     | NULL    |                |
| username          | varchar(255) | YES  |     | NULL    |                |
| crypted_password  | varchar(255) | YES  |     | NULL    |                |
| password_salt     | varchar(255) | YES  |     | NULL    |                |
| persistence_token | varchar(255) | YES  |     | NULL    |                |
| is_admin          | tinyint(1)   | YES  |     | NULL    |                |
| created_at        | datetime     | YES  |     | NULL    |                |
| updated_at        | datetime     | YES  |     | NULL    |                |
| posts_count       | int(11)      | YES  |     | 0       |                |
| last_seen_at      | datetime     | YES  |     | NULL    |                |
+-------------------+--------------+------+-----+---------+----------------+
11 rows in set (0.00 sec)

Also I can dump this database without any problem.

Error log:

100628 14:50:24  InnoDB: Assertion failure in thread 1096825152 in file dict/dict0load.c line 383
InnoDB: Failing assertion: ut_dulint_cmp(table->id, mach_read_from_8(field)) == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100628 14:50:24 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=536870912
read_buffer_size=4194304
max_used_connections=5
max_threads=300
threads_connected=3
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5442541 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x2aaadc00e890
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x41602f10 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x85365e]
/usr/sbin/mysqld(handle_segfault+0x322)[0x5c4cc2]
/lib64/libpthread.so.0[0x356ac0eb10]
/lib64/libc.so.6(gsignal+0x35)[0x356a430265]
/lib64/libc.so.6(abort+0x110)[0x356a431d10]
/usr/sbin/mysqld(dict_load_table+0xab6)[0x7652d6]
/usr/sbin/mysqld(dict_table_get+0x129)[0x7612f9]
/usr/sbin/mysqld(_ZN11ha_innobase4openEPKcij+0x123)[0x73d9b3]
/usr/sbin/mysqld(_ZN7handler7ha_openEP8st_tablePKcii+0x3f)[0x69f94f]
/usr/sbin/mysqld(_Z21open_table_from_shareP3THDP14st_table_sharePKcjjjP8st_tableb+0x54f)[0x61b6af]
/usr/sbin/mysqld[0x61080c]
/usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP11st_mem_rootPbj+0x66a)[0x61223a]
/usr/sbin/mysqld(_Z11open_tablesP3THDPP10TABLE_LISTPjj+0x377)[0x612d47]
/usr/sbin/mysqld(_Z28open_and_lock_tables_derivedP3THDP10TABLE_LISTb+0x62)[0x613122]
/usr/sbin/mysqld[0x6ba73a]
/usr/sbin/mysqld(_Z17mysql_check_tableP3THDP10TABLE_LISTP15st_ha_check_opt+0x5c)[0x6bb68c]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2402)[0x5d4122]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x1fa)[0x5d77fa]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x100b)[0x5d881b]
/usr/sbin/mysqld(_Z10do_commandP3THD+0xe4)[0x5d8de4]
/usr/sbin/mysqld(handle_one_connection+0x6f0)[0x5cbe40]
/lib64/libpthread.so.0[0x356ac0673d]
/lib64/libc.so.6(clone+0x6d)[0x356a4d3d1d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x6368fa0 = CHECK TABLE `poll_options`
thd->thread_id=5704
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100628 14:50:24 mysqld_safe Number of processes running now: 0
100628 14:50:24 mysqld_safe mysqld restarted
100628 14:50:24 [Warning] option 'table_cache': unsigned value 33554432 adjusted to 524288
100628 14:50:24 [Warning] 'record_buffer' is deprecated and will be removed in a future release. Please use 'read_buffer_size' instead.
100628 14:50:24 [Warning] '--skip-locking' is deprecated and will be removed in a future release. Please use '--skip-external-locking' instead.
100628 14:50:24 [Warning] '--log-long-format' is deprecated and will be removed in a future release. Please use '--log-short-format' instead.
100628 14:50:24 [Warning] '--log_slow_queries' is deprecated and will be removed in a future release. Please use ''--slow_query_log'/'--slow_query_log_file'' instead.
100628 14:50:24 [Warning] Changed limits: max_open_files: 65535  max_connections: 300  table_cache: 32612
100628 14:50:24 [Note] Plugin 'FEDERATED' is disabled.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100628 14:50:24  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
100628 14:50:24  InnoDB: Started; log sequence number 7 1522142495
100628 14:50:24 [Note] Event Scheduler: Loaded 0 events
100628 14:50:24 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.46-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)

Answer : How to fix corruption of the innodb tables?

Hi

If your database doesn't contain any data's then move the log and ibdata files to some other folder and restart your mysql server. if you have data's then try to dump all the database using the http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html. then reinstall mysql and restore mysql databases.

This might helps