Home
Arjen's Journal - MySQL 5.x starts even with fatal InnoDB errors
Moved to http://openquery.com/blog/

Arjen Lentz
Date: 2008-09-25 12:48
Subject: MySQL 5.x starts even with fatal InnoDB errors
Security: Public

This bug (http://bugs.mysql.com/19027) was reported by Peter Zaitsev over two years ago... it may not be critical, but it's nasty and surely it can be fixed. This is the problem.

If InnoDB incurs a fatal error during initialisation, for instance changing logfile size, or from a typo in another parameter in the config, mysqld still starts but InnoDB will be disabled. According to Heikki, a storage engine cannot make mysqld not start at that stage, so the only option would be to chuck an assertion - frankly, I'd prefer that over the current misbehaviour!

So you can have a running server that kinda looks ok, but not have access to some or all of your tables.
In a word: ridiculous!

Post A Comment | 14 Comments | Add to Memories | Tell a Friend | Link



User: (Anonymous)
Date: 2008-09-25 07:26 (UTC)
Subject: MySQL 5.x starts even with fatal InnoDB errors

So it can by a typo in a config file? I had some real hard time once, when all my InnoDB tables become 'corrupted' after some toying with configuration. I had no idea which parameter could be wrong, so I basically had to roll back all changes...

Reply | Thread | Link



Arjen Lentz
User: [info]arjen_lentz
Date: 2008-09-25 11:50 (UTC)
Subject: Re: MySQL 5.x starts even with fatal InnoDB errors

there's no corruption. the InnoDB subsystem just gets disabled.

Reply | Parent | Thread | Link



$tony2001
User: [info]tony2001
Date: 2008-09-25 07:33 (UTC)
Subject: (no subject)

>According to Heikki, a storage engine cannot make mysqld not start at that stage

Makes perfect sense to me.
After all, it's just a storage engine, a plugin. Not starting whole server because of some plugin initialization failure would be completely wrong IMO.
Imagine a Linux kernel refusing to boot because of certain module failed to initialize.

Reply | Thread | Link



User: [info]http://openid.cafuego.net/index.php
Date: 2008-09-25 09:00 (UTC)
Subject: (no subject)

> Imagine a Linux kernel refusing to boot because of certain module failed to initialize.

I think you'll find that if the FS module for the root FS doesn't initialise, the kernel does just that; refuse to boot (or continue booting anyway).

If all your data is in InnoDB and not accessible, having the server chuck a hissy-fit would be rather nice.

Reply | Parent | Thread | Link



User: [info]jamesd
Date: 2008-09-25 11:29 (UTC)
Subject: (no subject)

InnoDB doesn't and shouldn't have the right to prevent all other storage engines and the applications using them from running. It's already very ill-behaved when it kills the whole server if there's a checksum failure.

With MyISAM, Archive, Falcon, Maria and third party engines including InnoDB and PBXT we're no longer in a world where it makes sense to let one storage engine prevent the others from getting on with their jobs. Unless the user specifically asks for that, after deciding what they really have to have as their minimal set of applications. That would be a useful feature request.

Reply | Thread | Link



User: (Anonymous)
Date: 2008-09-25 12:53 (UTC)
Subject: The right to crash

InnoDB should have the configurable (my.cnf option) right to crash the server when it cannot initialize. MySQL has the not so nice behavior of creating MyISAM tables when you asked for InnoDB but InnoDB did not initialize. Besides that, I would not want to find out a few minutes after starting a server and walking away from it that InnoDB wasn't running there -- or any other storage engine that I depended on.

Mark

Reply | Parent | Thread | Link



Arjen Lentz
User: [info]arjen_lentz
Date: 2008-09-25 13:06 (UTC)
Subject: Re: The right to crash

sql_mode=NO_ENGINE_SUBSTITUTION can be used to take care of that "silently use default engine if chosen engine is not available" issue.
And indeed, what you describe is *exactly* the problem; it's nasty!

Reply | Parent | Thread | Link



User: [info]jamesd
Date: 2008-09-25 15:18 (UTC)
Subject: Re: The right to crash

Yes, it can be convenient for those who want that. Servers running just one application, say.

In some cases the way MyISAM will report only one table as damaged and let all of the others continue could also be more unhelpful than just refusing to run.

Reply | Parent | Thread | Link



peter_zaitsev
User: [info]peter_zaitsev
Date: 2008-09-25 21:20 (UTC)
Subject: Here is what I think

1) I would vote for safety by default. There are many systems with poor monitoring etc which can get this thing unnoticed for a while causing more downtime and problems when should be otherwise. So I would like an option to ALLOW starting even if not all storage engines are available not have it default.

2) The other issue is the absolutely misleading error message. Why not to report something like table XYZ uses Innodb storage engine which is DISABLED check Error Log for possible details.

Reply | Thread | Link



User: [info]arabxptyltd
Date: 2008-09-26 03:31 (UTC)
Subject: (no subject)

I've been caught this as well previously.

Setting SQL_MODE only helps with creating new tables.

Perhaps an option for a storage engine is to mark a server as unavailable, having the ability to signal a shutdown of a FATAL error would be clearly a security risk.

Reply | Thread | Link



User: [info]nicklasbw.pip.verisignlabs.com
Date: 2008-10-01 15:50 (UTC)
Subject: bailing out if innodb fails

Well, there's also the option of changing default storage engine to innodb, in which case the server will not start if there's a innodb problem.

Reply | Thread | Link



$tony2001
User: [info]tony2001
Date: 2008-10-15 09:56 (UTC)
Subject: (no subject)

one more thing:
how are you going to disable the faulty storage engine (i.e. uninstall the plugin) if the server doesn't even start?

Reply | Thread | Link



Arjen Lentz
User: [info]arjen_lentz
Date: 2008-10-15 10:05 (UTC)
Subject: hmm...

Interesting and indeed a fair point!

Either way there's a lack of effective error-reporting and/or infrastructure to handle the issue, right?

Reply | Parent | Thread | Link



$tony2001
User: [info]tony2001
Date: 2008-10-15 12:21 (UTC)
Subject: Re: hmm...

Well, this looks clear enough to me:
080905 13:09:41 [ERROR] Can't open shared library '/opt/mysql/lib/mysql/plugin/.so' (errno: 0 undefined symbol: safe_mutex_lock)
080905 13:09:41 [Warning] Couldn't load plugin named '' with soname '.so'

Reply | Parent | Thread | Link



browse
my journal
links
April 2009
High Performance MySQL (2nd ed.)
summary