Mysql won't start - ibdata1 corrupt? - operating system error number 13 - permissions issue

Question

Server shutdown from power failure.
Mysql will not start now.
Disk is not full. Syslog is below

Oct 11 15:03:31 joe mysqld_safe[24757]: started
Oct 1         


        

Answer 1

If you are using ubuntu or apparmor you should permit this change in apparmor.

Edit /etc/apparmor.d/usr.sbin.mysqld and change /var/lib/mysql with the new DATADIR.

It should work.

Answer 2

In short, (especially on RHEL/CentOS/Fedora) try

getenforce

if it replies with Enforcing you have SELinux up and running. Temporarily deactivate it with setenforce 0 and see if MariaDB starts now! Rather common, especially on RHEL/CentOS/Fedora.

There's more about this further down, as well as in this official article.

In general

There are more things in a UNIX environment that might prevent file access, than just user access rights.

• Security modules like SELinux (see above) or AppArmor (as Dan mentioned) could disallow it
• Access Control Lists (ACL) could be specifically set, for the required files/directories
• Any of the parent folders could be owned by another user, and have no x (="dir access") set for others

Additionally there could be other unexpected factors, like ...

• The mysql datadir being set to a place, where mysql doesn't have permissions (see /etc/my.cnf)
• Mysql could (strangely) be running as a different user, or the file could be simply owned by someone else

Just to mention a view things off the top of my head (feel free to edit/add to this answer btw).

In the case, SELinux is "the problem"

For a permanent solution, you could try to restore the appropriate security context, ...

restorecon -R /var/lib/mysql/

... or just deactivate SELinux (but think about this one a little bit before doing so), by editing the config (typically in /etc/selinux/config) and setting SELINUX=disabled as suggested in following article.

• Here the official help page from mariadb.com: What to do if MariaDB doesn't start
• And here something from redhat.com: MariaDB Changing Database Location

Obviously those are applicable to MySQL just the same way.

Answer 3

I had exactly the same problem on my CentOS box. After moving mysql data directory around I couldn't start the service anymore, even as I had copied the files with the same owner and permissions.

I had a problem with the SELinux security context. If you run your CentOS stock it has good chance to be enabled and won't let do what you want with MySQL. To fix this :

First compare the old dir and new dir using

ls -Z /var/lib/mysql 

and

ls -Z /new/mysql/dir

If you see any difference it's likely to be your problem. To modify this :

chcon -R --type=mysql_db_t /new/mysql/dir

The -R switch is for recursion. If you only need to change one file you can omit it.

Answer 4

please check this:

chown -R mysql:mysql /var/lib/mysql

Answer 5

Error:

101130 14:42:51 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
101130 18:07:58 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
101130 18:07:58  InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
InnoDB: File name ./ibdata1
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.

Solution SeLinux SeLinux security:

[root@localhost ~]# service mysqld restart
Deteniendo mysqld:                                         [  OK  ]
Iniciando mysqld:                                          [  FALLÓ  ]
[root@localhost ~]#  restorecon -R /var/lib/mysql/
[root@localhost ~]# service mysqld restart
Deteniendo mysqld:                                         [  OK  ]
Iniciando mysqld:                                          [  OK  ]
[root@localhost ~]#

Answer 6

The file is not corrupt. You can find out the source of these errors with 'perror'. i.e.

toaster:~ morgo$ perror 13
OS error code  13:  Permission denied

InnoDB has corruption detection (page checksums) and would happily tell you if that were the problem.

Either the directory permissions have changed, or your my.cnf file has been hosed, and it's trying to recreate data files somewhere else.