MariaDB or MySQL won't start? How to find the reason and fix it

1. Find the reason

sudo systemctl status mariadb
sudo journalctl -u mariadb -n 50 --no-pager

Look for lines with [ERROR]. The last few, just before Aborting, say why. If you set log_error in the settings, the lines are in that file instead (often /var/log/mysql/error.log).

Debian's MariaDB service restarts the database 5 seconds after it crashes. If the cause is still there, it crashes again, so you may see the same error over and over.

2. A typo in the settings

[ERROR] /usr/sbin/mariadbd: unknown variable 'innodb_bufer_pool_size=256M'
[ERROR] Aborting

One misspelled setting name stops the whole database. The message names the exact setting. Find the file it's in:

sudo grep -Rn "innodb_bufer" /etc/mysql/

Fix the spelling, then start it again: sudo systemctl start mariadb.

Two things we expected to break, but didn't:

• 256MB instead of 256M was accepted, and the size was right.
• A setting without a [mysqld] line above it didn't stop the database. But it was ignored without a word in the database's own log, and every mariadb command afterwards printed error: Found option without preceding group in config file. If a setting has no effect, check that it sits under [mysqld].

To see the values the database will really use:

sudo mariadbd --help --verbose 2>/dev/null | grep -E "^max-connections|^innodb-buffer-pool-size"

3. More memory than the server has

[ERROR] mariadbd: Out of memory (Needed 2515186168 bytes)
[ERROR] InnoDB: Cannot map innodb_buffer_pool_size_max=921600m
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

innodb_buffer_pool_size is the memory the database keeps for your data. Set too high, the database can't start. Set too close to the server's total memory, it starts, but Linux kills it later when memory runs out. On a small server that also runs PHP, a quarter to a half of the memory is a safe start. Check with free -m, and see out-of-memory errors.

"Unknown/unsupported storage engine: InnoDB" sounds scary, but here it only means the memory setting failed. Read the lines above it.

4. The disk is full

This was the worst case in our test. We filled the disk while the database was running and a site was writing to it. The database didn't just refuse the write. It crashed:

[ERROR] [FATAL] InnoDB: Error (Out of disk space) in rollback.
[ERROR] /usr/sbin/mariadbd got signal 6 ;

And it couldn't start again while the disk was full:

[ERROR] InnoDB: preallocating 98304 bytes for file ./t/x.ibd failed with error 28

Error 28 means "no space left on device". Check the disk:

df -h /var/lib/mysql

Free some space, then start the database. In our test, it started normally as soon as there was room again. Good places to look are old logs, old backups and database dumps. See stop logs filling the disk for how to find what's using the space. Don't delete files inside /var/lib/mysql to make room, not even ones with odd names like ib_logfile0 or ibdata1. They are your data.

Then make sure it doesn't happen again: get a warning when the disk is 80% full. See simple monitoring for your own server.

5. Wrong owner on the data folder

[ERROR] mariadbd: Can't create/write to file './ddl_recovery.log' (Errcode: 13 "Permission denied")

The database runs as the user mysql, and must own its data folder. This happens after copying data from another server or restoring a backup as root. Fix the owner:

sudo chown -R mysql:mysql /var/lib/mysql
sudo systemctl start mariadb

6. Port 3306 is already taken

[ERROR] Can't start server: Bind on TCP/IP port. Got error: 98: Address already in use
[ERROR] Do you already have another server running on port: 3306 ?

Another program uses the port. Often it's an old database process that didn't stop, or MySQL and MariaDB both installed. Find out which:

sudo ss -ltnp 'sport = :3306'

If it's an old mariadbd or mysqld, stop it cleanly with sudo systemctl stop mariadb (or mysql), and check with pgrep -a mariadbd before starting again.

Before you change anything: back up

If the database won't start and you're not sure why, copy the data folder first, while it's stopped. Then nothing you try can make it worse:

sudo cp -a /var/lib/mysql /root/mysql-copy-$(date +%F)

Make sure the disk has room for the copy, and never run this while the database is running. See backing up a PHP website and testing the restore for regular backups.

Quick list

• Find the reason: journalctl -u mariadb -n 50, read the [ERROR] lines before Aborting.
• unknown variable: a typo in a setting name. grep -Rn it in /etc/mysql/.
• Out of memory / Cannot map innodb_buffer_pool_size: lower innodb_buffer_pool_size.
• Out of disk space / error 28: free space (never inside /var/lib/mysql), then start.
• Errcode: 13 Permission denied: chown -R mysql:mysql /var/lib/mysql.
• Address already in use: ss -ltnp 'sport = :3306', stop the old process.
• A setting has no effect: check that it's under [mysqld].

Official sources

• MariaDB: option files ([mysqld] groups, where settings live)
• MariaDB: innodb_buffer_pool_size
• MariaDB: error log

Next steps: once it runs again, check your sites with fixing "Error establishing a database connection", and keep the database safe with MySQL and MariaDB security.

ServerNest is being built to watch disk space and the database on each server, and warn you before they cause downtime. Join early access to try it first.

← Back to the security blog