MySQL user created temporary table is full

Question

I have a created a temporary table using the memory engine as followed:

CREATE TEMPORARY TABLE IF NOT EXISTS some_text (
    id INT DEFAULT 0,
    string varchar(400) DEFAULT ''
) engine = memory;

When I insert rows into it I run into a #1114 error because the table is full. It explains in the mysql docs that changing the tmp_table_size and the max_heap_table_size don't do anything for increasing the size of user created temp tables, which is what I think I have here. How do I go about making this table larger?

I would love to be able to do this dynamically with a call to SET, but I've tried setting tmp_table_size and the max_heap_table_size and they are both well beyond the amount of data I am expecting in this table. So does anyone know how to resolve this error on this table? Thank you to anyone who helps.

Answer 1

max_heap_table_size is a limit on the size of any table (temporary or otherwise) that uses the MEMORY storage engine. You'll have to increase this config option to store more data in the table.

Answer 2

In MySQL, by default, the temp tables created with the memory engine can quickly grow beyond the 16mb limit of max-heap-table-size and tmp-table-size because more memory is allocated per row than is usually required. In the example you provided, the VARCHAR(400) will allocate memory based upon the maximum string size, not the actual size of your data. Each character may require more than 1 byte for storage. Suppose each row requires 2kb, then it only takes 8k rows to reach the limit.

For many applications, this issue can be addressed by using ROW_FORMAT=DYNAMIC as explained here:

http://www.percona.com/doc/percona-server/5.5/flexibility/improved_memory_engine.html