Too many tables; MySQL can only use 61 tables in a join

Question

What is the best way to export data from multiple tables in MySQL. I\'m basically working with product details. Say a product has 150 attributes of data. How can I export t

Answer 1

You're using an EAV design, and trying to re-construct a single row from a variable number of attributes. This points out one of the many landmines you'll encounter using the EAV design: there's a practical limit on the number of joins you can do in a single SQL query.

Especially in MySQL -- there's a hard limit, as you've found. But even in other RDBMS brands, there's an effective limit because the cost of joins is geometric with respect to the number of tables.

If you use EAV, don't try to re-construct a row in SQL as if you had a conventional database design. Instead, fetch the attributes as rows, sorted by the entity id. Then post-process them in your application code. This does mean that you can't dump the data in one step -- you have to write code to loop over the attribute rows, and reform each row of data before you can output it.

EAV is not a convenient database design. There are many expensive drawbacks to using it, and you've just hit one of them.

---

See http://www.simple-talk.com/opinion/opinion-pieces/bad-carma/ for a great story about how using EAV doomed one business.

And also see http://en.wikipedia.org/wiki/Inner-platform_effect because EAV is an example of this Anti-pattern.

---

I understand the need to support a dynamic set of attributes per product in a catalog. But EAV is going to kill your application. Here's what I do to support dynamic attributes:

• Define a real column in the base table for each attribute that's common to all product types. Product name, price, quantity in stock, etc. Work hard to imagine the canonical product entity so you can include as many attributes as possible in this set.

• Define one more column of type TEXT for all additional attributes of each given product type. Store in this column as Serialized LOB of the attributes, in whatever format suits you: XML, JSON, YAML, your own homemade DSL, etc.

  Treat this as a single column in your SQL queries. Any searching, sorting, or display you need to do based on these attributes requires you to fetch the whole TEXT blob into your application deserialize it, and analyze the attributes using application code.