MYSQL query performs very slow

Question

I have developed a user bulk upload module. There are 2 situations, when I do a bulk upload of 20 000 records when database has zero records. Its taking about 5 hours. But w

Answer 1

When you need to find out why a query takes long, you need to inspect individual parts. As you shown in the question Explain statement can help you very much. Usually the most important columns are:

• select_type - this should always be simple query/subquery. Related subqueries give a lot of troubles. Luckily you don't use any
• possible keys - What keys is this select going to search by
• rows - how many candidate rows are determined by the keys/cache and other techniques. Smaller number is better
• Extra - "using" tells you how exactly are the rows found, this is the most useful information

Query analysis

I would have posted analytics for the 1st and 3rd query but they are both quite simple queries. Here is the breakdown for the query that gives you troubles:

EXPLAIN SELECT cf.*, ctyp.typeName, cfv.id as customId, cfv.customFieldId, 
   cfv.relatedId, cfv.fieldValue, cfv.createdAt 
FROM `CustomField` `cf` 
    INNER JOIN CustomType ctyp on ctyp.id = cf.customTypeId 
    LEFT OUTER JOIN CustomValue cfv on cf.id = cfv.customFieldId 
                and relatedId = 0 
    LEFT JOIN CustomFieldSubArea cfsa on cfsa.customFieldId = cf.id 
WHERE ((relatedTable = 'people' and enabled = '1') 
  AND (onCreate = '1')) 
  AND (cfsa.subarea='peoplebulkinsert') 
ORDER BY cf.sortOrder, cf.label

• INNER JOIN CustomType ctyp on ctyp.id = cf.customTypeId
• LEFT OUTER JOIN CustomValue cfv on cf.id = cfv.customFieldId and relatedId = 0
• LEFT JOIN CustomFieldSubArea cfsa on cfsa.customFieldId = cf.id
• WHERE ((relatedTable = 'people' and enabled = '1') AND (onCreate = '1')) AND (cfsa.subarea='peoplebulkinsert')
• ORDER BY cf.sortOrder, cf.label

Solution

Let me explain above list. Bold columns totally must have an index. Joining tables is expensive operation that otherwise needs to go through all rows of both tables. If you make index on the joinable columns the DB engine will find much faster and better way to do it. This should be common practice for any database

The italic columns are not mandatory to have index, but if you have large amount of rows (20 000 is large amount) you should also have index on the columns that you use for searching, it might not have such impact on the processing speed but is worth the extra bit of time.

So you need to add indicies to theese columns

• CustomType - id
• CustomField - customTypeId, id, relatedTable, enabled, onCreate, sortOrder, label
• CustomValue - customFieldId
• CustomFieldSubArea - customFieldId, subarea

To verify the results try running explain statement again after adding indicies (and possibly few other select/insert/update queries). The extra column should say something like "Using Index" and possible_keys column should list used keys (even two or more per join query).

Side note: You have some typos in your code, you should fix them in case someone else needs to work on your code too: "reqruiteCount" as table column and "fileUplaod" as array index in your refered code.

Answer 2

see to it you can try to reduce the query and check with sql online compiler check the time period then include under the project.