问题
Is it faster to programmatically join tables or use SQL Join statements when one table is much smaller?
More specifically, how does grabbing a string from a hashmap<int, string> of the smaller table and setting its value to the objects returned from the larger table compare to pre-joining the tables on the database? Does the relative sizes of the two tables make a difference?
Update: To rephrase my question. Does grabbing the subset of the larger table (the 5,000 - 20,000 records I care about) and then programmatically joining the smaller table (which I would cache locally) out perform an SQL join? Does the SQL join apply to the whole table or just the subset of the larger table that would be returned?
SQL Join Statement:
SELECT id, description
FROM values v, descriptions d
WHERE v.descID=d.descID
AND v.something = thingICareAbout;
Individual Statements:
SELECT id, descID
FROM values
WHERE v.something = thingICareAbout;
SELECT descID, description
FROM descriptions d;
Programmatic join:
for (value : values){
value.setDescription(descriptions.get(value.getDescID))
}
Additional Info: There are a total of 800,000,000 records in the larger table that corresponding to 3,000 values in the smaller table. Most searches return between 5,000 - 20,000 results. This is an oracle DB.
回答1:
In general, joining tables like this is the sort of operation that SQL databases are optimized for, so there is a good chance that they're fairly hard to beat on this sort of operation.
The relative size of the two tables might make a difference if you attempt to do the join "manually" as you have to factor in the additional memory consumption to hold the bigger table data in memory while you're doing your processing.
While this example is pretty easy to get right, by doing the join yourself you also lose a built-in data integrity check that the database would give you if you let it do the join.
回答2:
Don't even think it. The database can do things locally at least as fast as you can, and without having to ship all the data over the network.
回答3:
Probably SQL would do the work faster. From my understanding, if you do it in your program then it would have to load the 800,000,000 records from the database into memory for your application, then the 3,000 for the small table, then match each record, discard almost all of them (you're only expecting a couple of thousand results) and display to the user.
If you put indexes on the right columns in oracle (descID in both tables) then it would be able to find the joining records very quickly and just load up the 5,000-20,000 that you're expecting.
That said though the easiest way to find out is to test both out and take numbers!
回答4:
If you do the join in memory, you will need to download 800,000,000 + 3,000 records. If you do the join in the database you will need to download 5,000 - 20,000 results each time. Which sounds faster to you? Hint: If you do 100,000 searches the first option might be faster.
来源：https://stackoverflow.com/questions/14130695/is-it-faster-to-programmatic-join-tables-or-use-sql-join-statements-when-one-tab