Is it possible to accelerate (dynamic) LINQ queries using GPU?

后端 未结 5 879
青春惊慌失措
青春惊慌失措 2021-02-07 13:35

I have been searching for some days for solid information on the possibility to accelerate LINQ queries using a GPU.

Technologies I have \"investigated\" so far:

5条回答
  •  青春惊慌失措
    2021-02-07 14:18

    select *
    from table1  -- contains 100k rows
    left join table2 -- contains 1M rows
    on table1.id1=table2.id2 -- this would run for ~100G times 
                             -- unless they are cached on sql side
    where table1.id between 1 and 100000 -- but this optimizes things (depends)
    

    could be turned into

    select id1 from table1 -- 400k bytes if id1 is 32 bit 
    -- no need to order
    

    stored in memory

    select id2 from table2 -- 4Mbytes if id2 is 32 bit
    -- no need to order
    

    stored in memory, both arrays sent to gpu using a kernel(cuda,opencl) like below

    int i=get_global_id(0); // to select an id2, we need a thread id
    int selectedID2=id2[i];
    summary__=-1;
    for(int j=0;j

    On the host side, you can make

     select * from table1 --- query3
    

    and

     select * from table2 --- query4
    

    then use the id list from gpu to select the data

     // x is table1 ' s data
     myList.AsParallel().ForEach(x=>query3.leftjoindata=query4[summary[index]]);
    

    The gpu code shouldn't be slower than 50ms for a gpu with constant memory, global broadcast ability and some thousands of cores.

    If any trigonometric function is used for filtering, the performance would drop fast. Also when left joined tables row count makes it O(m*n) complexity so millions versus millions would be much slower. GPU memory bandwidth is important here.

    Edit: A single operation of gpu.findIdToJoin(table1,table2,"id1","id2") on my hd7870(1280 cores) and R7-240(320 cores) with "products table(64k rows)" and a "categories table(64k rows)" (left join filter) took 48 milliseconds with unoptimized kernel.

    Ado.Net 's "nosql" style linq-join took more than 2000 ms with only 44k products and 4k categories table.

    Edit-2:

    left join with a string search condition gets 50 to 200 x faster on gpu when tables grow to 1000s of rows each having at least hundreds of characters.

提交回复
热议问题