Using SIMD/AVX/SSE for tree traversal

Question

I am currently researching whether it would be possible to speed up a van Emde Boas (or any tree) tree traversal. Given a single search query as input, already having multip

Answer 1

Based on your code, i've went ahead and benchmarked 3 options: AVX2-powered, nested branching (4 jumps) and a branchless variant. These are the results:

// Performance Table... // All using cache-line size 64byteAligned chunks (van Emde-Boas Layout); loop unrolled per cacheline; // all optimizations turned on. Each Element being 4 byte's. Intel i7 4770k Haswell @3.50GHz

Type        ElementAmount       LoopCount       Avg. Cycles / Query
===================================================================
AVX2        210485750           100000000       610 cycles    
AVX2        21048575            100000000       427 cycles           
AVX2        2104857             100000000       288 cycles 
AVX2        210485              100000000       157 cycles   
AVX2        21048               100000000       95 cycles  
AVX2        2104                100000000       49 cycles    
AVX2        210                 100000000       17 cycles 
AVX2        100                 100000000       16 cycles   


Type        ElementAmount       LoopCount       Avg. Cycles / Query
===================================================================  
Branching   210485750           100000000       819 cycles 
Branching   21048575            100000000       594 cycles 
Branching   2104857             100000000       358 cycles 
Branching   210485              100000000       165 cycles 
Branching   21048               100000000       82 cycles
Branching   2104                100000000       49 cycles 
Branching   210                 100000000       21 cycles 
Branching   100                 100000000       16 cycles   


Type        ElementAmount       LoopCount       Avg. Cycles / Query
=================================================================== 
BranchLESS  210485750           100000000       675 cycles 
BranchLESS  21048575            100000000       602 cycles 
BranchLESS  2104857             100000000       417 cycles
BranchLESS  210485              100000000       273 cycles 
BranchLESS  21048               100000000       130 cycles 
BranchLESS  2104                100000000       72 cycles 
BranchLESS  210                 100000000       27 cycles 
BranchLESS  100                 100000000       18 cycles

So my conclusion looks like: when memory access is kinda optimal, AVX can help with Tree's bigger than 200k Elements. Below that there is hardly any penalty to pay (if you dont use AVX for anything else). It's been worth the night of benchmarking this. Thanks to everybody involved :-)