What is a fast pythonic way to deepcopy just data from a python dict or list ?

后端 未结 3 638
陌清茗
陌清茗 2020-12-14 01:51

When we need to copy full data from a dictionary containing primitive data types ( for simplicity, lets ignore presence of datatypes like datetime etc), the most obvious cho

3条回答
  •  予麋鹿
    予麋鹿 (楼主)
    2020-12-14 02:20

    @MSeifert The suggested answer is not accurate

    So far i found ujson.loads(ujson.dumps(my_dict)) to be the fastest option which looks strange (how translating dict to string and then from string to new dict is faster then some pure copy)

    Here is an example of the methods i tried and their running time for small dictionary (the results of course are more clear with larger dictionary):

    x = {'a':1,'b':2,'c':3,'d':4, 'e':{'a':1,'b':2}}
    
    #this function only handle dict of dicts very similar to the suggested solution
    def fast_copy(d):
        output = d.copy()
        for key, value in output.items():
            output[key] = fast_copy(value) if isinstance(value, dict) else value        
        return output
    
    
    
    from copy import deepcopy
    import ujson
    
    
    %timeit deepcopy(x)
    13.5 µs ± 146 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
    
    %timeit fast_copy(x)
    2.57 µs ± 31.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
    
    %timeit ujson.loads(ujson.dumps(x))
    1.67 µs ± 14.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
    

    is there any other C extension that might work better than ujson? it very strange that this is the fastest method to copy large dict.

提交回复
热议问题