timeit

I was playing around with timeit and noticed that doing a simple list comprehension over a small string took longer than doing the same operation on a list of small single character strings. Any explanation? It's almost 1.35 times as much time. >>> from timeit import timeit >>> timeit("[x for x in 'abc']") 2.0691067844831528 >>> timeit("[x for x in ['a', 'b', 'c']]") 1.5286479570345861 What's happening on a lower level that's causing this? Veedrac TL;DR The actual speed difference is closer to 70% (or more) once a lot of the overhead is removed, for Python 2. Object creation is not at fault.

Im running several machine learning algorithms with sklearn in a for loop and want to see how long each of them takes. The problem is I also need to return a value and DONT want to have to run it more than once because each algorithm takes so long. Is there a way to capture the return value 'clf' using python's timeit module or a similar one with a function like this... def RandomForest(train_input, train_output): clf = ensemble.RandomForestClassifier(n_estimators=10) clf.fit(train_input, train_output) return clf when I call the function like this t = Timer(lambda : RandomForest(trainX,trainy)

I want to capture and plot the results from 5 or so timeit calls with logarithmically increasing sizes of N to show how methodX() scales with input. So far I have tried: output = %timeit -r 10 results = methodX(N) It does not work... Can't find info in the docs either. I feel like you should be able to at least intercept the string that is printed. After that I can parse it to extract my info. Has anyone done this or tried? PS: this is in an ipython notebook if that makes a diff. Iguananaut This duplicate question Capture the result of an IPython magic function has an answer demonstrating that

I noticed that if I iterate over a file that I opened, it is much faster to iterate over it without "read"-ing it. i.e. l = open('file','r') for line in l: pass (or code) is much faster than l = open('file','r') for line in l.read() / l.readlines(): pass (or code) The 2nd loop will take around 1.5x as much time (I used timeit over the exact same file, and the results were 0.442 vs. 0.660), and would give the same result. So - when should I ever use the .read() or .readlines()? Since I always need to iterate over the file I'm reading, and after learning the hard way how painfully slow the .read

For instance, documentation says : Note however that timeit will automatically determine the number of repetitions only when the command-line interface is used. Is there a way to call it from within a Python script and have the number of repetitions be determined automatically, with only the shortest number returned? When you call timeit from the command line like this: python -mtimeit -s'import test' 'test.foo()' The timeit module is called as a script. In particular, the main function is called: if __name__ == "__main__": sys.exit(main()) If you look at the source code , you'll see that the

I need to measure the time certain parts of my program take (not for debugging but as a feature in the output). Accuracy is important because the total time will be a fraction of a second. I was going to use the time module when I came across timeit , which claims to avoid a number of common traps for measuring execution times . Unfortunately it has an awful interface, taking a string as input which it then eval's. So, do I need to use this module to measure time accurately, or will time suffice? And what are the pitfalls it refers to? Thanks According to the Python documentation it has to do

在分布式集群中，如何保证相同请求落到相同的机器上，并且后面的集群机器可以尽可能的均分请求，并且当扩容或down机的情况下能对原有集群影响最小。 round robin算法：是把数据mod后直接映射到真实节点上面，这造成节点个数和数据的紧密关联、后期缺乏灵活扩展。 一致性哈希算法：多增加一层虚拟映射层，数据与虚拟节点映射、虚拟节点与真实节点再映射。 一般都会采用一致性哈希或者hash slot的方法。一致性哈希的ketama算法实现在扩容或down的情况下，需要重新计算节点，这对之前的分配可能会有一些影响。所以可以引入hash slot的方式，即某些hash slot区间对应一台机器，对于扩容或down机情况下，就改变某个hash slot区间就可以了，改动比较小，对之前分配的影响也较小。 虚拟桶是取模和一致性哈希二者的折中办法。 采用固定节点数量，来避免取模的不灵活性。 采用可配置映射节点，来避免一致性哈希的部分影响。 先来看一下hash slot的基本模型： 记录和物理机之间引入了虚拟桶层，记录通过hash函数映射到虚拟桶，记录和虚拟桶是多对一的关系；第二层是虚拟桶和物理机之间的映射，同样也是多对一的关系，即一个物理机对应多个虚拟桶，这个层关系是通过内存表实现的。对照抽象模型章节，key-partition是通过hash函数实现的，partition