python-multithreading

“OSError: size mismatch in get!” when retrieving files via SFTP using python

耗尽温柔 提交于 2019-12-08 13:24:52
问题 I wrote a python script to download files from SFTP server using python using multithreading so that it could connect to multiple servers at a time and download files from them parallelly. It works fine for up to 10 connections but if there are 25 connections then this error shows up suppose there are 5000 files of size 130mb(almost) on each server to be downloaded The code often works successfully on subsequent tries or will run successfully for the first few files in the date range and then

Cython3 Using Threading.Thread with args inside the class

强颜欢笑 提交于 2019-12-08 12:48:16
问题 Cython,Support me on this error...I have shown the example code and error generated in my code.I'm using Anaconda3 latest distribution for windows, How to use Thread with args inside the class #cytest.pyx from threading import Thread cdef class t(object): cdef int a,b,nthreads cpdef readonly list retlist def __init__(self,int nthreads,int a=10,int b=10): self.a = a self.b = b self.retlist = [] self.nthreads = nthreads cdef mul(self,int c,int d): cdef int n n = (self.a*c)+(self.b*d) self

How to launch win32 applications in separate threads in Python

北城余情 提交于 2019-12-08 07:37:48
问题 So, I am having this following snippet which attempts to start Microsoft Powerpoint through the win32api: import threading import win32com.client import sys class myDemo(threading.Thread): def __init__(self): threading.Thread.__init__(self) def run(self): try: myObject = win32com.client.Dispatch("Powerpoint.Application") print "OK" except: print "Failed to start Powerpoint!" sys.exit(1) print "Now attempting to shutdown..." try: myObject.quit() except: print "Error" if __name__ == "__main__":

How to implement threading to run two bash shell commands in python?

北城以北 提交于 2019-12-08 06:40:32
问题 I have to record a wav file and at the same time I have to analyze it with sox. I am using fifo type file for this operation. So here I need to start 2 threads at the same time but even if I use the threads I am not able to achieve what I wanna do. Always one executing first and then the other. I want them to be in parallel so that I can do some stuff. #this should be in one thread def test_wav(self): """ analyze the data """ bashCommand = "sox {} -n stat".format(self.__rawfile) while self._

Python Really Thread Safe Dictionary (“RuntimeError: dictionary changed size during iteration” avoidance)

假装没事ソ 提交于 2019-12-08 05:38:29
问题 I've accidentialy got a threaded application due to the use of web.py and pyinotify packages :) The application keeps sorta state in JSON files, which it may change and which can be changed by anyone else. Changed JSON is reloaded back into a dict by inotify watcher thread. The application runs a dozen of threads in quite simple conditions. Well, finally I observe RuntimeError: dictionary changed size during iteration. I realized, I need a lock around each and every dictionary operation. On

Python real time plotting ROS data

假如想象 提交于 2019-12-07 22:24:02
问题 I am trying to plot real time data coming to the computer using python. Data comes in a ROS topic and I use 'rospy' to subscribe to the topic in order to get data. This is the code I wrote import rospy from sensor_msgs.msg import ChannelFloat32 import matplotlib.pyplot as plt N = 200 i = 0 topic = "chatter" x = range(N) lmotor = [0]*N rmotor = [0]*N plt.ion() fig = plt.figure() ax = fig.add_subplot(111) ax.set_xlim([0,N]) ax.set_ylim([-1,1]) line1, = ax.plot(lmotor, 'r-') line2, = ax.plot

Python threading issue, raw_input() blocks thread, runaway thread

主宰稳场 提交于 2019-12-07 20:23:26
问题 I'm having an issue with threading in python, the issue seems to be that when I call a thread subsequently calling raw_input() blocks the thread. Here is the minimal example import threading import time class tread_test(threading.Thread): def __init__(self): self.running = True threading.Thread.__init__(self) # def run(self): self.foo() # def foo(self): print "Spam, Spam, Spam" self.task = threading.Timer(0.5, self.foo) self.task.start() # def stop(self): self.running = False self.task.cancel

Using threading/multiprocessing in python to do multiple calculations at the same time

微笑、不失礼 提交于 2019-12-07 18:45:00
问题 I have a list of numbers. I want to perform some time-consuming operation on each number in the list and make a new list with all the results. Here's a simplified version of what I have: def calcNum(n):#some arbitrary, time-consuming calculation on a number m = n for i in range(5000000): m += i%25 if m > n*n: m /= 2 return m nums = [12,25,76,38,8,2,5] finList = [] for i in nums: return_val = calcNum(i) finList.append(return_val) print(finList) Now, I wanted to take advantage of the multiple

Define writable method in asyncore client makes sending data very slow

血红的双手。 提交于 2019-12-07 12:18:27
问题 I wrote a asynchorous client using python asyncore, and met some problems. I have solved this with the help of this: Asyncore client in thread makes the whole program crash when sending data immediately But now I meet some other problem. My client program: import asyncore, threading, socket class Client(threading.Thread, asyncore.dispatcher): def __init__(self, host, port): threading.Thread.__init__(self) self.daemon = True self._thread_sockets = dict() asyncore.dispatcher.__init__(self, map

ThreadPoolExecutor: how to limit the queue maxsize?

泄露秘密 提交于 2019-12-07 04:08:19
问题 I am using ThreadPoolExecutor class from the concurrent.futures package def some_func(arg): # does some heavy lifting # outputs some results from concurrent.futures import ThreadPoolExecutor with ThreadPoolExecutor(max_workers=1) as executor: for arg in range(10000000): future = executor.submit(some_func, arg) but I need to limit the queue size somehow, as I don't want millions of futures to be created at once, is there a simple way to do it or should I stick to queue.Queue and threading