twisted

Running Scrapy on PyPy

陌路散爱 提交于 2019-12-05 06:24:18
问题 Is it possible to run Scrapy on PyPy ? I've looked through the documentation and the github project, but the only place where PyPy is mentioned is that there were some unit tests being executed on PyPy 2 years ago, see PyPy support. There is also Scrapy fails in PyPy long discussion happened 3 years ago without a concrete resolution or a follow-up. From what I understand, the main Scrapy's dependency Twisted is known to work on PyPy. Scrapy also uses lxml for HTML parsing, which has a PyPy

Python : why a method from super class not seen?

霸气de小男生 提交于 2019-12-05 06:13:09
i am trying to implement my own version of a DailyLogFile from twisted.python.logfile import DailyLogFile class NDailyLogFile(DailyLogFile): def __init__(self, name, directory, rotateAfterN = 1, defaultMode=None): DailyLogFile.__init__(self, name, directory, defaultMode) # why do not use super. here? lisibility maybe? # self.rotateAfterN = rotateAfterN def shouldRotate(self): """Rotate when N days have passed since file creation""" delta = datetime.date(*self.toDate()) - datetime.date(*self.toDate(self.createdOn)) return delta > datetime.timedelta(self.rotateAfterN) def __getstate__(self):

How can I use a raw_input with twisted?

允我心安 提交于 2019-12-05 05:57:52
I am aware that raw_input cannot be used in twisted. However here is my desired application. I have an piece of hardware that provides an interactive terminal serial port. I am trying to connect to this port and send commands in an async manner. I need it this way because this is a motor controller that once I issue a command it will "block" and run away (my current code). I need to be able to enter another command such as ESTOP in case of problems or danger. I have read some stuff about twisted.internet.stdio.StandardIO however I have not had much luck.. Any advice / help on this would be

Running scrapy from script not including pipeline

拜拜、爱过 提交于 2019-12-05 05:30:10
I'm running scrapy from a script but all it does is activate the spider. It doesn't go through my item pipeline. I've read http://scrapy.readthedocs.org/en/latest/topics/practices.html but it doesn't say anything about including pipelines. My setup: Scraper/ scrapy.cfg ScrapyScript.py Scraper/ __init__.py items.py pipelines.py settings.py spiders/ __init__.py my_spider.py My script: from twisted.internet import reactor from scrapy.crawler import Crawler from scrapy.settings import Settings from scrapy import log, signals from Scraper.spiders.my_spider import MySpiderSpider spider =

How do I run twisted from the console?

試著忘記壹切 提交于 2019-12-05 04:56:28
I'm using Python 3 with Anaconda on Windows 7. I installed Twisted with conda install twisted , and now I'm trying to run twisted (or twistd ?) from the console, but I get this error 'twisted' is not recognized as an internal or external command, operable program or batch file. which makes me think a directory is missing from the path, as in this question . Anaconda is installed in C:\Anaconda3 , but even in C:\Anaconda3\Lib\site-packages\twisted , there isn't a twisted.py or twistd.py file. Am I doing something wrong, or am I looking in the wrong place for the file(s)? Is this an issue

Python twisted reactor - address already in use

旧时模样 提交于 2019-12-05 02:32:17
I'm following a tutorial http://www.raywenderlich.com/3932/how-to-create-a-socket-based-iphone-app-and-server for creating a sample using socket programming in Mac OS X enviromment. I'm writing using post 80 for reactor.listenTCP(80, factory). When I run the server.py file, getting an error: File "server.py", line 10, in <module> reactor.listenTCP(6, factory) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/posixbase.py", line 436, in listenTCP p.startListening() File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python

Making a python program wait until Twisted deferred returns a value

限于喜欢 提交于 2019-12-05 02:11:15
I have a program that fetches info from other pages and parses them using BeautifulSoup and Twisted's getPage. Later on in the program I print info that the deferred process creates. Currently my program tries to print it before the differed returns the info. How can I make it wait? def twisAmaz(contents): #This parses the page (amazon api xml file) stonesoup = BeautifulStoneSoup(contents) if stonesoup.find("mediumimage") == None: imageurl.append("/images/notfound.png") else: imageurl.append(stonesoup.find("mediumimage").url.contents[0]) usedPdata = stonesoup.find("lowestusedprice") newPdata =

How create a python application with two thread each which has a autobahn application

别来无恙 提交于 2019-12-05 02:07:16
问题 I have not found any solution for my problem. I need to create a python application with two thread, each of which is connected to a WAMP Router using autobahn library. Follow I write the code of my experiment: wampAddress = 'ws://172.17.3.139:8181/ws' wampRealm = 's4t' from threading import Thread from autobahn.twisted.wamp import ApplicationRunner from autobahn.twisted.wamp import ApplicationSession from twisted.internet.defer import inlineCallbacks class AutobahnMRS(ApplicationSession):

Scrapy randomly crashing with celery in django

不羁的心 提交于 2019-12-05 02:04:04
问题 I am running my Scrapy project within Django on a Ubuntu Server. The problem is, Scrapy randomly crash even if Its only one spider running. Below is a snippet of the TraceBack. As a none expert, I have googled _SIGCHLDWaker Scrappy but couldn't comprehend the solutions found for the snippet of below: --- <exception caught here> --- File "/home/b2b/virtualenvs/venv/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 602, in _doReadOrWrite why = selectable.doWrite()

Twisted XmlStream: How to connect to events?

大城市里の小女人 提交于 2019-12-05 00:04:52
I would like to implement a Twisted server that expects XML requests and sends XML responses in return: <request type='type 01'><content>some request content</content></request> <response type='type 01'><content>some response content</content></response> <request type='type 02'><content>other request content</content></request> <response type='type 02'><content>other response content</content></response> I have created a Twisted client & server before that exchanged simple strings and tried to extend that to using XML, but I can't seem to figure out how to set it all up correctly. client.py: #