twisted

I have a twisted application which now needs to monitor processes running on several boxes. The way I manually do is 'ssh and ps', now I'd like my twisted application to do. I have 2 options. Use paramiko or leverage the power of twisted.conch I really want to use twisted.conch but my research led me to believe that its primarily intended to create SSHServers and SSHClients. However my requirement is a simple remoteExecute(some_cmd) I was able to figure out how to do this using paramiko but I didnt want to stick paramiko in my twisted app before looking at how to do this using twisted.conch

I have an audio broadcasting server written in Python and based on Twisted. It works fine, but its memory usage is increasing when there are more users on server, but the memory usage never goes down when those users get off line. As you see in following figure: You can see the curve of memory usage goes up where the curve of listeners/radios goes up, but after the peak of listener/radios, the memory usage is still high, never goes down. I have tried following method for solving this problem: Upgrade Twisted from 8.2 to 9.0 Use guppy to dump heapy, but doesn't help at all Switch selector

The strength of Twisted (for python) is its asynchronous framework (I think). I've written an image processing server that takes requests via Perspective Broker. It works great as long as I feed it less than a couple hundred images at a time. However, sometimes it gets spiked with hundreds of images at virtually the same time. Because it tries to process them all concurrently the server crashes. As a solution I'd like to queue up the remote_calls on the server so that it only processes ~100 images at a time. It seems like this might be something that Twisted already does, but I can't seem to

What techniques are people using to utilize multiple processors/cores when running a TwistedWeb server? Is there a recommended way of doing it? My twisted.web based web service is running on Amazon EC2 instances, which often have multiple CPU cores (8, 16), and the type of work that the service is doing benefits from extra processing power, so i would very much like to use that. I understand that it is possible to use haproxy, squid or a web server, configured as a reverse proxy, in front of multiple instances of Twisted. In fact, we are currently using such a setup, with nginx serving as a

I have been reading up on the threaded model of programming versus the asynchronous model from this really good article. http://krondo.com/blog/?p=1209 However, the article mentions the following points. An async program will simply outperform a sync program by switching between tasks whenever there is a I/O. Threads are managed by the operating system. I remember reading that threads are managed by the operating system by moving around TCBs between the Ready-Queue and the Waiting-Queue(amongst other queues). In this case, threads don't waste time on waiting either do they? In light of the

Everything I've read and experienced ( Tornado based apps ) leads me to believe that ePoll is a natural replacement for Select and Poll based networking, especially with Twisted. Which makes me paranoid, its pretty rare for a better technique or methodology not to come with a price. Reading a couple dozen comparisons between epoll and alternatives shows that epoll is clearly the champion for speed and scalability, specifically that it scales in a linear fashion which is fantastic. That said, what about processor and memory utilization, is epoll still the champ? For very small numbers of

第二部分 : 低效的诗歌服务器来启发对 Twisted 机制的理解 这个系列是从 这里 开始的，欢迎你再次来到这里来。现在我们可能要写一些代码。在开始之前，我们都做出一些必要的假设。 关于对你的假设 在展开讨论前，我假设你已经有过用 Python 写同步程序的经历并且至少知道一点有关 Python 的 Sockt 编程的经验。如果你从没有写过 Socket 程序，或许你可以去看看 Socket 模块 的文档，尤其是后面的示例代码。如果你没有用过 Python 的话，那后面的描述对你来说可能比看周易还痛苦。 你所使用的计算机的情况（ 想的真周到，：） ） 我一般是在 Linux 上使用 Twisted ，这个系列的示例代码也是在 Linux 下完成的。首先声明的是我并没有故意让代码失去平台无关性，但我所讲述的一些内容确实可能仅仅适应于 Linux 和其它的类 Unix （比如 MAC OSX 或 FreeBSD ）。 WIndows 是个奇怪诡异的地方（？？为什么这么评价 Windows 呢），如果你想尝试在它上面学习这个系列，抱歉，如果出了问题，我无法提供任何帮助。 并且假设你已经安装了近期版本的 Python 和 Twisted 。我所提供的示例示例代码是基于 Python2.5 和 Twisted8.2.0 。 你可以在单机上运行所有的示例代码，也可以在网络系统上运行它们

抽象地构建客户端 在第四部分中，我们构建了第一个使用 Twisted 的客户端。它确实能很好地工作，但仍有提高的空间。 首先是，这个客户端竟然有创建网络端口并接收端口处的数据这样枯燥的代码。 Twisted 理应为我们实现这些例程性功能，省得我们每次写一个新的程序时都要去自己实现。 Twisted 这样做也将我们从像异步 I/O 操作中包括许多像异常处理这样的细节处理解放出来。更多的细节处理存在于多平台上运行我们的代码中。如果你那个下午有空，可以翻翻 Twisted 的 WIN32 实现源代码，看看里面有多少小针线是来处理跨平台的。 另一问题是与错误处理有关。当运行版本 1 的 Twisted 客户端来从并没有提供服务的端口上下载诗歌时，它就会崩溃。我们是可以修正这个错误，但通过下面我们要介绍 Twisted 的 APIs 来处理这些类型的错误会更简单。 最后，那个客户端也不能复用。如果有另一个模块需要通过我们的客户端下载诗歌呢？人家怎么知道你的诗歌已经下载完毕？我们不能用一个方法简单地将一首诗下载完成后再传给人家，而在之前让人家处于等待状态。这确实是一个问题，但我们不准备在这个部分解决这个问题—在未来的部分中一定会解决这个问题。 我们将会使用一些高层次的 APIs 和接口来解决第一、二个问题。 Twisted 框架是由众多抽象层松散地组合起来的。因此，学习 Twisted

打造可以复用的诗歌下载客户端 我们在实现客户端上已经花了大量的工作。最新版本的（ 2.0 ）客户端使用了 Transports ， Protocols 和 Protocol Factories ，即整个 Twisted 的网络框架。但仍有大的改进空间。 2.0 版本的客户端只能在命令行里下载诗歌。这是因为 PoetryClientFactory 不仅要下载诗歌还要负责在下载完毕后关闭程序。但这对于” PeotryClientFactory“ 的确是一项分外的工作，因为它除了做好生成一个 PoetryProtocol 的实例和收集下载完毕的诗歌的工作外最好什么也别做。 我需要一种方式来将诗歌传给开始时请求它的函数。在同步程序中我们会声明这样的 API: def get_poetry(host, post): """Return a poem from the poetry server at the given host and port.""" 当然了，我们不能这样做。诗歌在没有全部下载完前上面的程序是需要被阻塞的，否则的话，就无法按照上面的描述那样去工作。但是这是一个交互式的程序，因此对于阻塞在 socket 是不会允许的。我们需要一种方式来告诉调用者何时诗歌下载完毕，无需在诗歌传输过程中将其阻塞。这恰好又是 Twisted 要解决的问题。 Twisted 需要告诉我们的代码何时