cloud

Closed. This question is off-topic. It is not currently accepting answers. Learn more . Want to improve this question? Update the question so it's on-topic for Stack Overflow. I'm looking for examples of large websites that are hosted on Amazon EC2/S3/Cloudfront/etc. I worked at a company where we ran a site that did 500k unique visitors per month on EC2 running 10 instances but that's still relatively small potatoes compared to some larger sites. I know smug mug and foursquare are also hosted on EC2. What other large websites are on EC2? Reddit uses EC2 How large is 'large'? I know that

I have a web crawling python script that takes hours to complete, and is infeasible to run in its entirety on my local machine. Is there a convenient way to deploy this to a simple web server? The script basically downloads webpages into text files. How would this be best accomplished? Thanks! Since you said that performance is a problem and you are doing web-scraping, first thing to try is a Scrapy framework - it is a very fast and easy to use web-scraping framework. scrapyd tool would allow you to distribute the crawling - you can have multiple scrapyd services running on different servers

I'm looking for tools that implement the distributed agent/service model ... I guess we could also call this grid or cloud computing, but I'm not sure the term is exactly analagous. A distributed agent model would be one where developers build agents that are dispatched to a framework and the framework executes the agent somewhere in the distributed environment. Specifically, I'm looking for a framework that support dynamic distribution across the grid/cloud, but replaces the transient agent with a more persistent service. The goal here is to minimize the amount of time and effort it requires

应用名称: Cloud Foundry Console 应用URL地址: http://cf-console.cfapps.io 应用说明及使用场景： Cloud Foundry提供了vmc上传和管理应用的方式,但总是感觉不够直观。而且管理应用时,必须在机器上装上环境才可以。所以开发出一个web版的vmc，可以更加直观方便的管理应用，查看自己的CloudFoundry当前的状态。 使用的技术及软件： 前端使用的框架及js插件： 1.bootstrap 2.modernizr 3.datatable plugin 4.bootstrap-carousel 5.bootstrap-editable 服务器端： spring mvc 存储： 没有使用任何数据库及存储服务 创新点（亮点）： 它是一个基于CloudFoundry的应用； 它用来管理CloudFoundry应用； 它能够在任何可以上网的地方管理你的应用，而不需要安装特定的管理环境； 它能够让你对你的应用基本情况有更加直观的了解，而不仅局限于命令行； 应用截图： 登录页 概览页 应用程序页 应用程序实例信息页 cloudfoundry runtime框架页 命令行模式，可以通过命令行维护cloudfoundry(暂时功能还不全) 后续开发： 修改App信息，例如：实例个数，内存......

I am in the process of building an mobile application (iPhone/Android) and want to store the application data onto Amazon's SimpleDB, because we do not want to host our own server to provide these services. I've been going through all of the documentation and the maximum storage size of element values is 1024 bytes. In my case we need to store 1024 up to 10K of text data. I was hoping to find out how other projects are using SimpleDB when they have larger storage needs like our project. I read that one could store pointers to files that are then stored in S3 (file system). Not sure if that is

I have a project and I'm planning to start the web app as an Azure Web Site and then migrate it to an Azure Cloud Service (also called Hosted Service) if it is needed as a scale strategy. The decision is because I read that Azure Web Sites are more simple and fast to develop with almost no Azure-specific configurations or code. So starting fast and simple is a good starting point for the project. But, is that a good starting point for you? Is migrating an Azure Web Site to an Azure Cloud Service the same as you were migrating a normal ASP.NET Website to an Azure Cloud Service? Would you start

I am not able to access Amazon EC2 instance via ssh as i am behind a firewall. So, i thought of running ssh on port other than 22, like 80 or 443. I tried starting Amazon EC2 instance via Web Management Console with following 'user data': #!/bin/bash -ex perl -pi -e 's/^#?Port 22$/Port 80/' /etc/ssh/sshd_config service sshd restart || service ssh restart The idea being that the above script would execute on instance startup and switch ssh from port 22 to port 80. (Ref: http://alestic.com/2010/12/ec2-ssh-port-80 ) But ssh is still not accessible on port 80. Apparently 'user data' script is not

I'm trying to implement a cluster using Erlang as the glue that holds it all together. I like the idea that it creates a fully connected graph of nodes, but upon reading different articles online, it seems as though this doesn't scale well (having a max of 50 - 100 nodes). Did the developers of OTP impose this limitation on purpose? I do know that you can setup nodes to have explicit connections only as well as have hidden nodes, etc. But, it seems as though the default out-of-the-box setup isn't very scalable. So to the questions: If you had 5 nodes (A, B, C, D, E) that all had explicit