Can Scrapy be replaced by pyspider?

Question

I\'ve been using Scrapy web-scraping framework pretty extensively, but, recently I\'ve discovered that there is another framework/system called pyspider, which,

Answer 1

Since I use both scrapy and pyspider, I would like to suggest the following:

If the website is really small / simple, try pyspider first since it has almost everything you need

• Use webui to setup project
• Try the online code editor and view parse result instantly
• View the result easily in browser
• Run/Pause the project
• Setup the expiration date so it can re-process the url

However, if you tried pyspider and found it can't fit your needs, it's time to use scrapy. - migrate on_start to start_request - migrate index_page to parse - migrate detail_age to detail_age - change self.crawl to response.follow

Then you are almost done. Now you can play with scrapy's advanced features like middleware, items, pipline etc.