Scrapy: Pass arguments to cmdline.execute()

问题:

I know how to pass arguments when running a scrapy spider from the command line. However, I'm having problems when trying to run it programatically from a script using scrapy's cmdline.execute().

The arguments I need to pass are lists that I previously formatted as strings, just like this:

numbers = "one,two,three,four,five" colors = "red,blue,black,yellow,pink"  cmdline.execute('scrapy crawl myspider -a arg1='+numbers+' -a arg2='+colors) 

and the spider is...

    class MySpider(Spider):          name = "myS"          def __init__(self, arg1, arg2):             super(MySpider, self).__init__()  #Rest of the code 

However, when I run it I get this error:

  Traceback (most recent call last):   File "C:/Users/ME/projects/script.py", line 207, in run     cmdline.execute("scrapy crawl myS -a arg1="+numbers+" -a data="+colors)   File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 123, in execute     cmdname = _pop_command_name(argv)   File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 57, in _pop_command_name     del argv[i] TypeError: 'str' object doesn't support item deletion 

Any ideas?

OS: Windows7; Python version: 2.7.8

回答1:

The execute() function expects a list of arguments, not a string. Try this:

cmdline.execute([     'scrapy', 'crawl', 'myspider',     '-a', 'arg1='+numbers, '-a', 'arg2='+colors]) 

回答2:

Are you missing the .split()? try the following and see what happens.

cmdline.execute("scrapy crawl myspider -a arg1="+numbers+" -a arg2=" + colors + "".split()) 

文章来源: Scrapy: Pass arguments to cmdline.execute()