How to download images programmatically from Wikimedia Commons without registering for a Bot account?

那年仲夏 提交于 2019-12-03 12:27:55

问题


It seems like the only way to get approval for a Bot account is if it adds to or edits information already on Wikimedia. If you try to download any images, without a bot account, using some of the api libraries out there you get error messages instead of the images. Seems like they block anyone not coming in from a browser? Anyone else have any experience with this? Am I missing something here?


回答1:


Try explaining exactly what you want to do? And what you've tried? What error message did you get? You're not very clear...

What libraries have you tried? If you're not aggressive, there are no restrictions in downloading WM content. I've never heard of any restrictions. Some User-Agents are banned from editing to avoid stupid spamming, but really, I've never heard of downloading restrictions.

If you are trying to scrape a massive amount of images, downloading them through Commons, you're doing it wrong (tm). If you are trying to get a few images, anywhere from 10 to 200, you should be able to write a decent tool in a few lines of code, provided that you are respecting the throttling requirement: when the API tells you to slow down, if you don't do it, sysadmins are likely to kick you out.

If you need a complete image dump, (we're talking of a few TBs) try asking on wikitech-l. We had torrents available when there were less images, now it's more complicated, but still doable.

About bot accounts. How deep have you looked in the system? You need a bot account for fast, unsupervised edits. Bot privileges also open a few facilities such as increased query sizes. But remember: bot account? it's simply an augmented user-account. Have you tried running anything with a classical account?




回答2:


Having just done this myself I feel I should share:

http://www.mediawiki.org/wiki/API:Allimages

This API document does state that you can query the images:

http://en.wikipedia.org/w/api.php?action=query&list=allimages&aiprop=url&format=xml&ailimit=10&aifrom=Albert

with the aiprop=url you are given the url of the image you are looking for.




回答3:


Note that there used to be an issue with using LWP: it's not idealogical, it's practical, agents can create massive load on already stretched servers. There are sensible strategies that agent users can follow to reduce the load - ask on www.mediawiki.org, or en:Village pump - Technical




回答4:


If you need between ten and one million files, using Magnus Manske's tools to recurse categories is a good choice. http://tools.wmflabs.org/magnustools/can_i_haz_files.html produces a list of UNIX commands which you can then just run locally.

An alternative, whose interface is in Germany only but easy enough, is https://tools.wmflabs.org/wikilovesdownloads/




回答5:


Didn't really find the answer I'm looking for .. but this page is interesting:: http://www.makeuseof.com/tag/4-free-tools-for-taking-wikipedia-offline/

Especially #4.. but it seems the page is down.. project dead?



来源:https://stackoverflow.com/questions/1467336/how-to-download-images-programmatically-from-wikimedia-commons-without-registeri

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!