urlfetch

App Engine: Alternatives to urlfetch? Seems very unreliable

由 情到浓时终转凉″ 提交于 2019-12-01 16:47:25
I'm using urlfetch in my app and while everything works perfectly fine in the development environment, i'm finding urlfetch to be VERY unreliable when it's actually deployed. Sometimes it works as it should (retrieving data), but then a few minutes later it might return nothing, then it'll be working fine again a few minutes after that. This is very unacceptable. I've checked to make sure it's NOT the source URL that's the problem (YQL) and, again, everything works as it should in the development environment. Are there any third-party libraries I could try? Example code: url = "http://query

Google apps script urlfetch encoding URL

由 限于喜欢 提交于 2019-12-01 14:07:15
问题 I would like to use urlfetch to populate a spreadsheet with page data but the URL that I am trying to use is coming back with an error as an invalid argument. I think the problem is that I am using characters in the URL that are being misinterpreted (e.g. quotation marks and parentheses). I have tried to encode the URL with the command below but I am assuming that I am double encoding some of the characters and that is causing a problem. var encodedURL = encodeURIComponent(pageURL) 回答1: Try

Why is my urlFetchApp function failing to successfully login

由 限于喜欢 提交于 2019-12-01 07:30:24
问题 I'm trying to use google apps script to login to an ASP.Net website and scrape some data that I typically have to retrieve manually. I've used Chrome Developer tools to get the correct payload names (TEXT_Username, TEXT_Password, _VIEWSTATE, _VIEWSTATEGENERATOR), I also got a ASP Net session Id to send along with my Post request. When I run my function(s) it returns a Response Code = 200 if followRedirects is set to false and returns Response Code = 302 if followRedirects is set to true.

What happens if an application calls more than 10 asynchronous URL Fetch on Google App Engine?

由 有些话、适合烂在心里 提交于 2019-11-30 12:08:54
Reading the Google App Engine documentation on asynchronous URL Fetch: The app can have up to 10 simultaneous asynchronous URL Fetch calls What happens if an application calls more than 10 async fetch at a time? Does Google App Engine raise an exception or simply queue the remain calls waiting to serve them? Cloudbreak NZ Umm, Swizzec is incorrect. Easy enough to test: rpc = [] for i in range(1,20): rpc.append(urlfetch.createrpc()) urlfetch.make_fetch_call(rpc[-1],"http://stackoverflow.com/questions/3639855/what-happens-if-i-call-more-than-10-asynchronous-url-fetch") for r in rpc: response = r

Python GAE urlfetch credentials

由 ￡可爱￡侵袭症＋ 提交于 2019-11-30 08:39:45
I'd like to know if the Google App Engine URL Fetch Python API does not support accessing URLs that have credentials: http://username:password@www.domain.com/ It should be a basic feature, but I cannot get it to work and could not find any documentation about it being supported or not as well. SOLUTION - here's how it should be done: result = urlfetch.fetch("http://www.domain.com/", headers={"Authorization": "Basic %s" % base64.b64encode("username:password")}) 来源： https://stackoverflow.com/questions/8377298/python-gae-urlfetch-credentials

GAE - AppEngine - DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL:

由 房东的猫 提交于 2019-11-30 07:06:29
问题 I have a Google AppEngine application which runs great on my local machine. The app posts a image (from a url) to my facebook wall. However, when I deploy it to Google's servers, I get an error: DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL: The offending code is: facebook_access_token = facebook_info['access_token'] facebook_post_url = 'https://graph.facebook.com/me/photos?access_token=%s&url=%s&name=%s&method=post' % (facebook_access_token, url, caption)

Google App Engine - Error uploading file to blobstore from Python code

由 筅森魡賤 提交于 2019-11-30 05:33:59
问题 I am working on an app to process emails hitting my mailbox. I have modified my mail settings to forward incoming mails to myapp . The mails reaching myapp will be routed to the handler script (" handle_incoming_email.py ") where it will be processed. My app.yaml file looks like this app.yaml application: myapp version: 1-1 runtime: python27 api_version: 1 threadsafe: false default_expiration: "360d" handlers: - url: /_ah/mail/.+ script: myapp/utils/handle_incoming_email.py The mail handler

What happens if an application calls more than 10 asynchronous URL Fetch on Google App Engine?

由 萝らか妹 提交于 2019-11-29 17:47:00
问题 Reading the Google App Engine documentation on asynchronous URL Fetch: The app can have up to 10 simultaneous asynchronous URL Fetch calls What happens if an application calls more than 10 async fetch at a time? Does Google App Engine raise an exception or simply queue the remain calls waiting to serve them? 回答1: Umm, Swizzec is incorrect. Easy enough to test: rpc = [] for i in range(1,20): rpc.append(urlfetch.createrpc()) urlfetch.make_fetch_call(rpc[-1],"http://stackoverflow.com/questions

Using Google Apps Script to scrape Dynamic Web Pages

由 亡梦爱人 提交于 2019-11-29 17:39:43
I would like to read some data from other websites for a project using Google Script . The pages in questions are Dyanmic ; they contain content that is loaded after the initial page load, via JavaScript calls to the server. Usually, with somewhat static content, this works fine but I am new to JavaScript and to Google Apps Script and thus do not know how to get the content if it is loaded asynchronously via JavaScript (e.g. via AJAX). An example can be found here showing the last tracks played at a radio station. However, these tracks are loaded using JavaScript and instead of the table

Google Apps Script UrlFetchApp returning Unauthorized Error 401

由 女生的网名这么多〃 提交于 2019-11-29 17:15:42
This is the code I'm using. function doc_to_html(id) { var url = "https://docs.google.com/feeds/download/documents/export/Export?id="+id+"&exportFormat=html"; var param = { method : "get", headers : {"Authorization": "Bearer " + ScriptApp.getOAuthToken()}, muteHttpExceptions:true, }; var html = UrlFetchApp.fetch(url,param).getContentText(); Logger.log(html); } It worked several times for testing, but now it's returning "Unauthorized Error 401." More accurately, it is returning: <html> <head> <TITLE>Unauthorized</TITLE> </HEAD> <BODY BGCOLOR="#FFFFFF" TEXT="#000000"> <H1>Unauthorized</H1> <H2
订阅 urlfetch