google-app-engine

How to update 400,000 GAE datastore entities in parallel?

主宰稳场 提交于 2020-01-02 16:31:53
问题 I have 400,000 entities of a certain type, and I'd like to perform a simple operation on each of them (adding a property). I can't process them serially because it would take forever. I don't want to use the MapReduce library because it is complicated and overwhelming. Basically I'd like to create 100 tasks on the taskqueue, each task taking a segment of ~4,000 entities and performing this operation on each one. Hopefully this wouldn't take more than a few minutes to process all 400k entities

How to update 400,000 GAE datastore entities in parallel?

放肆的年华 提交于 2020-01-02 16:30:27
问题 I have 400,000 entities of a certain type, and I'd like to perform a simple operation on each of them (adding a property). I can't process them serially because it would take forever. I don't want to use the MapReduce library because it is complicated and overwhelming. Basically I'd like to create 100 tasks on the taskqueue, each task taking a segment of ~4,000 entities and performing this operation on each one. Hopefully this wouldn't take more than a few minutes to process all 400k entities

How to get current session with only the SID?

↘锁芯ラ 提交于 2020-01-02 14:39:13
问题 How can I retrieve a session using only its session identifier (SID)? I'm using gae-sessions. Update for response @David Underhill: I've done what you suggested, but in debug mode I got this: Session: SID=None {} | but it has db_key populated with a string. Here is my code: upload.py SID = self.request.get("SID") if not SID: return False from app.models.user import User user = User.get_current_user(SID) user.py def get_current_user(SID = None): if SID: session = Session(sid=SID) else: session

How to get current session with only the SID?

醉酒当歌 提交于 2020-01-02 14:38:04
问题 How can I retrieve a session using only its session identifier (SID)? I'm using gae-sessions. Update for response @David Underhill: I've done what you suggested, but in debug mode I got this: Session: SID=None {} | but it has db_key populated with a string. Here is my code: upload.py SID = self.request.get("SID") if not SID: return False from app.models.user import User user = User.get_current_user(SID) user.py def get_current_user(SID = None): if SID: session = Session(sid=SID) else: session

Caching of Google Cloud Endpoints?

社会主义新天地 提交于 2020-01-02 14:13:09
问题 Will requests to Cloud Endpoints get cached? The official docs are a little light on this matter. The docs read: Cloud Endpoints uses the distributed Extensible Service Proxy to provide low latency and high performance for serving even the most demanding APIs. [...] and can be used with Google App Engine, Google Container Engine, Google Compute Engine or Kubernetes. A 'distributed extensible service proxy' makes me think the Endpoint is distributed to the edge nodes for faster responses, but

GAE: Model loses track of parent->child relationship

[亡魂溺海] 提交于 2020-01-02 13:41:21
问题 I'm having what seems like a very strange problem with an Entity relationship in the google app engine data store. I'm work on a Python/GAE webapp (learning exercise), the full code to which can be found on sourceforge. I have 2 models: Gallery - a search term and (indirectly) a list of photos Photo - information about a photo, plus the gallery it belongs to (collection_index='photos') I have an ingestion process that creates galleries and adds photos to them I have a page that reads a

How to return and use an array of strings from a jQuery ajax call?

大兔子大兔子 提交于 2020-01-02 13:27:06
问题 I'm using Google App Engine (Python) along with jQuery for Ajax calls to the server. I have a page where I want to load up a list of strings in Javascript from an Ajax call to the server. The server method I want to invoke: class BrowseObjects(webapp.RequestHandler): def get(self): ids_to_return = get_ids_to_return() // TODO: How to return these ids to the invoking ajax call? self.response.out.write(ids_to_return) The HTML page where I want to be able to access the returned ids: var strings

Add scala class to DataNucleus enhancer CLASSPATH

末鹿安然 提交于 2020-01-02 13:04:11
问题 I am writing a Google App Engine web app and wish to use Scala on the server side. I'm using Eclipse and the Google App Engine plugin. However, as soon as I add an empty Scala class source file to my project, the DataNucleus enhancer warns: SEVERE: Class "com.my.package.UserAccount" was not found in the CLASSPATH. Please check your specification and your CLASSPATH. I will eventually get round to making the class persistent but I want to get rid of this error first. So far I've added the Scala

Hitting Max number of files and blobs for Google App Engine

亡梦爱人 提交于 2020-01-02 12:40:11
问题 When trying to deploy my application: appcfg.py update --oauth2 . I'm getting: PM Rolling back the update. Error 400: --- begin server output --- Max number of files and blobs is 10000. --- end server output --- How do I solve this? 回答1: You "solve" it by having less files deployed when you deploy your project. You have a few choices. You can use Skipfiles in your app.yaml to exclude some files from being deployed: skip_files: - ^(.*/)?#.*#$ - ^(.*/)?.*~$ - ^(.*/)?.*\.py[co]$ - ^(.*/)?.*/RCS/

How to always use the same PersistenceManager within the same RPC request on Google App Engine

笑着哭i 提交于 2020-01-02 12:09:57
问题 Is there a way to ensure the same PersistenceManager instance is used throughout the different code parts executed in the context of the same RPC request? Having to manually handle out the persistence manager instance from function to function is quite a pain: for example: private void updateItem(ItemModel listItem) throws UserNotLoggedInException { PersistenceManager pm = PMF.get().getPersistenceManager(); if (isItemIsNew(pm, listItem)) { workOnItem(pm, listItem); } workSomeMoreOnItem(pm,