google-cloud-datastore

PermanentTaskFailure: 'module' object has no attribute 'Migrate'

浪子不回头ぞ 提交于 2019-12-06 22:03:50
问题 I'm using Nick Johnson's Bulk Update library on google appengine (http://blog.notdot.net/2010/03/Announcing-a-robust-datastore-bulk-update-utility-for-App-Engine). It works wonderfully for other tasks, but for some reason with the following code: from google.appengine.ext import db from myapp.main.models import Story, Comment import bulkupdate class Migrate(bulkupdate.BulkUpdater): DELETE_COMPLETED_JOBS_DELAY = 0 DELETE_FAILED_JOBS = False PUT_BATCH_SIZE = 1 DELETE_BATCH_SIZE = 1 MAX

Why memory leaks occurs when using DataStore API on dev server (not tested in production)?

怎甘沉沦 提交于 2019-12-06 16:13:34
问题 Could guys please help me find what causes memory leak ? It drives me crazy :((( I'm using GAE SDK 1.6.1. I created sample project with single servlet that contains following doGet method protected void doGet(javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response) throws javax.servlet.ServletException, IOException { { DatastoreService datastore = DatastoreServiceFactory.getDatastoreService(); System.out.println(); for (long i = 1; i <= 10000000; ++i) {

Modeling a Friend relationship in GAE

女生的网名这么多〃 提交于 2019-12-06 15:52:26
I am working with GAE and have a user table. Now I want to model friend relationships between users. I am thinking of having 3 properties user1 , user2 and request_status , but the problem is in querying to find a particular user's friends. Since GAE does not allow OR filtering, I have to merge two queries. Clearly that is very inefficient. This is just one problem with my approach. If someone could suggest a solution that lets me keep track of all friend requests as well as the friendship, then that would be very helpful. Denormalization will let you more efficiently find a user's friends

Does GAE Datastore support eager fetching?

孤者浪人 提交于 2019-12-06 13:57:21
Let's say I want to display a list of books and their authors. In traditional database design, I would issue a single query to retrieve rows from the Book table as well as the related Author table, a step known as eager fetching . This is done to avoid the dreaded N+1 select problem : If the Author records were retrieved lazily, my program would have to issue a separate query for each author, possibly as many queries as there are books in the list. Does Google App Engine Datastore provide a similar mechanism, or is the N+1 select problem something that is no longer relevant on this platform? I

Google Datastore Querys return stale data

放肆的年华 提交于 2019-12-06 13:46:05
I`m using Google App Engine and when I remove from the datastore and I list again the results, I recover stale data. What is the problem? Here is the code I use. public void remove(long id) { EntityManager em = EMFService.get().createEntityManager(); try { Todo todo = em.find(Todo.class, id); em.remove(todo); } finally { em.close(); } } public List<Todo> getTodos(String userId) { EntityManager em = EMFService.get().createEntityManager(); Query q = em .createQuery("select t from Todo t where t.author = :userId"); q.setParameter("userId", userId); List<Todo> todos = q.getResultList(); System.out

Datastore list of lists

别等时光非礼了梦想. 提交于 2019-12-06 13:33:44
问题 I need to make a list property that will contain lists, something like: db.ListProperty(list(str)) I know list(str) is not a supported value type so as I imagined I received a "ValueError" exception. Thought maybe there is a creative idea out there of how to overcome this :) Thanks! 回答1: You could use pickle to serialize your list and store it in a BlobProperty field. 回答2: Expanding on Adam's suggestion, you can push the pickling into its own Property class. Here is an example that handles

GAE datastore list property serialization

大城市里の小女人 提交于 2019-12-06 12:49:33
问题 I've watched this video from Google I/O 2009: http://www.youtube.com/watch?v=AgaL6NGpkB8 where Brett shows microblogging example. He describes two datastore schemas: first one: class Message(db.Model): sender = db.StringProperty() body = db.TextProperty() receivers = db.StringListProperty() and second one: class Message(db.Model): author = db.StringProperty() message = db.TextProperty() class MessageIndex(db.Model) receivers = db.StringListProperty() And he says that in first example

Google App Engine singletons (Python)

久未见 提交于 2019-12-06 12:42:52
The standard way of doing singletons in Python is class Singleton(object): _instance = None def __new__(cls, *args, **kwargs): if not cls._instance: cls._instance = super(Singleton, cls).__new__(cls, *args, **kwargs) return cls._instance However, this doesn't work on App Engine, since there are may be many servers and we would get one instance per server. So how would we do it for an app engine entity? Something like: class MySingleton(db.models): def __init__(self): all = MySingleton.all() if all.count() > 0: return all.fetch(1).get() super(MySingleton, self).__init__ (*args, **kwargs) This

ndb Models are not saved in memcache when using MapReduce

蓝咒 提交于 2019-12-06 12:39:14
I've created two MapReduce Pipelines for uploading CSVs files to create Categories and Products in bulk. Each product is gets tied to a Category through a KeyProperty. The Category and Product models are built on ndb.Model, so based on the documentation, I would think they'd be automatically cached in Memcache when retrieved from the Datastore. I've run these scripts on the server to upload 30 categories and, afterward, 3000 products. All the data appears in the Datastore as expected. However, it doesn't seem like the Product upload is using Memcache to get the Categories. When I check the

Custom properties not saved correctly for Expando models in repeated StructuredProperty

余生长醉 提交于 2019-12-06 12:20:22
问题 I am trying to use an Expando model as a repeated StructuredProperty in another model. Namely, I would like to add an indefinite number of Accounts to my User model. As Accounts can have different properties depending on their types ( Accounts are references to social network accounts, and for example Twitter requires more information than Facebook for its OAuth process), I have designed my Account model as an Expando . I've added all basic information in the model definition, but I plan to