Twitter-like app using MongoDB

Question

I\'m making an app that uses the classic \"follow\" mechanism (the one used by Twitter and a lot of other apps around the web). I\'m using MongoDB. My system has a differenc

Answer 1

You have two possible ways in which a user can follow another user; either directly, or indirectly through a group, in which case the user directly follows the group. Let's begin with storing these direct relations between users and groups:

{
  _id: "userA",
  followingUsers: [ "userB", "userC" ],
  followingGroups: [ "groupX", "groupY" ]
}

Now, you'll want to be able to quickly find out which users user A is following, either directly or indirectly. To achieve this, you can denormalize the groups that user A is following. Let's say that group X and Y are defined as follows:

{
  _id: "groupX",
  members: [ "userC", "userD" ]
},
{
  _id: "groupY",
  members: [ "userD", "userE" ]
}

Based on these groups, and the direct relations user A has, you can generate subscriptions between users. The origin(s) of a subscription are stored with each subscription. For the example data the subscriptions would look like this:

// abusing exclamation mark to indicate a direct relation
{ ownerId: "userA", userId: "userB", origins: [ "!" ] },
{ ownerId: "userA", userId: "userC", origins: [ "!", "groupX" ] },
{ ownerId: "userA", userId: "userD", origins: [ "groupX", "groupY" ] },
{ ownerId: "userA", userId: "userE", origins: [ "groupY" ] }

You can generate these subscriptions pretty easily, using a map-reduce-finalize call for an individual user. If a group is updated, you only have to re-run the map-reduce for all users that are following the group and the subscriptions will be up-to-date again.

Map-reduce

The following map-reduce functions will generate the subscriptions for a single user.

map = function () {
  ownerId = this._id;

  this.followingUsers.forEach(function (userId) {
    emit({ ownerId: ownerId, userId: userId } , { origins: [ "!" ] });
  });

  this.followingGroups.forEach(function (groupId) {
    group = db.groups.findOne({ _id: groupId });

    group.members.forEach(function (userId) {
      emit({ ownerId: ownerId, userId: userId } , { origins: [ group._id ] });
    });
  });
}

reduce = function (key, values) {
  origins = [];

  values.forEach(function (value) {
    origins = origins.concat(value.origins);
  });

  return { origins: origins };
}

finalize = function (key, value) {
  db.subscriptions.update(key, { $set: { origins: value.origins }}, true);
}

You can then run the map-reduce for a single user, by specifying a query, in this case for userA.

db.users.mapReduce(map, reduce, { finalize: finalize, query: { _id: "userA" }})

A few notes:

• You should delete the previous subscriptions of a user, before running map-reduce for that user.
• If you update a group, you should run map-reduce for all the users that follow the group.

I should note that these map-reduce functions turned out to be more complex than what I had in mind, because MongoDB doesn't support arrays as return values of reduce functions. In theory, the functions could be much simpler, but wouldn't be compatible with MongoDB. However, this more complex solution can be used to map-reduce the entire users collection in a single call, if you ever have to.