Mgo aggregation: how to reuse model types to query and unmarshal “mixed” results?

问题

Let's say we have 2 collections: "users" and "posts", modeled by the following types:

type User struct {
    ID         string    `bson:"_id"`
    Name       string    `bson:"name"`
    Registered time.Time `bson:"registered"`
}

type Post struct {
    ID      string    `bson:"_id"`
    UserID  string    `bson:"userID"`
    Content string    `bson:"content"`
    Date    time.Time `bson:"date"`
}

These can be used when storing / retrieving individual or even collection of documents, e.g.:

usersColl := sess.DB("").C("users")
postsColl := sess.DB("").C("posts")

// Insert new user:
u := &User{
    ID:         "1",
    Name:       "Bob",
    Registered: time.Now(),
},
err := usersColl.Insert(u)
// Handle err

// Get Posts in the last 10 mintes:
var posts []*Post
err := postsColl.Find(
    bson.M{"date": bson.M{"$gt": time.Now().Add(-10 * time.Minute)}},
).Limit(20).All(&posts)
// Handle err

What if we use aggregation to fetch a mixture of these documents? For example Collection.Pipe():

// Query users with their posts:
pipe := collUsers.Pipe([]bson.M{
    {
        "$lookup": bson.M{
            "from":         "posts",
            "localField":   "_id",
            "foreignField": "userID",
            "as":           "posts",
        },
    },
})

var doc bson.M
it := pipe.Iter()
for it.Next(&doc) {
    fmt.Println(doc)
}
// Handle it.Err()

We query users with their posts in a single query. The result is a mixture of users and posts. How can we reuse our User and Post model types to not have to deal with the result as "raw" documents (of type bson.M)?

回答1:

The query above returns documents that "almost" match User documents, but they also have the posts of each users. So basically the result is a series of User documents with a Post array or slice embedded.

One way would be to add a Posts []*Post field to the User itself, and we would be done:

type User struct {
    ID         string    `bson:"_id"`
    Name       string    `bson:"name"`
    Registered time.Time `bson:"registered"`
    Posts      []*Post   `bson:"posts,omitempty"`
}

While this works, it seems "overkill" to extend User with Posts just for the sake of a single query. If we'd continue down this road, our User type would get bloated with lots of "extra" fields for different queries. Not to mention if we fill the Posts field and save the user, those posts would end up saved inside the User document. Not what we want.

Another way would be to create a UserWithPosts type copying User, and adding a Posts []*Post field. Needless to say this is ugly and inflexible (any changes made to User would have to be reflected to UserWithPosts manually).

With Struct Embedding

Instead of modifying the original User, and instead of creating a new UserWithPosts type from "scratch", we can utilize struct embedding (reusing the existing User and Post types) with a little trick:

type UserWithPosts struct {
    User  `bson:",inline"`
    Posts []*Post `bson:"posts"`
}

Note the bson tag value ",inline". This is documented at bson.Marshal() and bson.Unmarshal() (we'll use it for unmarshaling):

inline     Inline the field, which must be a struct or a map.
           Inlined structs are handled as if its fields were part
           of the outer struct. An inlined map causes keys that do
           not match any other struct field to be inserted in the
           map rather than being discarded as usual.

By using embedding and the ",inline" tag value, the UserWithPosts type itself will be a valid target for unmarshaling User documents, and its Post []*Post field will be a perfect choice for the looked up "posts".

Using it:

var uwp *UserWithPosts
it := pipe.Iter()
for it.Next(&uwp) {
    // Use uwp:
    fmt.Println(uwp)
}
// Handle it.Err()

Or getting all results in one step:

var uwps []*UserWithPosts
err := pipe.All(&uwps)
// Handle error

The type declaration of UserWithPosts may or may not be a local declaration. If you don't need it elsewhere, it can be a local declaration in the function where you execute and process the aggregation query, so it will not bloat your existing types and declarations. If you want to reuse it, you can declare it at package level (exported or unexported), and use it wherever you need it.

Modifying the aggregation

Another option is to use MongoDB's $replaceRoot to "rearrange" the result documents, so a "simple" struct will perfectly cover the documents:

// Query users with their posts:
pipe := collUsers.Pipe([]bson.M{
    {
        "$lookup": bson.M{
            "from":         "posts",
            "localField":   "_id",
            "foreignField": "userID",
            "as":           "posts",
        },
    },
    {
        "$replaceRoot": bson.M{
            "newRoot": bson.M{
                "user":  "$$ROOT",
                "posts": "$posts",
            },
        },
    },
})

With this remapping, the result documents can be modeled like this:

type UserWithPosts struct {
    User  *User   `bson:"user"`
    Posts []*Post `bson:"posts"`
}

Note that while this works, the posts field of all documents will be fetched from the server twice: once as the posts field of the returned documents, and once as the field of user; we don't map / use it but it is present in the result documents. So if this solution is chosen, the user.posts field should be removed e.g. with a $project stage:

pipe := collUsers.Pipe([]bson.M{
    {
        "$lookup": bson.M{
            "from":         "posts",
            "localField":   "_id",
            "foreignField": "userID",
            "as":           "posts",
        },
    },
    {
        "$replaceRoot": bson.M{
            "newRoot": bson.M{
                "user":  "$$ROOT",
                "posts": "$posts",
            },
        },
    },
    {"$project": bson.M{"user.posts": 0}},
})

来源：https://stackoverflow.com/questions/46774424/mgo-aggregation-how-to-reuse-model-types-to-query-and-unmarshal-mixed-results

标签

mongodb

types

aggregation-framework

mgo