Using Mongoose virtuals to populate documents

Throughout this series, we’ve created some schemas and models for documents. We’ve also established ways to define relationships between them. We sometimes find ourselves needing many different properties in our documents. They sometimes overlap each other, and we might want to avoid duplicating the data in the database. Also, we might want to avoid two-way referencing that we discussed in the fifth part of this series. The solution to all of the above issues might be virtuals. To understand them, we also inspect getters and setters first.

You can find all of the code from this series in this repository. Feel free to give it a star and share it.

Getters and Setters in Mongoose

The first concept to wrap our heads around is the idea of getters and setters. They allow us to execute custom logic whenever we are getting and setting properties in a document.

Getters

With getters, we can modify the data of a document. Let’s assume that the user has the credit card number that we don’t want to fully present.

1const userSchema = new mongoose.Schema(
2  {
3    email: String,
4    name: String,
5    password: String,
6    creditCardNumber: {
7      type: String,
8      get: (creditCardNumber: string) => {
9        return `****-****-****-${
10          creditCardNumber.substr(creditCardNumber.length - 4)
11        }`;
12      },
13    },
14  },
15  {
16    toJSON: { getters: true },
17  },
18);

In the above example, every time the User document is pulled from the database, we obfuscate the credit card number to protect it.

To make our changes reflect in the response, we need to pass additional options to the  mongoose.Schema constructor. When we respond with the data of a user, Express calls the toJSON function internally. If we want the getters to apply, we need to set  getters: true as above.

Documents also have the toObject method that we can also customize similarly.

A useful use-case for the above might be to make sure that the password of the user never gets sent back in response.

1const userSchema = new mongoose.Schema(
2  {
3    email: String,
4    name: String,
5    password: {
6      type: String,
7      get: (): undefined => undefined,
8    },
9  },
10  {
11    toJSON: {
12      getters: true,
13    },
14  },
15);

By creating such a getter, we avoid leaking sensitive data. We just need to remember that if we need to access the password, we need the Document.prototype.get() function with an additional option.

1const isPasswordMatching = await bcrypt.compare(
2  logInData.password,
3  user.get('password', null, { getters: false }),
4);

Since we rarely need to access the  password property, it is a more fitting approach to hide it by default.

Setters

Using setters, we can modify the data before it is populated in the database. With them, we can inject additional logic and modify the data.

1import createPasswordHash from './createPasswordHash'
2 
3const userSchema = new mongoose.Schema({
4  email: String,
5  name: String,
6  password: {
7    type: String,
8    set: (plaintextPassword: string) => {
9      return createPasswordHash(plaintextPassword);
10    },
11  },
12});

You might prefer to have the above logic inside your services instead of using setters. Even if that’s the case, it is good to be aware of the above functionality.

Mongoose virtuals

A Virtual is an additional property of a document. We can get it and set it, but it does not persist in the MongoDB database.

A typical example might be with names. First, let’s create a  firstName property without the use of virtuals.

1const userSchema = new mongoose.Schema({
2  email: String,
3  firstName: String,
4  lastName: String,
5  fullName: String,
6  password: String,
7});

​The above approach has a few issues. By persisting the  fullName data into MongoDB, we duplicate the information, which is unnecessary. Also, imagine that at some point, we would like to support a middle name and make it a part of the  fullName. A more fitting approach would be to attach the  fullName dynamically.

Getters

We can do the above with the use of virtuals. Let’s create it along with a getter.

1const userSchema = new mongoose.Schema(
2  {
3    email: String,
4    firstName: String,
5    lastName: String,
6    password: String,
7  },
8  {
9    toJSON: { virtuals: true },
10  },
11);
12 
13userSchema.virtual('fullName').get(function () {
14  return `${this.firstName} ${this.lastName}`;
15});

Now, every time we pull the user entity from the database, we also create the  fullName property dynamically.

Similarly to regular getters, we also need to pass additional options to the Document.prototype.get() constructor. Thanks to adding  toJSON: { virtuals: true } above, our virtual properties are visible when converting to JSON.

Now, when we inspect our database, we can see that the  fullName field is not populated:

Setters

Virtuals also support setters. With them, we can set multiple properties at once.

1userSchema.virtual('fullName')
2  .get(function () {
3    return `${this.firstName} ${this.lastName}`;
4  })
5  .set(function (fullName: string) {
6    const [firstName, lastName] = fullName.split(' ');
7    this.set({ firstName, lastName });
8  });

In the above example, we set the  firstName and  lastName properties based on the  fullName. Within the setter function of a Virtual, this refers to the document.

Populating virtuals

One of the most useful features of virtuals is populating documents from another collection.

In the fifth part of this series, we discuss the relationships between documents. The direction of the reference is an essential mention in the above article. Let’s bring one of the examples again:

1const postSchema = new mongoose.Schema({
2  author: {
3    ref: 'User',
4    type: mongoose.Schema.Types.ObjectId,
5  },
6  content: String,
7  title: String,
8});

In the above schema, we keep the reference to users in the Post document. Therefore, the documents look like that:

1{
2  "author": "5e40382687aa217496466bad",
3  "title": "Lorem ipsum",
4  "content": "Dolor sit amet"
5}

Once we settle for one of the above approaches to storing references, we don’t have the information on the other side of the relationship. If we store the reference inside of a post, the user document does not hold the information about posts:

1{
2  "_id": "5e40382687aa217496466bad",
3  "name": "John",
4  "email": "john@smith.com"
5}

We could do it the other way around instead.

1const userSchema = new mongoose.Schema({
2  email: String,
3  name: String,
4  password: String,
5  posts: [
6    {
7      ref: 'Post',
8      type: mongoose.Schema.Types.ObjectId,
9    },
10  ],
11});

Doing so would mean that the posts do not contain information about the authors. We could implement two-way referencing, as described in the mentioned article, but it comes with some disadvantages.

Solving the issue with virtuals

To tackle the above problem, we can implement a virtual property:

1userSchema.virtual('posts', {
2  ref: 'Post',
3  localField: '_id',
4  foreignField: 'author',
5});

In the above code, we connect the  _id of a user to the  author of a post. We also tell Mongoose through ref wich model to populate documents from.

The only thing that’s left is to use the  populate function.

1private initializeRoutes() {
2  this.router.get(`${this.path}/:id`, authMiddleware, this.getUserById);
3}

1private getUserById = async (request: Request, response: Response, next: NextFunction) => {
2  const id = request.params.id;
3  const user = await this.user.findById(id).populate('posts');
4  if (user) {
5    response.send(user);
6  } else {
7    next(new UserNotFoundException(id));
8  }
9}

You might want to implement some additional authorization to prevent every authenticated user to have the access to all the users

An important issue that we should address here is that the  populate function takes some additional time to finish. A way to tackle this issue is to implement an additional query param.

1private getUserById = async (request: Request, response: Response, next: NextFunction) => {
2  const id = request.params.id;
3  const userQuery = this.user.findById(id);
4  if (request.query.withPosts === 'true') {
5    userQuery.populate('posts').exec();
6  }
7  const user = await userQuery;
8  if (user) {
9    response.send(user);
10  } else {
11    next(new UserNotFoundException(id));
12  }
13}

Now we populate the posts array only if it was explicitly asked for through query parameters by calling  users/[id]?withPosts=true.

Summary

In this article, we’ve gone through the idea of virtuals. To fully grasp the concept, we’ve investigated getters and setters first. Thanks to doing so, we’ve managed to improve our code. By populating the virtual properties, we’ve found another solution to a common issue with references. Doing all of the above gives us lots of new possibilities on how to tackle everyday challenges.