MongoDB Embedded Documents vs. References

by | MongoDB

Table of Contents

Introduction to MongoDB Document Model

Overview

MongoDB is a document-oriented NoSQL database that uses JSON-like documents to store data. The document model offers a flexible schema design, allowing for both embedded documents and references.

Setup Instructions

To get started with MongoDB, you need to have MongoDB installed. If you haven’t installed it yet, follow these steps:

# For Ubuntu
wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
sudo apt-get update
sudo apt-get install -y mongodb-org

# Start MongoDB
sudo systemctl start mongod

MongoDB Document Model Basics

Documents

Documents are the basic units of data in MongoDB, which are analogous to rows in a relational database. Each document is represented as a BSON (Binary JSON) object.

Example MongoDB Document:

{
    "_id": ObjectId("507f1f77bcf86cd799439011"),
    "name": "Alice",
    "age": 30,
    "email": "alice@example.com"
}

Collections

Collections are groups of documents analogous to tables in relational databases. A single collection can have multiple documents with potentially different fields.

users
---------
| Document 1 |
| Document 2 |
| Document 3 |

Embedded Documents vs. References

Embedded Documents

When the related data is stored within a single document, it is called an embedded document. This approach improves read operations as all related data is in a single document but may increase the document’s size.

Example:

{
    "_id": ObjectId("507f1f77bcf86cd799439012"),
    "name": "Bob",
    "address": {
        "street": "123 Main St",
        "city": "Springfield",
        "state": "IL",
        "zip": "62701"
    }
}

References

References are used to normalize data and store relationships between documents. This decreases document size and redundancy but requires joins, which can be slower for read operations.

Example:

Document in the users collection:

{
    "_id": ObjectId("507f1f77bcf86cd799439013"),
    "name": "Charlie",
    "address_id": ObjectId("507f1f77bcf86cd799439014")
}

Document in the addresses collection:

{
    "_id": ObjectId("507f1f77bcf86cd799439014"),
    "street": "456 Oak St",
    "city": "Capitol City",
    "state": "IL",
    "zip": "62702"
}

Real-life Application

CRUD Operations with Embedded Documents

Creating a document with an embedded field:

db.users.insertOne({
    "name": "David",
    "age": 28,
    "address": {
        "street": "789 Birch St",
        "city": "Smallville",
        "state": "KS",
        "zip": "66002"
    }
});

CRUD Operations with References

Creating documents with references:

// Insert address document first
const addressId = db.addresses.insertOne({
    "street": "987 Pine St",
    "city": "Metropolis",
    "state": "NY",
    "zip": "10001"
}).insertedId;

// Insert user document with reference to address
db.users.insertOne({
    "name": "Eva",
    "age": 32,
    "address_id": addressId
});

Querying referenced documents:

// Find user document
const user = db.users.findOne({ "name": "Eva" });

// Find the associated address
const address = db.addresses.findOne({ "_id": user.address_id });

Conclusion

This introduction covers the basic structure and usage of MongoDB’s document model, especially focusing on the use of embedded documents and references. Understanding these concepts is key to designing efficient and scalable MongoDB applications.

Deep Dive into Embedded Documents in MongoDB

Definition and Use Case

Embedded documents, also referred to as nested documents, are subdocuments nested within a parent document. They store related data in a single document structure, promoting data locality and reducing the number of read operations needed for commonly accessed queries.

Pros and Cons of Embedded Documents

Pros

Atomicity: All changes to a single document are atomic.
Performance: Faster read operations due to fewer fetches.
Data Locality: Related data stored together.

Cons

Document Size: MongoDB has a document size limit of 16MB.
Duplication: Data duplication can occur, leading to potentially inconsistent data.
Scalability: Difficult to scale and manage large embedded documents.

Practical Implementation

Example Schema and Data Insertion

We’ll use an example case of a blogging platform where each blog post can have multiple comments.

Blog Post Schema:

{
  "_id": ObjectId("..."),
  "title": "Introduction to MongoDB",
  "content": "This is a blog post about MongoDB...",
  "author": "John Doe",
  "tags": ["mongodb", "database", "NoSQL"],
  "comments": [
    {
      "user": "Alice",
      "message": "Great post!",
      "date": ISODate("2023-10-07T10:00:00Z")
    },
    {
      "user": "Bob",
      "message": "Very informative.",
      "date": ISODate("2023-10-08T12:30:00Z")
    }
  ]
}

Adding a Blog Post with Embedded Comments:

db.blog_posts.insertOne({
  title: "Introduction to MongoDB",
  content: "This is a blog post about MongoDB...",
  author: "John Doe",
  tags: ["mongodb", "database", "NoSQL"],
  comments: [
    {
      user: "Alice",
      message: "Great post!",
      date: ISODate("2023-10-07T10:00:00Z")
    },
    {
      user: "Bob",
      message: "Very informative.",
      date: ISODate("2023-10-08T12:30:00Z")
    }
  ]
});

Querying Embedded Documents

Find Blog Posts with a Specific Tag:

db.blog_posts.find({ tags: "mongodb" });

Find Blog Posts with Comments by a Specific User:

db.blog_posts.find({ "comments.user": "Alice" });

Project Only the Title and Comments of Blog Posts:

db.blog_posts.find({}, { title: 1, comments: 1 });

Updating Embedded Documents

Add a New Comment to a Specific Blog Post:

db.blog_posts.updateOne(
  { _id: ObjectId("...") },
  {
    $push: {
      comments: {
        user: "Charlie",
        message: "Thanks for the info!",
        date: ISODate("2023-10-10T10:00:00Z")
      }
    }
  }
);

Update a Specific Embedded Comment:

db.blog_posts.updateOne(
  { _id: ObjectId("..."), "comments.user": "Alice" },
  {
    $set: { "comments.$.message": "Updated comment text" }
  }
);

Delete a Specific Embedded Comment:

db.blog_posts.updateOne(
  { _id: ObjectId("...") },
  {
    $pull: { comments: { user: "Charlie" } }
  }
);

Handling Large Complex Documents

When handling large documents, ensure they don’t exceed the 16MB limit. For nested arrays that might grow indefinitely, consider restructuring the database design or using referenced documents instead.

Conclusion

Embedded documents are most suitable when:

Data is accessed and updated together frequently.
The data set is small and confined within MongoDB’s document size limits.

By using embedded documents, you can achieve better performance for read-heavy operations and maintain atomic updates, ensuring data consistency within the embedded document structure.

Understanding References in MongoDB

When modeling relationships in MongoDB, references provide a mechanism to reduce document sizes and maintain data normalization. Let’s explore how to implement this using references.

Scenarios for using References

One-to-Many Relationships: An example is a blog where each author can have multiple posts.
Many-to-Many Relationships: An example is students enrolling in multiple courses and each course having multiple students.

Practical Implementation

Example: One-to-Many (Authors and Posts)

Insert Authors and Posts with References:

// Authors Collection
{
    "_id": ObjectId("Author1"),
    "name": "Jane Doe"
}

// Posts Collection
{
    "_id": ObjectId("Post1"),
    "title": "MongoDB Basics",
    "content": "Introduction to MongoDB",
    "author_id": ObjectId("Author1")
},
{
    "_id": ObjectId("Post2"),
    "title": "Advanced MongoDB",
    "content": "Deep dive into references",
    "author_id": ObjectId("Author1")
}

Retrieve Posts by Author:

db.posts.find({ author_id: ObjectId("Author1") });

Example: Many-to-Many (Students and Courses)

Insert Students and Courses with References:

// Students Collection
{
    "_id": ObjectId("Student1"),
    "name": "John Smith",
    "enrolled_course_ids": [ObjectId("Course1"), ObjectId("Course2")]
}

// Courses Collection
{
    "_id": ObjectId("Course1"),
    "name": "Database Systems",
    "student_ids": [ObjectId("Student1")]
},
{
    "_id": ObjectId("Course2"),
    "name": "Machine Learning",
    "student_ids": [ObjectId("Student1")]
}

Retrieve Courses by Student:

const student = db.students.findOne({ _id: ObjectId("Student1") });
const courses = db.courses.find({ _id: { $in: student.enrolled_course_ids } });

Retrieve Students by Course:

const course = db.courses.findOne({ _id: ObjectId("Course1") });
const students = db.students.find({ _id: { $in: course.student_ids } });

Handling References Efficiently

Indexes: Ensure you create indexes on the fields you frequently query, such as author_id in posts or student_ids in courses.

// Index for Posts
db.posts.createIndex({ author_id: 1 });

// Index for Students
db.students.createIndex({ enrolled_course_ids: 1 });

// Index for Courses
db.courses.createIndex({ student_ids: 1 });

Population: When retrieving documents with references, you might want to retrieve related documents within one query. This can be done through client-side processing or using a third-party library that supports population (like Mongoose in JavaScript).

Conclusion

By using references, you can optimize your MongoDB schema for certain use-cases, making it flexible and efficient in handling large datasets and complex relationships. This guide gives you practical steps to implement and query these relationships effectively.

When to Use Embedded Documents: Use Cases and Examples

Embedded documents in MongoDB provide a powerful way to model one-to-few and one-to-many relationships. When utilized correctly, they can optimize performance and simplify queries. This section outlines practical use cases and examples where embedded documents are highly effective.

1. Single Entity Aggregations

Use Case: An e-commerce application with orders containing multiple items.

Explanation: Each order entity needs to be treated as a single unit, including all its order items. Embedding the items within the order document makes retrieval faster as all the data is fetched within a single read operation.

Example:

{
    "_id": "order123",
    "customer_id": "cust456",
    "order_date": "2023-10-01",
    "items": [
        {
            "item_id": "item789",
            "name": "Laptop",
            "quantity": 1,
            "price": 1200
        },
        {
            "item_id": "item012",
            "name": "Mouse",
            "quantity": 2,
            "price": 20
        }
    ],
    "total_price": 1240
}

2. Embedded One-to-Few Relationships

Use Case: User profile with embedded address information.

Explanation: A user typically has only a few addresses, often just one or two. Embedding the address within the user document simplifies reads and writes, reducing the need for multiple lookups.

Example:

{
    "_id": "user789",
    "username": "john_doe",
    "email": "john@example.com",
    "addresses": [
        {
            "type": "home",
            "line1": "123 Main St",
            "city": "Hometown",
            "state": "TX",
            "postalCode": "12345"
        },
        {
            "type": "work",
            "line1": "456 Work Rd",
            "city": "Bigcity",
            "state": "CA",
            "postalCode": "67890"
        }
    ]
}

3. Hierarchical Data Structures

Use Case: Product categories and subcategories.

Explanation: A hierarchical structure such as product categories where each category can have multiple subcategories can be efficiently modeled with embedded documents.

Example:

{
    "_id": "cat123",
    "name": "Electronics",
    "subcategories": [
        {
            "id": "subcat456",
            "name": "Smartphones",
            "subcategories": [
                {
                    "id": "subsubcat789",
                    "name": "Android Phones"
                },
                {
                    "id": "subsubcat012",
                    "name": "iOS Phones"
                }
            ]
        },
        {
            "id": "subcat789",
            "name": "Laptops"
        }
    ]
}

4. Configuration and Metadata Documents

Use Case: Application settings and configurations.

Explanation: Settings or configurations are usually read together, making embeddings suitable, as it ensures atomic read and write operations.

Example:

{
    "_id": "config123",
    "application": "MyApp",
    "settings": {
        "theme": "dark",
        "language": "en",
        "notifications": {
            "email": true,
            "sms": false
        }
    }
}

Summary

Embedded documents in MongoDB are suited for modeling one-to-few relationships, nested structures, and scenarios requiring atomic updates. For use cases like orders, user profiles, hierarchical categorizations, and application configurations, embedding provides streamlined and efficient data interactions. This approach minimizes the number of read operations and maintains data integrity within a single document.

When to Use References: Use Cases and Examples

To understand when to use references in MongoDB, it’s critical to explore practical scenarios where references are beneficial. This section covers several use cases and provides concrete examples to illustrate the use of references.

Use Case 1: Many-to-Many Relationships

When dealing with many-to-many relationships, references can keep document sizes manageable and minimize redundancy. Consider a blogging platform where authors write multiple articles, and articles can have multiple tags.

Schema Design

Authors Collection

_id: Unique identifier for the author.
name: Name of the author.
  • Articles Collection

    _id: Unique identifier for the article.
    title: Title of the article.
    content: Main content of the article.
    author_id: Reference to the author.
  • Tags Collection

    _id: Unique identifier for the tag.
    name: Name of the tag.
  • ArticleTags Collection

    article_id: Reference to the article.
    tag_id: Reference to the tag.

Example

// Authors Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd672"),
    "name": "Jane Doe"
}

// Articles Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd673"),
    "title": "Understanding MongoDB",
    "content": "This article explores MongoDB...",
    "author_id": ObjectId("605c72dfd4eef5a9dfdbd672")
}

// Tags Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd674"),
    "name": "MongoDB"
}

// ArticleTags Collection
{
    "article_id": ObjectId("605c72dfd4eef5a9dfdbd673"),
    "tag_id": ObjectId("605c72dfd4eef5a9dfdbd674")
}

Use Case 2: Large Subdocuments

When dealing with large subdocuments that don’t need to be loaded every time the parent document is accessed, references can help improve performance by keeping the main document smaller. Consider a user profile that needs to store a lot of activity logs.

Schema Design

Users Collection

_id: Unique identifier for the user.
username: Username of the user.
email: Email of the user.

ActivityLogs Collection

_id: Unique identifier for the log entry.
user_id: Reference to the user.
activity: Description of the activity.
timestamp: Timestamp of the activity.

Example

// Users Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd675"),
    "username": "john_doe",
    "email": "john@example.com"
}

// ActivityLogs Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd676"),
    "user_id": ObjectId("605c72dfd4eef5a9dfdbd675"),
    "activity": "Logged in",
    "timestamp": "2023-10-05T14:48:00Z"
}

Use Case 3: Cross-Collection Retrieval

When data needs are highly interlinked but stored across different collections for logical separation, references facilitate cross-collection retrieval. Consider an e-commerce platform where orders and products are separated.

Schema Design

Orders Collection

_id: Unique identifier for the order.
user_id: Reference to the user who placed the order.
product_ids: Array of references to products.

Products Collection

_id: Unique identifier for the product.
name: Name of the product.
price: Price of the product.

Example

// Orders Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd677"),
    "user_id": ObjectId("605c72dfd4eef5a9dfdbd675"),
    "product_ids": [
        ObjectId("605c72dfd4eef5a9dfdbd678"),
        ObjectId("605c72dfd4eef5a9dfdbd679")
    ]
}

// Products Collection
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd678"),
    "name": "Laptop",
    "price": 999.99
},
{
    "_id": ObjectId("605c72dfd4eef5a9dfdbd679"),
    "name": "Mouse",
    "price": 49.99
}

By understanding these use cases and examining the structure of collections and references, you can effectively decide when to use references to maintain efficient and performant MongoDB databases.

Best Practices and Performance Considerations

Batch Processing for Bulk Inserts

When dealing with large datasets, use batch operations to improve performance and reduce resource consumption.

const bulk = db.collection.initializeUnorderedBulkOp();
for (let i = 0; i < 1000; i++) {
  bulk.insert({ /* document structure */ });
}
bulk.execute();

Indexing for Enhanced Query Performance

Use indexes to improve query performance. Indexes should be created on fields that are frequently queried.

db.collection.createIndex({ "user_id": 1 });

For compound indexes, ensure the order of fields in the index matches the order in queries.

db.collection.createIndex({ "username": 1, "email": 1 });

Shard Key Selection in Sharded Clusters

Choose a shard key that has high cardinality and evenly distributes the data across shards.

sh.shardCollection("database.collection", { "_id": "hashed" });

Use of Projection to Limit Document Fields

When querying large documents, use projection to return only necessary fields.

db.collection.find({ "username": "johndoe" }, { "email": 1, "username": 1 });

Handling Large Arrays in Documents

If arrays grow unbounded, consider refactoring to use references instead of embedded documents to maintain performance and manageability.

Using Embedded Documents

{
  "_id": 1,
  "name": "John Doe",
  "posts": [
    { "title": "First Post", "content": "Content of first post" },
    { "title": "Second Post", "content": "Content of second post" }
  ]
}

Using References

{
  "_id": 1,
  "name": "John Doe",
  "posts": [
    ObjectId("600dcf09eda6c6744401d30c"),
    ObjectId("600dcf09eda6c6744401d30d")
  ]
}

Post documents:

{
  "_id": ObjectId("600dcf09eda6c6744401d30c"),
  "title": "First Post",
  "content": "Content of first post",
  "authorId": ObjectId("600dcf09eda6c6744401d30a")
}
{
  "_id": ObjectId("600dcf09eda6c6744401d30d"),
  "title": "Second Post",
  "content": "Content of second post",
  "authorId": ObjectId("600dcf09eda6c6744401d30a")
}

Data Normalization to Reduce Data Duplication

Normalize data to minimize redundancy. Store shared data in separate collections.

Example

Authors Collection

{
  "_id": ObjectId("60af9249e13e4d3f91bDB56e"),
  "name": "John Doe"
}

Books Collection

{
  "_id": ObjectId("60af9249e13e4d3f91bDB56f"),
  "title": "Book Title",
  "authorId": ObjectId("60af9249e13e4d3f91bDB56e")
}

Using $lookup for Aggregation

For joining collections, use the $lookup stage in the MongoDB Aggregation Framework.

db.books.aggregate([
  {
    $lookup: {
      from: "authors",
      localField: "authorId",
      foreignField: "_id",
      as: "author_info"
    }
  }
]);

Limiting Results for Better Performance

When dealing with large result sets, limit the number of returned documents.

db.collection.find({}).limit(100);

Avoiding Frequent Schema Changes

Frequent schema changes can lead to performance overhead. Design your schema considering future expansion.

Profiling and Monitoring

Use MongoDB’s profiling and monitoring tools to identify performance bottlenecks and optimize queries.

db.setProfilingLevel(2); // Enable profiling for all operations
db.system.profile.find({}); // Query the profiling data

Conclusion

These practices and considerations are fundamental to achieving efficient MongoDB operations and ensuring optimal performance. Apply these techniques to manage data effectively, improve query performance, and maintain system scalability.

Related Posts

Mastering MongoDB Indexing Strategies

MongoDB Schema Design and Data Modeling

A comprehensive guide to mastering MongoDB schema design and data modeling. Explore best practices and strategies for creating efficient schemas tailored to your application’s needs.