MongoDB Best Practices

Production guidance for MongoDB covering embedding vs referencing schema design, compound indexing and index intersection limits, the aggregation pipeline, multi-document transaction costs, and when a document store beats a relational one.

Introduction / Overview

MongoDB remains the most widely deployed document database in 2026, and MongoDB 8.0 narrowed several long-standing gaps with relational engines: range queries on encrypted fields via Queryable Encryption, batched oplog entries for bulk inserts that reduce replication overhead, and lower-latency majority write concern acknowledgment. Its core value proposition is unchanged - a flexible, schema-optional document model that maps naturally onto object-oriented application code and scales horizontally via native sharding without the bolt-on complexity relational engines require for the same.

The central design skill in MongoDB is not "how do I query this" but "how do I shape this document" - the embedding-versus-referencing decision made at schema design time determines query performance, atomicity boundaries, and document growth behavior for the life of the collection. This document covers that decision along with indexing, the aggregation pipeline, and an honest accounting of multi-document transaction costs.

When to use this guidance: Designing a new MongoDB collection schema, optimizing slow queries or aggregations, deciding whether a workflow needs multi-document transactions, or evaluating MongoDB against a relational alternative for a new service.

Core Concepts

• Document: A BSON (binary JSON) object, MongoDB's fundamental unit of storage; a collection is a set of documents that need not share an identical shape, though in practice production schemas converge on a consistent structure per collection.
• Embedding: Nesting related data directly inside a parent document, read and written atomically as a single unit.
• Referencing: Storing a related document's _id and querying it separately (or via $lookup), analogous to a relational foreign key.
• Compound index: An index over multiple fields, whose field order determines which query shapes and sort operations it can serve - the MongoDB equivalent of the SQL left-anchored-prefix rule.
• Aggregation pipeline: A sequence of document-transformation stages ($match, $group, $lookup, $project, ...) executed server-side, MongoDB's primary mechanism for analytical queries and cross-collection joins.
• Multi-document transaction: ACID guarantees spanning multiple documents/collections/shards, available since MongoDB 4.0 (single replica set) and 4.2 (sharded clusters), but carrying meaningfully higher overhead than single-document operations, which are always atomic by default.

Best Practices

1. Decide Embedding vs. Referencing Based on Access Pattern and Growth, Not Convenience

Rationale: Embedding gives you atomic reads and writes of the whole entity in a single round trip, at the cost of document size limits (16 MB per document) and duplicated data if the embedded content is shared across parents. Referencing avoids duplication and unbounded growth, at the cost of requiring a second query or a $lookup for every read.

// Embedding: an order's line items belong exclusively to that order,
// are always read together, and have bounded cardinality.
db.orders.insertOne({
  _id: ObjectId(),
  customerId: ObjectId("..."),
  orderedAt: new Date(),
  items: [
    { sku: "WIDGET-1", quantity: 2, unitPrice: 19.99 },
    { sku: "GADGET-7", quantity: 1, unitPrice: 49.99 }
  ]
});

// Referencing: a customer's orders grow unboundedly over the account lifetime
// and are frequently queried/paginated independently of the customer document.
db.customers.insertOne({ _id: ObjectId("cust1"), email: "a@example.com" });
db.orders.insertOne({ _id: ObjectId(), customerId: ObjectId("cust1"), total: 69.98 });

db.orders.find({ customerId: ObjectId("cust1") })
  .sort({ orderedAt: -1 })
  .limit(20);

Hybrid pattern - extended reference: embed only the small, stable subset of a referenced document's fields actually needed for common reads (e.g., embed { customerId, customerEmail } on the order rather than the full customer document), avoiding a $lookup for the common case while still keeping the full customer record independently referenceable.

2. Design Compound Indexes with the ESR Rule: Equality, Sort, Range

Rationale: Like a SQL B-tree, a MongoDB compound index serves a query fully only when the query's usage of each indexed field is compatible with that field's position. The widely used ESR heuristic (Equality fields first, then Sort fields, then Range fields) produces indexes that support the widest range of query and sort patterns efficiently.

// Query: equality on status, sort by createdAt, range on amount
db.orders.find({ status: "pending", amount: { $gt: 100 } }).sort({ createdAt: -1 });

// ESR-ordered compound index
db.orders.createIndex({ status: 1, createdAt: -1, amount: 1 });

Verifying index usage:

db.orders.find({ status: "pending" }).explain("executionStats");
// Look for `stage: "IXSCAN"` (index used) vs `stage: "COLLSCAN"` (full collection scan),
// and compare `nReturned` against `totalDocsExamined` - a large gap indicates
// the index is not selective enough for this query shape.

3. Understand Index Intersection Limits - Do Not Rely on Combining Single-Field Indexes

Rationale: MongoDB can, in some cases, intersect two single-field indexes to satisfy a compound predicate (ANDSORTED/ANDHASH plan stages), but this is a fallback optimization, not a substitute for a well-designed compound index. Index intersection is typically slower than a single compound index built for the actual query shape, and the planner will not always choose it even when eligible.

Recommended approach: build compound indexes for your top N actual query shapes (identified via the profiler, see below), rather than creating many single-field indexes and hoping the planner intersects them effectively. Reserve single-field indexes for fields queried genuinely independently of any other field.

// Enable the profiler to capture slow operations for real query-shape analysis
db.setProfilingLevel(1, { slowms: 100 });
db.system.profile.find().sort({ ts: -1 }).limit(20);

4. Use the Aggregation Pipeline for Cross-Document Analysis, Not Application-Layer Loops

Rationale: Pulling raw documents into application code to filter, group, and join them wastes network bandwidth and application memory for work the database engine performs far more efficiently server-side, with the added benefit of index-aware early stages.

db.orders.aggregate([
  { $match: { orderedAt: { $gte: ISODate("2026-06-01") } } },      // index-eligible, run first
  { $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customer"
  }},
  { $unwind: "$customer" },
  { $group: {
      _id: "$customer.tier",
      totalRevenue: { $sum: "$total" },
      orderCount: { $sum: 1 }
  }},
  { $sort: { totalRevenue: -1 } }
]);

Why this works well in production: placing $match (and $sort when it can use an index) as early as possible in the pipeline lets MongoDB push those stages down to use indexes before the more expensive $lookup/$group stages run, dramatically reducing the working set those later stages process.

5. Reserve Multi-Document Transactions for Genuine Cross-Document Invariants

Rationale: Every single-document write in MongoDB is already atomic - this covers a large fraction of use cases without any transaction overhead at all, which is precisely why the embedding pattern above is so valuable. Multi-document transactions exist for the remaining cases where an invariant genuinely spans multiple documents or collections, but they carry real costs: additional latency from two-phase locking-like coordination, retry logic requirements on transient errors, and reduced throughput under contention.

const session = client.startSession();
try {
  session.startTransaction({ readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } });

  await accounts.updateOne(
    { _id: fromAccountId, balance: { $gte: amount } },
    { $inc: { balance: -amount } },
    { session }
  );
  await accounts.updateOne(
    { _id: toAccountId },
    { $inc: { balance: amount } },
    { session }
  );

  await session.commitTransaction();
} catch (error) {
  await session.abortTransaction();
  throw error;
} finally {
  await session.endSession();
}

Design-first alternative: before reaching for a transaction, ask whether the schema can be reshaped (typically by embedding the two pieces of state in one document) so the operation becomes a single atomic document write. This is frequently possible and eliminates transaction overhead entirely - the fund-transfer example above is one of the few cases where two independent documents genuinely must change together.

Common Pitfalls & Anti-Patterns

Pitfall: The "Unbounded Array" Anti-Pattern

Problem: Embedding a child collection that grows without bound (e.g., every comment ever made on a popular post, every event in a user's activity log) inside a parent document eventually approaches the 16 MB document size limit, and even well before that limit, causes expensive document-rewrite overhead on every append as the document is relocated on disk.

Recommended approach: reference unbounded collections as separate documents with a foreign-key-style field (postId, userId) and paginate via indexed queries, exactly as the referencing pattern above describes.

Pitfall: Massive Fan-Out $lookup Chains Replacing What Should Be a Redesigned Schema

Problem: Chaining several $lookup stages to reassemble data that is queried together on almost every request is a strong signal the schema should embed that data instead - $lookup does not use indexes as efficiently as a native embedded read and adds pipeline overhead per document.

Recommended approach: revisit the embedding-vs-referencing decision for the specific access pattern causing the fan-out, per Best Practice #1.

Pitfall: No Schema Validation Because "MongoDB Is Schemaless"

Problem: Schema flexibility at the storage layer is frequently mistaken for license to skip validation entirely, leading to inconsistent document shapes that silently break application code expecting specific fields/types.

Recommended approach: use MongoDB's native $jsonSchema validator on collections, in addition to application-layer validation (Pydantic on the Python side), so the database itself enforces the shape contract.

db.createCollection("orders", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["customerId", "total", "orderedAt"],
      properties: {
        total: { bsonType: "decimal", minimum: 0 },
        orderedAt: { bsonType: "date" }
      }
    }
  }
});

Pitfall: Overusing Multi-Document Transactions Where Embedding Would Suffice

Problem: Teams migrating from a relational background reach for transactions reflexively to "be safe," incurring latency and throughput costs that a document redesign would have avoided entirely.

Recommended approach: apply Best Practice #5's design-first check before defaulting to a transaction.

Testing Strategies

• Schema validation tests: assert that documents violating $jsonSchema constraints are rejected, and that the application-layer Pydantic models used with a Python driver stay in sync with the database-level validator.
• Index coverage tests: for each critical query, assert via explain("executionStats") that the plan uses IXSCAN, not COLLSCAN, as a CI regression guard against accidental index removal or query-shape drift.
• Transaction retry tests: simulate a TransientTransactionError (e.g., via a write conflict from a concurrent test transaction) and assert the application's retry logic recovers correctly rather than surfacing the error to the caller.
• Aggregation pipeline unit tests: run pipelines against a seeded test database (via mongodb-memory-server or a containerized instance) and assert exact output shape, since aggregation logic is easy to get subtly wrong (e.g., $unwind on an empty array silently dropping the parent document unless preserveNullAndEmptyArrays is set).
• Document growth tests: for collections using embedding, add a test asserting realistic maximum embedded-array sizes stay well under the 16 MB document limit under production-representative data volumes.

Performance, Security & Scalability Considerations

• Sharding: MongoDB's native sharding distributes collections across shards by a chosen shard key; an poorly chosen shard key (low cardinality, or monotonically increasing like a timestamp) creates hot shards that receive disproportionate write traffic - choose a shard key with high cardinality and even write distribution (e.g., a hashed _id or a compound key incorporating a naturally distributed field).
• Write concern and durability: writeConcern: { w: "majority" } guarantees a write survives a primary failover, at the cost of waiting for replica acknowledgment; MongoDB 8.0 reduced this latency by acknowledging as soon as a majority of replica set members have written the oplog entry, rather than waiting for full application. Never use w: 0 (fire-and-forget) for data that must not silently disappear.
• Queryable Encryption: MongoDB 8.0 added range query support ($lt, $lte, $gt, $gte) on encrypted fields, and 8.2 added equality, range, and (in preview) prefix/suffix/substring queries - evaluate for compliance-sensitive fields (PII, financial data) where field-level encryption with server-side query capability is required, rather than encrypting at the application layer and losing queryability entirely.
• Read scaling: secondary reads (readPreference: "secondaryPreferred") scale read throughput horizontally but introduce replication-lag-bound staleness; use only for workloads tolerant of eventually-consistent reads.
• Connection pooling: MongoDB drivers maintain their own internal connection pool per client instance; size maxPoolSize based on expected concurrent operations per application instance rather than accepting driver defaults unexamined, especially in horizontally scaled deployments with many application instances each opening a pool.

Edge Cases & Advanced Usage

• Time-series collections: MongoDB's dedicated time-series collection type (timeseries: { timeField, metaField }) stores measurements far more compactly than a naive document-per-reading design and is purpose-built for IoT/metrics workloads, similar in spirit to a BRIN-indexed append-only table in Postgres.
• Change streams: subscribing to a collection's change stream (watch()) provides a real-time, resumable feed of inserts/updates/deletes, useful for cache invalidation, search-index synchronization, or feeding a microservices event pipeline without a separate change-data-capture tool.
• Bulk write batching: since MongoDB 8.0, bulk inserts outside a transaction are typically recorded as a single oplog entry rather than one per document, reducing replication overhead and lag for high-volume insert workloads - relevant when designing high-throughput ingestion paths.
• Capped collections: fixed-size collections that overwrite the oldest documents on overflow, useful for bounded logs where retention is naturally FIFO and unbounded growth (the anti-pattern above) must be prevented at the storage layer itself.
• Read-your-own-writes consistency: causal consistency sessions (session.advanced().options() with causalConsistency: true) guarantee a client reads its own prior writes even against a secondary, without requiring a full multi-document transaction.

When Not to Use / Alternatives

Choose PostgreSQL (best practices) when the domain is naturally relational (many-to-many relationships, strict referential integrity, complex ad-hoc reporting queries) - see General SQL Patterns for the normalization principles that signal a relational fit.

Choose Redis (best practices) when the requirement is sub-millisecond access to a bounded, mostly-ephemeral working set rather than a durable system of record.

Choose MongoDB when the application's natural unit of work is a self-contained, variably shaped document (user profiles, product catalogs with heterogeneous attributes, content management), and horizontal write scaling is a first-class requirement from day one.

Related Documentation

• General SQL Design Patterns - normalization principles as a contrast point for the embedding-vs-referencing decision.
• PostgreSQL Best Practices - JSONB as Postgres's own document-modeling option.
• Redis Best Practices - caching layer often paired with MongoDB for hot-path acceleration.
• FastAPI Best Practices - using Motor/PyMongo async drivers within FastAPI's dependency injection and lifespan pattern.
• Django Best Practices - integrating MongoDB alongside Django's relational-first ORM via MongoEngine or Djongo where required.
• Microservices Architecture - per-service database ownership and change streams for event-driven integration.

Glossary

• BSON: Binary JSON, MongoDB's on-disk and wire document format.
• Embedding: Storing related data as a nested subdocument/array within a parent document.
• Referencing: Storing a related document's identifier and querying it separately, MongoDB's analogue to a foreign key.
• Compound index: An index spanning multiple fields, whose field order determines which queries and sorts it serves efficiently.
• ESR rule: Equality, Sort, Range - the recommended field ordering heuristic for compound indexes.
• Aggregation pipeline: A sequence of server-side document-transformation stages used for analytics and cross-collection joins.
• Index intersection: A fallback query plan combining two single-field indexes, generally slower than a purpose-built compound index.
• Multi-document transaction: ACID guarantees spanning more than one document/collection/shard, with added latency and throughput cost versus single-document atomic writes.
• Shard key: The field(s) MongoDB uses to distribute a sharded collection's documents across shards.
• Change stream: A resumable, real-time feed of a collection's write operations.