Memory Limits and allowDiskUse
Parent: MongoDB Aggregation Pipeline · Published reference · snapshot 2026-10-24 · skill mongodb-cohort-8f2fb9450e90 · viability 3.05 / 5 (skip)
Also known as: Per-operation memory limit in 9.0
↓ Facts as markdownall context files
Source-anchored research for Memory Limits and allowDiskUse. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- The profiler and diagnostic logs expose usedDisk when an aggregation stage writes temporary files because of memory restrictions. Correlate that signal with latency and disk utilization to distinguish successful spilling from efficient in-memory execution. (high confidence) [source]
Measurements and reference values
- Eligible blocking stages have a 100 MB memory threshold and can spill intermediate data to temporary files when disk use is permitted. This is a per-stage threshold, not a total pipeline memory budget. (high confidence) (verified as of 2026-10-14) [source]
Problems, failure modes and limitations
- Disk spilling introduces storage I/O and can increase latency. Reduce input with selective early $match filters, use a supporting index for sorting, and use $sort followed by $limit when only the highest-ranked results are needed. Validate the resulting plan. (high confidence) [source]
- $facet cannot spill to disk: allowDiskUse does not remove its 100 MB constructed-document limit. Its final output document must also satisfy the 16 MiB BSON document limit. (low confidence) (verified as of 2026-10-21) [source]
- MongoDB issue SERVER-73315 contains a reproducer using $group with $push and allowDiskUse: true that still fails with a 104857600-byte accumulator memory error while merging accumulated arrays. Treat this as evidence of an accumulator limit, not proof that all $group operations cannot spill. (low confidence) (verified as of 2023-10-29) [source]
- Atlas Free and Flex clusters do not support writing temporary files to disk. Atlas ignores allowDiskUse there and behaves as if it were false, so setting the option cannot resolve their spill-related aggregation failures. (low confidence) (verified as of 2026-10-14) [source]
- MongoDB 9.0 adds a separate per-operation limit across memory-tracked stages. Its default is the greater of 1 GB and 20% of memory available to the server process. This does not replace the per-stage spill threshold; internalQueryMaxMemoryUsageBytesPerOperation configures it. (high confidence) (verified as of 2026-10-09) [source]
- Your Spotify issue #272 reports an aggregation failure with code 292, QueryExceededMemoryLimitNoDiskUseAllowed, when $group lacked disk-use permission. Diagnose the failing stage and effective options before treating such an error as server-wide RAM exhaustion. (high confidence) (verified as of 2023-08-07) [source]
- The 100 MB limit concerns stage memory, not total collection size. Spill-capable stages include $group, $bucket, $bucketAuto, $setWindowFields, $sortByCount, and $sort without index support; allowDiskUse permits temporary disk storage when needed. (high confidence) [source]
- $facet cannot spill to disk. Its constructed result document has a 100 MB limit that allowDiskUse does not change; exceeding it raises ExceededMemoryLimit. (low confidence) [source]
- Each returned aggregation document must fit the 16 MiB BSON document limit. A $facet result must also satisfy this final output limit; enabling disk use does not enlarge the returned-document allowance. (high confidence) [source]
- allowDiskUseByDefault applies to mongod, not mongos. mongos never writes temporary files to disk, so changing this parameter on a router cannot enable router spilling. (low confidence) (verified as of 2026-10-24) [source]
- MongoDB 9.0 adds a combined limit across memory-tracked stages of one operation. The default is the greater of 1 GB and 20% of memory available to the server process. This is separate from the 100 MB stage limit and does not change stage spilling behavior. (high confidence) (verified as of 2026-10-24) [source]
- In MongoDB 9.0, internalQueryMaxMemoryUsageBytesPerOperation configures the operation-wide limit at startup, runtime, or through query settings. A separate internalQueryMaxSingleExpressionMemoryUsageBytes limit defaults to 100 MB for a subset of memory-intensive expressions outside the operation-wide limit. (high confidence) (verified as of 2026-10-24) [source]
- Reduce aggregation memory demand by using an index to satisfy sorting, placing $limit after $sort when only top results are required, and filtering with $match early so later stages process fewer documents. (low confidence) [source]
Comparisons and alternatives
- Starting in MongoDB 6.0, allowDiskUseByDefault controls automatic spilling and defaults to true. An aggregate command can override it with allowDiskUse: true or allowDiskUse: false; inspect the server setting rather than assuming the deployment default. (high confidence) (verified as of 2026-10-24) [source]
- In mongosh and the Grafana MongoDB data source query syntax, specify aggregate(pipelineArray, {allowDiskUse: true}); the option belongs in the options object after the array, rather than inside a pipeline stage. (high confidence) (verified as of 2026-10-23) [source]
Facts and statements
- Spill-capable examples include $group, non-index-supported $sort, $bucket, $bucketAuto, and $setWindowFields. Permission to spill does not make every stage or expression spillable. (high confidence) (verified as of 2026-10-14) [source]
- In mongosh, pass { allowDiskUse: true } as the options argument to aggregate(pipeline, options). This permits temporary disk writes; it does not force every eligible operation to spill. (high confidence) [source]
- Netdata MongoDB Top Queries exposes a Uses Disk field for disk use during sorting or aggregation and a Has Sort Stage field for an in-memory sort. Both are hidden columns in the documented results table. (low confidence) (verified as of 2026-10-07) [source]
- Spill files last for the pipeline execution and consume storage on the instance; enabling disk use therefore also requires considering temporary storage capacity. (high confidence) [source]
Children
- No children recorded.