Skip to main content

Sharding

Sharding is a method for distributing data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations.

Why Use Sharding in MongoDB?​

Database systems with large data sets or high throughput applications can challenge the capacity of a single server. For example, high query rates can exhaust the CPU capacity of the server. Working set sizes larger than the system’s RAM stress the I/O capacity of disk drives.

There are two methods for addressing system growth: vertical and horizontal scaling.

  • Vertical Scaling involves increasing the capacity of a single server, such as using a more powerful CPU, adding more RAM, or increasing the amount of storage space. Limitations in available technology may restrict a single machine from being sufficiently powerful for a given workload. Additionally, cloud-based providers have hard upper limits based on available hardware configurations.

  • Horizontal Scaling involves dividing the system dataset and load over multiple servers, adding more machines to increase capacity as required. While the overall speed or capacity of a single machine may not be high, each machine handles a subset of the overall workload, potentially providing better efficiency than a single high-speed high-capacity server. Expanding the capacity of the deployment involves adding more machines to the network.

Sharding is MongoDB's approach to horizontal scaling.

How Does MongoDB Sharding Work?​

In MongoDB, the sharding process is fairly straightforward. MongoDB distributes the data in a collection across multiple databases, called shards. Each shard holds a subset of the sharded collection. MongoDB distributes the documents based on the shard key.

Here are the main components involved in MongoDB sharding:

  • Shard: This is a MongoDB instance that holds a subset of sharded data. You can have several shards in a MongoDB cluster.

  • Config Server: This is a MongoDB instance that holds metadata about the cluster. Basically, it knows what data is on which shard.

  • Query Router (mongos): This is a MongoDB instance that routes queries from the application to the appropriate shard(s).

Shard Keys​

The shard key is a field that exists in every document in the collection. MongoDB uses the shard key to distribute the collection’s documents across shards. The choice of shard key cannot be changed after sharding.

A good shard key can:

  • Distribute write operations across the shards.
  • Distribute queries across the shards.

When to Use Sharding​

Sharding is ideal for deployments with very large data sets and high throughput operations, and enables you to leverage the resources of multiple servers for read and write operations.

However, sharding requires careful planning and consideration. The choice of shard key and the overall design of the sharded cluster can affect the performance of the cluster.

Conclusion​

Sharding is a powerful feature in MongoDB that allows you to partition data across multiple servers, and balance the load of high-traffic databases. It's an essential technique for handling large datasets and high throughput operations. However, it requires careful planning and design to ensure optimal performance.