Software Engineering · Distributed Systems
Splitting a dataset across multiple machines — often by a key like user ID — so each machine holds only a slice of the total data.
Mental model: Like splitting one giant phone book alphabetically across several volumes.
Example: A user database split into 10 shards by user ID, so each shard holds a tenth of the total accounts.
In practice: Choosing the shard key is the critical decision — a poor choice creates 'hot shards' that get disproportionate traffic while the others sit idle.
Debated: Often reached for prematurely — many teams shard well before they have the write volume to justify it, taking on real operational complexity for a scaling problem they don't actually have yet.