In my personal definition Sharding is Shredding. That's how I like to memorize this complex term. Instead of saying: Sharding is a type of database partitioning that separates large databases into smaller, faster and more easily managed parts called data shards, I simply think of a shredding machine dividing a piece of paper into smaller parts. And each smaller part is a shard. The word shard means small part of a whole.
If you are following this blog, you already know that one of the Achilles Heels of Blockchain is scalability. As the world gets more interconnected and applications run on the decentralized distributed network, we have higher amounts of data to be processed (whether as the form of a transaction or, as computational code of a Smart Contract). Decentralized Applications (DApps) will need to process fast to be competitive. People are already used to having a credit card process a payment in less than 4 seconds, or a YouTube video streaming in just a few seconds. We want technology to be equally good, but decentralization of servers and databases come at a cost of slow processing times.
Blockchain Engineers are working very hard to develop a fast solution for all of us. And sharding is an intrinsic part of building a scalable blockchain architecture. Some projects are working to shard their original virtual machines. Others already start the protocol with a sharded architecture with a promise that they will be scalable since day 1.
So, what is the main issue of sharding? Is that, you want a FAST distributed network but also a SECURE one. The more nodes you have to verify a transaction, the more secure it is, right? But also...the more nodes you have to validate, the more "eyes" that need to check each transaction, so, it will take a longer time.
But...what if we could divide this computational power into shards?
It's quite a clever idea. Instead of requiring that every transaction pass through a single pool of validators, they now could be divided into subgroups called shards.
But...what exactly do the validators do?
Validators ...hmmm...they validate! They do validation basically through Proof of Work (via mining) or through Proof of Stake (via voting based mechanisms). Each shard would now have its own validators.
The thing is...it is significantly cheaper to corrupt one shard than to corrupt the whole network. And all what you need is to corrupt one shard to make the whole chain less secure. Imagine a non-sharded chain with x validators (let's say 1000) decides to hard fork into a sharded chain. To corrupt the initial chain, you would need to need 510 Byzantine validators to take control of the chain. But now, in the new sharded version of this network you would have let's say 10 new shards, with 100 validators in each shard.
To corrupt one shard you would need only 51 Byzantine validators, which is much easier to get than the initial 510 to corrupt the whole chain. If those 51 validators are colluded together in one shard, they can double-spend and do all sort of malicious attacks together in that shard. And the security of the chain is gone.
but now... WHO CHOSES THE VALIDATORS FOR EACH SHARD?
Almost all sharded designs rely on randomness to assign validators to shards. It is important that validators are not grouped together as they wish, but as a system assigns them. Randomness is a current challenge in sharded architectures.
In some protocols they have a separate blockchain to determine this randomness. This "ultra-powerful" chain is called the Beacon Chain (in Ethereum and NEAR Protocol) , is called the Cosmos Hub in Cosmos and is called the Relay Chain in Polkadot. In the end, they all have some important "admin" features that will be able to arbitrate the flow of validators. How long do they validate transactions in each shard and if they deserve to be slashed.
In my next post I will open up some other important features of the Beacon Chain. I hope you are enjoying, please remember to check the YouTube channel and always feel welcome to share your knowledge with me too. As blockchain evolves, it is important to have a good critical mass exploring those topics, and you are welcome to participate in this journey.