Distributed Systems & Consensus Mechanisms
Artemis Week-1
After months of preparation, the wait is finally over and we started the proof-of-concept course at Artemis. For those of you who don't know, Artemis is an educational project focused on web3 development, and the team’s goal is to help craft the next wave of talented developers of the ecosystem.
This industry grows fast, and there is plenty of information to be aware of if you want to be a successful developer. Because of that, Artemis offers an 8-week course that covers the very basics of blockchain development (to set a solid foundation), and which will finally go into more advanced stuff (to give students the required tools to keep improving on their own).
The course will not only focus on development but will also have complimentary weekly talks regarding different interesting topics that will help students get a clearer picture of the industry. Despite these talks will be really insightful and given by reputable industry personalities, I won't cover them in this dev journal because I want it to be focused on technical stuff. Instead, I will most likely write about some of them in my main blog (check it out if you haven't already).
Let's begin. Artemis Week-1
First things first. Introduction week, where we went through the history of money, the history of blockchains, we did some basic cryptography (asymmetric keys, hash functions, etc.) and also went through consensus mechanisms. Despite all of the topics being really interesting, today we will focus on distributed systems and consensus mechanisms.
Distributed Systems
A distributed system contains multiple nodes that are physically separated but connected using a network. All the nodes in the system communicate and coordinate with each other to reach an agreement on a variety of topics (single data values, new transactions, the state of the network, etc.).
On top of that, it is necessary to not only coordinate but also maintain system reliability even in the presence of faulty processes. Distributed systems are especially useful because of their ability to not have a single point of failure.
The Byzantine Generals’ Problem
Distributed systems must be able to handle downtime/failure of their components, and must also be resistant to malicious operators who attempt to alter the state of the network.
In 1982, Leslie Lamport, Robert Shostak, and Pease Marshall created an abstraction of this problem and presented it in a paper called The Byzantine Generals Problem. One of the hardest problems in distributed computing systems, and which was thought to be unsolvable for many years.
The generals face several challenges such as:
Finding out whether the content of a message is true (sent by an honest general) or not (sent by a traitor).
Being certain that the information in a message hasn’t been tempered.
The only way for the generals to succeed is if they stick to an algorithm that gives them an action plan that helps them move from a deterministic problem to a probabilistic one.
Byzantine Fault Tolerance
A Byzantine Fault Tolerance (BFT) system is able to overcome the Byzantine Generals problem, and therefore successfully operate even if some nodes fail or act maliciously.
With the release of the Bitcoin whitepaper, Satoshi Nakamoto was able to design a BFT system on a peer-to-peer network. This achievement was possible thanks to the combination of cryptography security, and the development of a consensus mechanism called proof-of-work (PoW).
Cryptographic Security
Satoshi used cryptographic hash functions and public-key encryption to prevent data tampering and to verify the identity of networks.
Cryptographic hash functions are algorithms that take an arbitrary amount of data as an input and produce a fixed-size output of enciphered text called a hash.
Hash functions are especially useful for security purposes due to their properties:
Non-reversible. Hash functions are unidirectional, they make it very hard to reconstruct the original input from the hash.
Avalanche effect / Non-predictable. A change in just one bit of the original input results in a completely different hash.
Determinism. A given input always generates the same hash value.
Collision resistance. Different inputs shouldn’t produce the same hash.
With the usage of cryptographic hash functions, transactions can be secured in a block that connects to other blocks by its hash value. All hashes can be traced back to an initial block that is the root of all hashes. Note that by ensuring that each block stores the hash of the prior block, we can be certain that previous data hasn’t been altered. This hash structure is known as a Merkle Tree, and it is used to verify hashes that originate from the genesis block.
Proof-of-Work
PoW is a consensus mechanism that established a set of rules to validate and add new blocks into the network, and, therefore, solve the Byzantine generals’ problem.
According to Satoshi, the generals must agree on a difficulty metric. Let’s say that this metric states that the final hashed value must start with three 0s. The generals will then create a message and hash it until the output sticks to the agreed difficulty condition.
Note that if someone (the messenger or other generals) tries to tamper with the message, the final hash will change drastically and it won’t be taken as valid because it won’t stick to the pre-agreed rules. The only option for traitors is to find another nonce that will result in a final hash that sticks to the difficulty condition. Since we already know that this is a resource-intensive process, the traitors won’t easily be able to find a new hash.
When the message is received, it can easily be validated by hashing the nonce and the message and checking it against the provided hash.
Recommended Resources
If you want to visually understand everything that is being discussed in this piece, would highly encourage you to check this video by Austin Griffith.
If you prefer a more academic description of the topic, I also encourage you to go through the resources of Distributed Systems course at the University of Cambridge.
Hey, what is the Artemis Course, and where can I know about it?