TL;DR: This blog discusses the development of a decentralized file storage network called Exa Protocol, which utilizes free storage space on mobile devices. We faced challenges with existing Layer 1 protocols and designed a new one called Darwin, using a parallel blockchain approach. Darwin aims to achieve high throughput and scalability with independent child chains. The TestNet has been built, offering a new approach to decentralization, empowering individuals to take part in maintaining the network and serving as a backbone infrastructure of the internet.
I still remember the time when Google decided to withdraw the FREE unlimited storage on their “Google Photos” App. For a self-proclaimed photographer like me, it was terrible news and it only indicated one thing – to purchase the Google Storage subscription. It made me think: “Oh, another subscription to live on this planet. Wasn’t subscribing to the government by paying them part of my earnings enough? (aka Taxes)”
Times like these make you wonder if there was something to solve this problem. My head had these thoughts:
- “Let’s build a free unlimited photo storage service and give it to the people so that people use it and then you run to the investors so that you can purchase and run more servers and then when you are making losses you say – I built a Unicorn” – Realistically, building a service that asks investors to pay for your photo storage.
- “But what if I charge for the service?!” Then the brick of reality hits my head – “So you want to build another Dropbox?”
- So no SaaSy service. What else?
Thankfully one answer lay in the Web3 world…
A smartphone today is as capable as a PC just 2 years ago (computationally) – but lies dormant for a substantial amount of time doing nothing – or worse yet, uses minimal computing resources while watching TikToks or Reels. What if this dormant phone could be used to create massive storage infrastructure using free available storage space on phones?
Rough math suggests that each year, around 80 EXAbytes of storage is added to this world via new mobile devices. The protocol is here to provide some of that space to the public. Hence we are calling it EXA…Protocol.
Here’s what we thought “Let’s build a token on Ethereum and set its intrinsic value to the amount of storage people share on their phones. They will be compensated with the native token whenever someone stores or retrieves files from their phone.” With the mobile-first approach, people who do not have the means to harness the power of decentralization and blockchain will be able to do it for the first time. This, in a true sense, will be the democratization of blockchain technology.
I won’t deny while working on this I got reminded of this quote by Joe Vitale –
A goal should scare you a little, and excite you a lot.
Joe Vitale
Ideation
I remember when Arpan Garg and I went to the whiteboard many months and were still puzzled as to how to solve the problem. The earliest version of the solution was – “We will use Ethereum and deploy a smart contract that will compensate users with our token whenever they allow people to use their mobile storage.” and this was riddled with major problems:
- Ethereum offers a limited TPS of 12 (before the PoS fork). We would have choked the Ethereum network with just 10K users who on average transact 100 times a day (one file request could mean 50 requests because of sharding – described ahead). This will have a cascading effect and push the gas price to execute a transaction through the roof! – This will not scale!
- How do we convince people to save their data on others’ phones?
- Phones go on and off and die after 2 years. How to make sure that the data is safe even when the phone dies?
First, we tackled problems 2 and 3.
- Allow users to encrypt the file before it would be sent to other nodes for storage by using AES 256 encryption which offers sufficient security.
- Once the file is encrypted we can shard the file and store the pieces on different devices. Multiple devices are used to store the shard for redundancy and for the scenario – “WhAt HapPeNs WhEN oNE oF tHe DeViCE gOEs oFFlIne, I’LL LOsE AlL mY DAta!?”. Yeah, we get this question a lot…
To be fair, the above solution is tried and tested and we were particularly calm about solving this problem. The major problem was with the Layer 1 protocols. They were slow. Like really slowww. The throughput they offered was just not enough to sustain even 1000 users on our storage network. As described earlier we would have choked the network easily. L2s offered no hope either.
Hence we decided to design a new Layer 1 that scales and runs on mobile devices natively. We are calling our L1 – Darwin.
Solution
One fine morning, Arpan and I were sitting in the office discussing, arguing over the new Blockchain architecture. We asked ourselves this question – “What if instead of 1 blockchain there were multiple blockchains running in parallel?” “Sounds good, but how would you govern it?” “Just like we govern countries, states, and cities, by allowing them to work independently.” “Let’s think it through, when there were a few humans on this planet, the entire world was one big country for them.” “Right.” “And when there were more humans, they divided the land and created separate countries.” “True.” “How about we do the same with mobile nodes?” “We can try…”
And we did try! We spent the entire week on the whiteboard designing the L1 with analogies from the real world. Our whiteboard used to look like this figuring out and ironing out the details of the architecture:
A month and a half later, we finalized the architecture and called it Darwin.
Darwin has been architectured just like how civilizations have evolved over time and created cities, states, and nations as we know it. Instead of viewing all the nodes as a part of one whole blockchain, Darwin segments them into multiple chains in order to achieve better governance and improve network performance.
Civilizations evolve and create groups such as cities, Darwin also evolves as more nodes are added to the network. The evolution of the Darwin Chain splits it into multiple Child Chains so as to group nodes that are nearby so as to improve network performance.
Civilizations also have a robust governance system that allows them to function properly. Darwin brings governance using server nodes that audit the child chains regularly.
How does Darwin Work?
Darwin Chain continuously evolves to increase the TPS as the network grows and also shards and optimizes the chain dynamically to create parallel child chains that work independently and have their own set of full nodes to validate the transactions. These nodes form chains of nodes that can transact with each other with low latency, and each chain then gets split into different chains when it matches some conditions – called Mutation.
When a chain mutates, each child chain grows independently and has its own independent nodes to validate the transactions. The nodes are distributed between child chains by using an algorithm to reduce the communication and transaction latency in each child chain. This exponentially increases the chain throughput as each child chain works independently to create and validate transactions.
The resources are available faster as the nodes are grouped based on closer proximity just like it works in the case of Web2 applications hosted on a server closer to the area which has the maximum traffic toward the servers. Darwin chain also uses factors including computing power, shared storage space, and network availability/reputation of a node while creating child chains.
Creating Darwin solved the biggest bottleneck for us: an L1 that can support a decentralized storage infrastructure and run natively on mobile devices.
Darwin Performance (theoretical max.): ~7.8 Million TPS.
This all has been formally laid down in our Yellow Paper which is available on our website.
But does it even work!?
Yes, it does!
We recently built a TestNet by converting our theory to actual code. In the above screenshot, you can see how the Darwin blockchain visually looks. The tree structure holds each independent blockchain.
Next Steps
Darwin is just a Layer 1 right now that can transact our native token. Our next step is to integrate DFS (Distributed File System) logic which will fulfil our goal of a truly decentralized storage infrastructure using mobile devices.
Looking at the scalability of the chain, enabling smart contract deployment by creating an EVM-compatible VM could be the next step after integrating DFS.
For some, it may be just another blockchain, for us, we see a brand new approach towards decentralization. When Satoshi wrote the Bitcoin Paper, he envisioned a world where people like you and me can take part in the network and maintain it. Over a decade has passed and it seems like the power to maintain the network is in the hands of few (I’m talking about ultra-large mining farms).
With Darwin, we intend to give people (not corporations, not rich few) to take custody of the decentralized system that enables crypto asset transfer and will serve as a backbone infrastructure of the internet.
Will you take part in this journey with us?
