Coordicide: The Road Ahead

With the help of our vibrant community and industry partners working towards real-world adoption, the IOTA project has come a long way in the past three years. While the Coordinator has been an essential component thus far for securing the network, it is often described as “training wheels for the network while it is still in its infancy.” We have been hard at work understanding how to safely remove those training wheels.

We recently published our Coordicide blueprint, which outlines the progress we have made so far, as well as the challenges left to tackle. But solving those problems is just the first step; the solutions still need to be implemented, tested, and ultimately ready to support the IOTA network in the real world.  

Removing the Coordinator is the number one priority for the IOTA Foundation’s Research & Development teams. The Coordicide project is a joint effort between both the Research and Engineering teams, and we also welcome those in the IOTA community who would like to get involved.

At a high level, the strategy looks like this:

This strategy can be understood in two distinct phases: the Research and Implementation phase, and the Testnet phase. The final goal, then, is to perform a Mainnet Transition, where the current state of the ledger is brought into the new network.  

During this phase, we have three specific goals:

1. Solve the open research questions around each Coordicide module
2. Write a comprehensive technical specification, which can be used by both the IOTA Foundation and any others who may wish to implement an IOTA-compatible node
3. Implement the specification as the IOTA Foundation’s future reference node software; for now, the working title of this node software is “Bee”

As each module is fully developed, work on the specification and implementation will be started in parallel. By following this strategy, ideally, the reference node software will be available without much delay after the last research questions have been solved.

As a further speed-boost towards the production node software, we are also working on a rapid prototype called “Go-Shimmer”, which aims to test the consensus algorithm and some of the other Coordicide modules. This will allow us to have a simplified “Alpha Net” for development, feedback, and to simply provide a sanity-check against our Coordicide implementation without first formalizing the specification or writing production-ready code.

In the meantime, preliminary work on Bee has already begun. The modular, plug-in based architecture is being defined, and many modules are already well-known or do not have any significant open research questions. These modules include:

• Ternary
• Hash function
• Networking
• Gossip
• Transaction layout
• Tangle graph structure
• Ledger state & balance calculator
• Generic HTTP API
• Client API built on HTTP API
• Local snapshots

Further research into the remaining modules is well underway. These include mainly those listed on the Coordicide website, namely:

• Shimmer (consensus algorithm)
• Mana (Sybil protection)
• Auto-peering
• Tip selection
• Node identifiers
• Rate control
• Timestamps
• Node bootstrapping & syncing

In addition to these Coordicide node-specific modules, many existing projects such as the client libraries and the IOTA Hub will be updated as the new specification is defined.

The Foundation’s Research & Engineering teams are working closely together to achieve these goals, and we invite community members from all walks — individuals, companies, academics, or any other interested parties — to reach out and participate where you are able to.

Unlike the rapid prototype (Go-Shimmer), the Coordicide Testnet is what we consider the first Release Candidate for the Coordinator-free network. The most difficult obstacle to overcome during this phase is to answer the question: “when are we (the IOTA Foundation and the IOTA community) confident enough to entrust the entire valuation of the IOTA network to our new node software?”

New projects starting from scratch do not have to worry as much about this question since the amount of money at stake may be small by comparison. For IOTA, however, it is imperative that the Testnet mimics as closely as possible the real-world network, including both:

• Enough honest actors to meet the required security assumptions, and;
• Enough highly-skilled malicious actors and security experts doing their best to find any and all breaking points

In any cryptocurrency network, the real test of security and resilience is done over time as the network grows and attracts users (both honest and malicious) and provides a strong economic incentive for both types of users to participate.

In a Testnet, as tokens do not typically have any real dollar value associated with them, we, therefore, need alternative solutions to incentivize this kind of use. We are currently planning two bug-bounty programs, which will run in parallel:

1. A standard bug-bounty program, for finding errors or bugs in the code or on the running network
2. An “integrated” bug-bounty program, where we insert Testnet tokens that are redeemable 1-to-1 with mainnet IOTA tokens, in the event a malicious actor is able to move them onto an address he/she controls

Both bug-bounty programs will be graduated, starting with a relatively small amount of value and increasing the bounty over time. Initially, IOTA Foundation will provide these bounties, but we may set up a bounty pool for community members to “stake” their own (real) IOTA tokens against Testnet tokens. This provides an excellent mechanism for people who trust the new system to prove it, by (effectively) putting their own tokens on the Testnet before the full ledger is copied over.

This is another area where we welcome help from the IOTA community — to come together and ensure that we are putting this network through the best, most thorough penetration tests possible.

The final step is to transition the ledger balances from the existing network to the new network, which at that point becomes the new, Coordinator-free Mainnet.

Once the community reaches consensus that the Testnet is stable and will support the IOTA Mainnet in full, we will be able to specify the exact time of the transition, which will be performed by snapshot. This time will be the final period during which exchanges and all other network participants should prepare their applications for the upgrade.

We plan to maintain backward compatibility for users of IOTA Foundation supported libraries and software as far as possible. This includes the Trinity wallet, client libraries, IOTA Hub, and more.

Since we anticipate that the hash function and signature scheme, and thus wallet addresses, will change during this upgrade, the Trinity wallet will include a fully automated transition process. Users who open their wallet for the first time after the network transition will see their funds moved to a new address. Funds received at old addresses would also be automatically sent to a new address.

Additionally, historic data already written into the Tangle will remain available through permanodes. These will enable querying for historical transactions, independent of snapshots or other protocol changes.

And of course, it is important to mention: work to improve the current node software — IRI — is an ongoing effort and will not be affected by the Coordicide project. Current IOTA users can expect to see ongoing progress and network improvements, even before Coordicide is complete.

As you can see, we have our work cut out for us. We hope this post sheds some light on our plans for Coordicide, and we hope to see you around on Discord or Github to help make that happen!