On Stress Tests and Improving the IOTA Networks

The stress tests over the past few weeks probably have not escaped (m)any of you. Be it due to the numbers they produced, or the discussions surrounding them.

The IOTA Foundation considers stress tests to be an indispensable part of the network for multiple reasons:

They allow us to evaluate the overall state of the network(s).
They yield extremely useful data to analyze.
They elucidate bottlenecks in the node software, IRI.
They allow us to collaborate with some of the most dedicated members of our community — the spammers.

The IRI team — the team responsible for the performance and health of the Tangle — discussed stress testing the network on numerous occasions last year. However, we chose to focus our efforts on finalizing additions like local snapshots and making other important improvements instead.

Once local snapshots were released, we resurrected the whole discussion and have come up with a plan. We have decided to start stress testing the Mainnet in phases, with the first phase taking place this March. This, we believe, gives us enough time to fix the current issues with the 1.6.0 IRI build, before attempting any extreme load on the network.

The first phase will happen on a portion of the Mainnet network for which we will be laying the foundations in the coming weeks. We are calling this “the Mirrornet” (but in this case it is more of a one-way mirror). This arrangement will allow us to mimic the Mainnet environment faithfully enough and at the same time relieve some stress from the rest of the Mainnet. At least in this phase.

The first phase will allow us to identify and fix the most obvious bottlenecks and issues, as well as adjust the spamming patterns before we unleash traffic onto the rest of the network. That means there will be some delay between the phases for us to analyze the results and act accordingly.

We will coordinate with community spammers and the rest of the community on specific dates for the individual phases.

We do. And we appreciate all the effort you guys put into this. At the end of the day, we are building a permissionless ecosystem.The network needs to be resilient to any kind of behavior, and if it is not, we at least know where we need to improve.

Do we stay calm during these network events? Of course not. It is a strange mix of stress and excitement. I imagine engineers who are responsible for car safety feel similar when the car they have been working on for years is about to undergo crash tests. Stress on the Mainnet puts us on display, it surfaces issues we did not know about and it emphasizes the ones we did know about.

We have a number of solutions planned ahead to improve the node software— many of the improvements are short-term and will be visible right away. Other improvements are long-term, but also extremely important, like making the IRI code more adaptive, easier to maintain, and safer to release. Improvements like these are not immediately visible, but will more than pay off in the long term. Gal, the team lead of IRI, has been putting a lot of effort into improving the code standard to achieve this.

As an example, an immediately-visible improvement should be how nodes handle get Transactions To Approve and other API calls. But IRI in general needs to become more stable, less resource hungry, and able to process more transactions. Optimizing the way we hash incoming transactions is just one piece of the puzzle.

Stress tests on the Mainnet should be a norm. We will organize and perform stress tests on a regular basis and after releases that are targeted at improving node throughput. The purpose of this is to evaluate the progress we make and to make ourselves more accountable to the community. After each test, we will focus on improvements and analyzing the results and the data it yields. What happens in between the tests with regards to spam is of course up to anyone — we expect community stress tests to be run as the community sees fit.

Please make sure you join the discussion on the IOTA Discord server here. If you are interested in stress tests, in particular, the#tanglespam channel is a good place to be.

You are more than welcome to set up your own node and become a part of the network, please check out the IRI repo. To see development updates on the node software, check out the #IRI-discussion channel, and be on the lookout for our regular tech updates on this blog.