April 26 Outage: Synthetix Impact And Alignment

Hello, Optimism!

Following the events of April 26th, I wanted to make the Optimism team and community aware of the impacts that degraded network performance had on Synthetix, as the event and the subsequent lack of communication left a lot to be desired with respect to alignment between Optimism and its ecosystem.

I am writing this post not as a representative of the Synthetix protocol, as I have no active role as a core contributor, but rather as a community member and ambassador who would like to see much clearer communication in the future.

Event Recap

On April 26th, Optimism experienced a significant downgrade in performance. As stated in the event post-mortem, this was caused by a large uptick in ā€˜eth_sendRawTransaction’ requests. While the sequencer technically remained online, performance with respect to user transactions, node syncs, etc. caused the chain to be essentially unusable.

Synthetix relies on Optimism’s infrastructure and liveness to function properly. User transactions, oracle updates, keeper activities, debt positions, and much more rely on continuous uptime from Optimism. Without it, as in the case of April 26th, Synthetix is likely to experience protocol and user losses.

During this outage, losses were incurred due to liquidations caused by market volatility, leading to imbalanced market skew that couldn’t be arbitraged. This skew directly led to losses being absorbed by Synthetix stakers.

Public communication from the Optimism team regarding the event, its causes and implications, was not given until May 9th, approximately 13 days following the event. The initial response to the outage was that it was caused by infrastructure providers. This initial information was later revealed to be incorrect. Incorrect information and delays in providing an accurate post mortem have large consequences for the protocols and users that depend on the network. Poor and untimely communication erodes confidence in the network.

Impact on Synthetix

Regular Market Conditions

Under normal circumstances, the ETH market may be balanced, with equal positions on both the long and short sides, e.g. $1M. This balance ensures that for every $1 in profit there is $1 in losses to offset it, making the market delta neutral for the liquidity providers.

Consider a scenario where a large $500k short position gets liquidated. The market composition changes to $1M long and $500k short, creating a $500k long skew…

Arbitrage incentives built into Synthetix Perps, such as dynamic funding rates and price impact, encourage arbitrageurs to close this skew.

Experience shows these incentives work as expected, returning the market to a balanced state, (i.e. $1M long and $1M short) very quickly. Stakers only temporarily assume skew from imbalanced markets

Network Downtime

During chain-wide downtime or congestion, traders are unable to open or close their positions, which prevents arbitrage. If market prices shift, stakers lose funds while traders gain, or vice versa.

In summary, the mechanisms in place effectively manage skew under normal market conditions, but network downtime presents a unique challenge, locking traders and stakers in their current positions exposing stakers to price volatility risk.

The incident on April 26th led to approximately $1M in PnL losses for stakers.

SNX staking is 16.8% of Optimism’s entire TVL. Therefore losses to Synthetix stakers due to network downtime represent a large threat to the Optimism ecosystem TVL and market share. Synthetix Perps also represent 70% of all perps volume on Optimism over the last seven days.

Aligning Incentives

The event highlighted what I believe to be a need for improved and continued incentive alignment between the network and its largest ecosystem partner. The success of Optimism is tied to the success of the Synthetix ecosystem. It’s therefore important Optimism strives to remain aligned with the success of its largest protocols.

Going forward, Optimism should make it a priority to take actions that align the incentives of its ecosystem to put the network in the best position to succeed. This means more active communication regarding upgrades or downgrades in the network’s performance. As a big fan of both Optimism and Synthetix, I hope this can serve as a starting off point towards continued but improved ecosystem alignment that leads to the success of the wider Optimism ecosystem.

3 Likes

Gretings,
Thanks for this excellent post. I strongly agree with the impact of downgrade in the Optimism network and the negativities that it can bring.

1 Like

Disclosure: I’m currently part of Synthetix Ambassadors
Even if you are not Synthetix user, the fact that Optimism response is embarrassingly slow, blaming third party first, then release post-mortem week later on @OptimismStatus (which then got low-quality response from another campaign). As an user we deserved better communication and responsiveness to outage.

This was my tweet from around that time

1 Like

great one… trhanks for contributing ser…

Greetings,

well yes I am experiencing the same issure on Optimism. I hope this isuse gets resolves soon. There is a stiff competition in web 3.0.