[FINAL] Upgrade #1: Bedrock Protocol Upgrade - v2

Executive Summary

In short, what is this upgrade proposal?
Impacted stakeholders & expected outcomes
Why the Collective should upgrade

The Optimism Foundation is proud to propose the first protocol upgrade to the Optimism Collective: Bedrock. The Bedrock release of the OP Stack represents the culmination of years of research and development by the Ethereum scaling community and is a complete rewrite of the core components of the Optimism architecture. This upgrade offers a new level of modularity, simplicity, and Ethereum equivalence for Layer 2 solutions, providing unprecedented performance and functionality.

Most users will not be impacted by the upgrade as the current mainnet is already EVM-equivalent. Users and projects that run both full and archive nodes, make use of deposits and withdrawals, and make assumptions about the block time on Optimism Mainnet will need to take action to prepare for the upgrade. To help ensure a smooth transition, a thorough changelog and stakeholder-specific requirements can be found below.

In addition to these technical improvements, the Bedrock upgrade is a significant step towards the multi-chain future. By creating a shared standard, the OP Stack, Bedrock sets the stage for a Cambrian explosion of aligned L2s. This is an opportunity for leadership in the Ethereum ecosystem.

We are confident that the post-Bedrock experience will be a positive change for developers in the Optimism ecosystem and have received consistent excitement for the upgrade from our partners. We are committed to making this upgrade a success and are eager to see the results in the months and years to come.

Technical Summary

Overview of architectural changes
Link to all protocol/API/tech specifications
Overview of ongoing security considerations incl. all audits and findings

Overview/Architecture

At the highest level, the Bedrock release implements a modular architecture, separating the OP Stack into 3 components: consensus, execution, and settlement. These designs borrow heavily from the Engine API introduced for the Merge, and, as a result, an Ethereum execution client can be converted into an Optimism execution client in <2,000 lines of code. This is a huge win, paving the way for a multi-client future, and allowing for maximal use of Ethereumā€™s battle-tested code.

Technical documentation for Bedrock is split into 3 main parts:

Security Considerations

A summary of the initial security measures and considerations for Bedrock can be found in this blog post by OP Labs, which includes links to security audits and testing measures. For easier access, a list of security audits are hosted in the OP Stack monorepo. Audits relevant to the Bedrock release are prefaced with 2022/3.

Notably, the Bedrock release introduces a 2-phase withdrawal process, doubling down on bridge security, which proved to be a primary risk factor for chains in 2022.

Since the previous draft of this proposal in voting cycle 10, the Sherlock community completed a successful bug hunt, which most notably resulted in 3 High severity findings and 11 Medium severity findings. None of these enable theft of assets, but some could have caused usersā€™ assets to be locked until a subsequent upgrade made them recoverable.

Impact Summary

Changes in performance characteristics
Time-of-upgrade considerations (downtime, etc)
Links to exhaustive upgrade documentation for impacted stakeholders

The Bedrock release enables performance improvements across the board, including transaction costs, throughput characteristics, and sync speeds. Exact numbers are dependent on real-world chain activity, which canā€™t be known until actual deployment. However, we estimate a 47% reduction in protocol costs/security fees. You can read more in this post on the OP Labs dev blog.

The Bedrock migration will require pausing deposits and sequencer transaction ingestion during the upgrade, effectively resulting in network downtime. We estimate that this will take <4 hours. Unlike previous upgrades to Optimism, this release will not require a ā€œregenesis,ā€ and historic chain data will still be accessible after the upgrade. As such, little action is required from end-users of Optimism other than being aware of that timing.

While the upgrade strictly improves on EVM equivalence, some application developers may be affected. While we have been in active touch with major partners, the developer community has also been actively notified over the past few months in 1:many communications. We maintain the following documentation pages for impacted developers:

Action Plan

Mainnet Upgrade Timing
Contingency plans in case of last-minute bugs or issues
Plan for communication and education to the community

Since the previous version of this proposal, we have made the decision to favor a set of rigorous go-live criteria for the upgrade, as opposed to specifying to the exact date in the proposal itself. Specifying go-live criteria allows the community to focus on the ideal conditions for the upgrade, rather than committing to a timeline that could result in a rush to the finish line. Primarily, these criteria consist of:

  • A successful upgrade of Goerli to the ā€œRegolithā€ release, which incorporates the improvements recently identified by Sherlock.
  • A follow up audit to test the fixes made after the previous audit.
  • Internal rehearsals of both the migration and recovery from an invalid output root
  • 2 weeks of testnet stability on a consensus- and feature-frozen release, followed by a public announcement with 3 weeks of notice to the community.

Regolith Goerli Hard Fork

The Regolith hard fork is a hard fork that will be activated at a set block time on Optimism Goerli around March 17, 2023 @ 7:00:00 pm UTC. Because the hard fork will be activated before the governance vote completes, Optimism Mainnet will be upgraded to Bedrock with the Regolith hard fork already activated. Regolith improves Ethereum equivalence, primarily for deposit system transactions which are required by L2. Changes are defined in the specs here. These features would be activated from day 1 of Bedrock on Optimism Mainnet.

Go-live Criteria

Once the Regolith fork is complete, the next step is to upgrade mainnet. If this proposal passes, we will upgrade to Bedrock given that the following criteria are met:

Consensus and Feature Frozen

  1. No consensus changes are to be made to the op-geth op-node, op-batcher, or op-proposerpackages. No major refactors, architectural changes, or non-trivial feature additions are to be made to op-gethand op-node packages .
    1. ā€œConsensus changesā€ are defined as any changes that modify the batch derivation function, the EVM, or would otherwise require coordination between node operators.
      1. Examples include: L1 Cost function / Receipt changes in op-geth, introduction of a new batch version or compression, changing signing hashes.
      2. No changes should be made to op-geth unless they are clearly justified as non-consensus
      3. No changes should be made to the in the rollup package in the op-node unless they are clearly justified as non-consensus
    2. No major refactors or architectural changes (e.g.: pulling in an upstream geth version, modifying the derivation pipeline, fundamental changes to peer-to-peer networking)
    3. No non-trivial features should be added to op-node or op-geth (e.g. Snap Sync)
  2. No upgrades to the L1 or L2 Goerli smart contracts are required or executed.

Note: Low severity bug fixes, additional testing, small scope refactors will not be considered ā€œun-freezing.ā€ These small refactors may precipitate minor releases of our software.

2 Weeksā€™ Stability

After Optimism Goerli is change-frozen, the sequencer, batcher & proposer must show 14 consecutive days of stability. We define stability in this context as:

  1. No Sequencer failures.
    • The sequencer does not persistently drop any valid L2 transactions.
    • The sequencer does not produce any invalid blocks.
    • The sequencer consistently produces blocks at the intended 2-second intervals.
  2. No Batcher downtime exceeding 6 hours.
    • Batch transactions are regularly submitted and confirmed on L1 at the intended frequency.
    • The batcher is ā€œkeeping upā€ with the sequencer (i.e. the number of not-yet-batched L2 blocks is not growing over time).
    • The majority of batch transactions confirmed in each window are valid.
  3. No Proposer downtime exceeding 12 hours.
    • There is no gap in output root submission exceeding 12 hours.
    • The proposer does not otherwise violate its responsibility to keep the output roots live, e.g. by unnecessarily deleting valid output roots.
    • There is no invalid output root submitted.

Internal Rehearsals

  1. We have completed two end-to-end internal upgrade rehearsals, using forks of the Mainnet L1 & L2 networks.
  2. We have completed a rehearsal deleting an L2 Output, to prepare for the unexpected event of a faulty proposal.

Community Awareness

Only after hitting the above criteria will the Optimism Foundation announce an upgrade date, set at least 3 weeks after the time of announcement. Tentatively, we expect to announce the nearest Tuesday, at 9am PST, which fulfills this requirement. Note that the upgrade will be triggered at a specific time, and not a block height.

The Optimism Foundation and OP Labs will continue public communication efforts around the upgrade, including public communications and tweets reminding stakeholders of the upgrade in the weeks, days, and hours leading up to the upgrade.

Security Cancellation Clause

Note that this proposal would allow for bugfixes and stability improvements to be incorporated after the date that this proposal passes, potentially resetting the clock for the change freeze criteria. However, in the event that a significant, new security risk is discovered, we may cancel the upgrade, requiring a new vote once the community has had time to react.

Conclusion

This proposal outlines the Optimism Collectiveā€™s first Protocol upgrade of Optimism Mainnet to the Bedrock release. This upgrade aims to provide unprecedented modularity, simplicity, and Ethereum equivalence to the L2 network, with positive impacts on performance and security. Bedrock is the codebase which will help Optimism stand the test of time, and is also preparatory step towards the multi-chain future.

We expect the upgrade will incur around 4 hours of downtime, with no loss of historical data. Detailed technical specifications and impact summaries are available for the upgrade, as well as a thorough run-book of go/no-go criteria.

We strongly believe that the Collective should upgrade Optimism Mainnet to the Bedrock release, take advantage of the benefits it brings, and to continue to play a leadership role in the scaling community.

32 Likes

Hey folks, just wanted to provide some context on this v2 of the Bedrock proposal beyond what makes sense in the proposal itself. While much of this proposal is the same as the initial version from last cycle, I thought it would be worth making some explicit callouts on what has changed:

  • Incorporation of Sherlock results: the post includes a summary of the results of the our most recent audit competition, and adds direct links to all historic audits for Bedrock. Iā€™ve also asked that someone from the OP Labs team provides an even more detailed breakdown than we think would be appropriate for the proposal itself.
  • Explicit callout of downtime: one important piece of feedback we got was that it should be clearer in the post that during the upgrade, transaction ingestion will be paused. Itā€™s more explicit and bolded now; I thought worth restating here.

Lastly and most notably, instead of committing to a date up front this version of the proposal includes a rigorously specified go-live criteria for the upgrade, which includes circumstances that would block or delay the upgrade even after voting concludes.

  • The TLDR of this criteria is: 2 weeks of testnet stability on a feature and consensus-frozen Goerli, followed by at least 3 weeks of notice to the community.

The goal of this approach is to provide the community with a more rigorous standard for the proposed upgrade, and avoid any circumstances which could cause the release to be rushed. For example, in the previous version of the proposal, if we had discovered a potential stability improvement after completion of the vote, we would not have had 2 weeks to test on Goerli. In this version of the proposal, that scenario would reset the clock on the consensus and feature frozen criteria of the proposal, strictly blocking the upgrade. We think this strikes a good balance, which allows us to focus on the ideal conditions for the upgrade, rather than committing to a timeline that could result in a rush to the finish line, while still providing rigorous standards for the community to uphold.

12 Likes

Hey folks, Maurelian from OP Labs here to provide a more detailed rundown on the final results of the Sherlock audit contest.

First to provide some context: an audit contest is a variation on a typical smart contract security audit, in which a total reward pot is posted to incentivize auditors to look for issues in a codebase. The rewards are paid out based on the number and severity of findings. You can find more details about audit contests in general in the Sherlock docs and about the Bedrock contest in particular in the contest description.

Recap of the numbers

  • A total reward of $720k was posted
  • 333 people signed up for the contest
  • 314 issues were submitted in total (note: this does not mean there were 314 unique issues)
  • 60 people submitted at least one issue

Summary of Findings

After reviewing all 314 issues, identifying duplicates, and assigning severities, weā€™ve identified:

  • 3 High severity findings (which you can see here)
  • 11 Medium severity findings (which you can see here)

You may notice that the numbers in this update are different from Benā€™s previous update.

This is because the audit contest has recently completed the escalation period, during which auditors could stake funds on any issues they feel have been misjudged. This has led to a re-review of 32 issues by the Sherlock team with support from OP Labs.

Importantly, one of the high severity findings we previously reported has since been identified as a false positive.

The ā€˜evolving understandingā€™ of the results is a necessary by-product of the audit competition model, as a significant effort was required to understand and validate the many reports we received.

Next, Iā€™ll walk through the details of the various issues. Each of these issues has already either been fixed, or else has a fix in progress.

Overview of high severity findings

The high severity findings were all of a similar nature, namely permissionless griefing attacks which would allow any malicious user to interfere with a victimā€™s withdrawals from L1 to L2, causing their assets to be locked in the system. If the contracts were not upgradable, the locked assets would be irrecoverable.

Relevant to each of the high severity findings is the fact that all withdrawal transactions are sent via the OptimismPortal, which finalizes the withdrawal regardless of the result (so if the call from the portal fails, it cannot be ā€˜replayedā€™). In order to provide additional safety guarantees for most users, CrossDomainMessenger contracts are included in the codebase, which are designed to ensure that deposits and withdrawals sent using these contracts can always be retried.

It is also important to note that finalizing a withdrawal transaction is permissionless, ie. Bob can finalize Aliceā€™s withdrawals and vice versa.

With that context, the specifics of each attack are as follows:

High severity finding 1: Reentrancy lock up

This finding specifically affects withdrawals that are sent via the messenger contracts. It would enable an attacker to lock up usersā€™ withdrawals one at a time, meaning that they could identify a specific withdrawal and ensure that it is finalized in the OptimismPortal, but cannot be replayed by the L1CrossDomainMessenger. Details can be found in issue 87.

High severity finding 2: High gas limit lock up

This finding specifically affects withdrawals which have a very high gas limit (greater than 11,245,655 gas). In such cases an attacker can finalize the withdrawal but ensure that slightly less than the requested gas amount is forwarded. This is an edge case, but breaks the important guarantee that the amount of gas specified for the withdrawal on L2 will be made available to it on L1. Details can be found in issue 109.

High severity finding 3: Low gas limit lock up

This finding specifically affects withdrawals with gas limits set close to the lower bound of what is required to finalize it on L1. In such cases, an attacker can finalize the withdrawal with just enough gas to ensure that is is finalized, but not enough for the call to succeed. Details can be found in issue 80.

Overview of Medium severity findings

These findings (which you can see here) fall into 4 categories:

  1. Griefing
    1. Permissioned withdrawal griefing, would allow the Proposer to lockup a withdrawal, or the Challenger to invalidate finalized withdrawals. These attacks are similar to the High severity ā€˜permissionless griefingā€™ attacks except that they require increased authorization. Details can be found in issues 53, 57, 75 and 138.
    2. Deposit griefing, which would allow a malicious user to interfere with other usersā€™ deposits. Details can be found in issue 277.
  2. Errors in the migration process. Issue 105 would cause the migration process to halt thus increasing down time. Issue 235 may result in some pre-bedrock withdrawals being locked in the system.
  3. DoS attacks on the op-geth or op-node services. Details can be found in issues 71, 177, 276, and 279
  4. Token compatibility issues. Details can be found in issue 220.

Next steps

Each of the identified issues has already either been fixed, or else has a fix in progress.
We are now performing our own internal audit, using these findings to:

  1. Retroactively search for similar issues;
  2. Identify testing and process improvements which would have prevented these issues from being introduced; and
  3. Create monitoring to ensure that we can detect and respond to similar issues should they occur in production.

We will also be conducting a follow up audit, and will announce the date as soon as it is confirmed.

14 Likes

Iā€™m unclear about the timelines. Ideally, Iā€™d like to see voting on this proposal after Go-live criteria is established, and 2 weeksā€™ stability is demonstrated, but itā€™s unclear at what point this proposal goes to vote. Cycle 11 seems a bit sketchy while Goerli testing is still pending on a new version with multiple changes post-Sherlock. Without that, governance voting is more like ā€œgiving Optimism Foundation rights to manage the upgrade as definedā€ rather than actually voting on the upgrade. Which may be the defacto situation anyway, and I donā€™t have an issue with that in this early stage, but it should be clarified. In the medium-to-long term, though, upgrade proposals should only be submitted after readiness for mainnet has been abundantly demonstrated.

Appreciate the detailed recap of Sherlock. Can the ā€œinternal upgrade rehearsalsā€ be expanded akin to shadowforks L1 devs do? I.e. after the fork, mainnet transactions are replicated on the shadowfork. Thatā€™s a great way to test upgrades.

8 Likes

I dont have the technical expertise to check the fixes in detail myself. I understand that an external audit is being planned, before this is going to vote (and implementation).

I wonder though - with such an important update for the whole ecosystem - if not (at least) TWO external audits from two Tier-1 auditors would be the smart thing to do?

3 Likes

omg what a big text^_^

1 Like

Hey everyone,

Ross and Mason here from a16z. We are excited to see the Bedrock upgrade come to fruition in the near future. Thanks to @ben-chain for the highly detailed overview and the entire OP team for the work theyā€™ve done on Bedrock. The communityā€™s identification of 14 findings in the Sherlock audit contest is a testament to the practical value of decentralized governance processes.

The introduction of the OP stack untethers users from the bootstrapping and technical requirements typically necessary to stand up a chain and allows for experimentation via its highly modular design. We foresee this framework of open experimentation and rapid iteration resulting in the proliferation of novel L2 deployments. As more developers introduce modified deployments of the OP stack via their own chains, further cohorts of developers will be inspired and incentivized to deploy their chain using the OP stack. This cyclical feedback will feed a flywheel effect wherein developers launch their chain based on the OP stack and contribute to the codebase and surrounding tooling ā†’ the OP stackā€™s codebase and developer experience matures and improves over time ā†’ more developers launch their chains using the OP stack and contribute to the ecosystem.

The Bedrock upgrade also lays the groundwork for the Superchain vision. OP chains with shared infrastructure will inherit stronger security assumptions, better tooling, and new sequencing models that open up a massive design space for innovation.

Finally, this proposal is a symbolic opportunity to mark Optimismā€™s transition to on-chain voting. We look forward to hearing the communityā€™s assessment of on-chain votingā€™s impact on transparency, decentralization, and efficiency. We also look forward to hearing community feedback on Agoraā€™s implementation of Optimismā€™s on-chain voting module.

6 Likes

Thanks for the thoughtful followup as always! :slightly_smiling_face: Yes, this proposal does defer additional discretion to the Foundation regarding timing and deployment for the upgrade. While it also attempts to provide a rigorous standard for the community to uphold, and various conditions which would require a re-vote, that is definitely a bottom line. As I mentioned in my previous comment, we believe this is the best path forward in this early stage, given the involved nature of this upgrade.

In the medium term (i.e. post-Bedrock), I believe we should work as a community to make upgrades which are smaller, more explicit, and more incremental. There are meaningful advantages to upgrades which are implementable more similarly to L1 Ethereum, in which all code can be deployed in advance and then subsequently triggered by a pre-set block number. Weā€™re just not there pre-Bedrock.

WRT the shadowforks, this is an interesting callout ā€” let me loop back with the team to get some thoughts.

5 Likes

Thanks for the detailed overview. The go-live criteria seem reasonable. Very excited for this one to happen!

I am an Optimism delegate [ Delegate Commitments - #141 by Tjark ] with sufficient voting power, and I believe this proposal is ready to move to a vote.

7 Likes

Scalability- The potential that this Bedrock update has for the continued scaling capabilities on layer two is incredible and provides the true potential for mass adoption to occur. This is a critical time for such an event to take place.
I believe the OP stack creates a standard for blockchain technology that is unheard of for modern day development teams who are looking for new opportunities provided from the retroactive public goods fund. Itā€™s amazing that the funds continue to grow larger for the builders of the network who make the most impact as they are attracting new users with state of the art development products.

IMPACT OVER PROFIT

Centralization- There are still many aspects of the network that are currently still centralized including the way the discord server is setup at the moment. The actual sequencers for the network are also centralized from my understanding. As far as security for the network is concerned there is much more that can be done to ensure the safety of the users on the network and who are active in the community discussion. A major amount of work needs to be done in this category to continue on a steady path towards decentralizing these aspects so that way we do not rely on the convenience of the old legacy system which provides a point of failure for the way optimism operates at the current moment. When we rely on convenience we not only sign away our rights by using this type of technology, we also risk our security by storing information in a centralized database or servers that are vulnerable to attack.

Decentralization - The DAO for optimism has provided a sense of participation from the token holders who use the native $OP token to cast votes for proposals or important milestones for the network. Although it seems there is still a majority of the growth to be made on the network for the way grants are structured as well as the way international applicants are communicated with during the application process to ensure fairness for all regions of the planet. This aspect of an autonomous organization helps provide checks and balances to keep the community true to the original vision that it is focused on with the growth and expansion of the network while providing security for the users.

These three pillars are essential in the way they are balanced and itā€™s important to remember as we shift closer towards one of them we lean further away from the other two. This should help explain to people who do not understand the difference between a central bank and decentralized finance.

I am very excited for the bedrock update and have become fascinated with this technology over the past few years.
We are aiming towards building our own iteration of a layer two blockchain similar to Optimism with planned interoperability and retroactive contributions heading back to the mother of the OP Stack.
It would be amazing to see exactly what we can all we come up with when we put our brains together. Bedrock is the type of technical update that gives me a deeper appreciation for blockchain or development teams who are dedicated to pushing this scaling technology forward !

We are ready to commit our efforts into building a sustainable ecosystem for creators whether they make art or code in solidity. Alongside Optimism, Base, and any others who join the OP stack over the course of the future.

3 Likes

I am one of the Synthetix Ambassadors, and a Optimism Badgeholder.

I am an Optimism delegate [Delegate Commitments - #65 by mastermojo ] with sufficient voting power, and I believe this proposal is ready to move to a vote.

7 Likes

We are an Optimism delegate with sufficient voting power and believe this proposal is ready to move to a vote.

8 Likes

Thank you very much for the information

1 Like

Thanks for the proposal.

Looks like thereā€™s a lot of work going on to ensure a smooth upgrade. kudosā€‹:100::100:

One area I would like to get a little bit of clarity is the education part. Are there any elaborate efforts to keep the community and the world informed about everything going on with the upgrade? Or is mainly tweets, forums and discord for now?

2 Likes

We are an Optimism delegate with sufficient voting power and believe this proposal is ready for a vote.

After months of playing on the testnet to ensure Rubicon is prepared for the upgrade, our team is excited to see Bedrockā€™s lower fees, consistent block time, and multi-client L2 ecosystem in production! :rock:

Network upgrades are no easy feat; we were very pleased to see the strong commitment to security and transparency throughout this process. In particular, we love the ā€œresetā€ functionality in this vote and think it will be great practice for future network upgrades!

8 Likes

This is Ethan from Pika Protocol, an Optimism delegate with sufficient voting power and believe this proposal is ready for a vote.

Since Pika Protocolā€™s users are very sensitive to gas fees and block stability, we are excited to see Bedrockā€™s low gas fees and consistent block time and believe the upgrade will bring large benefits to our users.

6 Likes

Can the ā€œinternal upgrade rehearsalsā€ be expanded akin to shadowforks L1 devs do? I.e. after the fork, mainnet transactions are replicated on the shadowfork. Thatā€™s a great way to test upgrades.

Good suggestion, thank you. We are in fact using a shadow fork to run our upgrade rehearsals. Of course there are also security concerns related to running a shadow fork, namely the risk of a signed transaction being leaked and replayed to mainnet, potentially leading to an unintended upgrade of the production system!

To prevent this, we use our op-wheel tool to replaces the address of the multisig which controls the upgrade process with a throw away account which we use during rehearsal.

I should also note that only L1 transactions are being replayed to our shadow fork. Due to the fact that the Optimism Mainnet L2 is still running the legacy system and does not support EIP-1559 transactions, the L2 state roots will immediately diverge, meaning weā€™d receive little insight from replaying L2 transactions.

2 Likes

I am one of the Kwenta Elite Council.

I am an Optimism delegate [Delegate Commitments - #173 by Kevin-IzzRite] with sufficient voting power and I believe this proposal is ready to move to a vote.

4 Likes