Upgrade Proposal #13: OPCM and Incident Response improvements

Executive Summary

Hi I’m Maurelian, a Security Engineer at OP Labs. This proposal was written in collaboration with Lewej (@0xEscanor) and @Kelvin from the OP Labs Team.

This proposal is intended to be voted on in cycle 34.

Neither OP Labs nor I represent or speak on behalf of the Optimism Foundation.

This proposal:

  • outlines the implementation of a new system for upgrading L1 contracts across the Superchain (OP Contracts Manager),
  • introduces technical improvements to several key contracts to enable more flexible and less disruptive ways to respond to any potential incidents in the OP Stack fault proof system (Fault Proofs Incident Response Improvements), and
  • introduces a new Safe Module called the DeputyPauseModule to be installed into the Optimism Foundation Safe to simplify the process of quickly responding to security incidents via the Superchain-wide pause mechanism (Superchain Pause Improvements).

Impacted Stakeholders & Expected Outcomes

  • Optimism Foundation: Simplified process for triggering Superchain-wide pause quickly in case of an emergency.
  • Security Council: introduces a method to perform “batched” upgrades to multiple chains at once

Motivation

The goal of this upgrade is to implement the above changes to 1) make contract upgrades more scalable, less prone to error, and easier to test the fully upgrade system with OP Contracts Manager, 2) simplify the process of responding to fault proofs related incidents, and 3) improve the process of triggering the Superchain-wide “pause” functionality without changing any existing security properties.

Specifications

The major features in this upgrade were previewed in mini-posts to encourage earlier community discussion. Links to each discussion are attached in the relevant section.

Blockspace Charter

Technical Details

OP Contracts Manager and Maintenance Changes

This proposal was previewed here.

  • The specific protocol changes included in this proposal are:

    1. OP Contracts Manager (OPCM): Each release will have its own OPCM that can deploy new proxies and upgrade existing OP Chains. It is important to note that the OP Contracts Manager is not considered to be part of the protocol; it has no special role, and no contracts reference it. It is more like an on-chain script for orchestrating deployments and upgrades.
    2. In-protocol Contract Modifications: The changes to the L1 contracts covered by this proposal are relatively minor. They help to improve our engineering process, but do not affect governance approved features and functionality:
      1. Stack too deep fixes in three PRs: 1, 2, 3. These changes enable us to generate code coverage measurements.
      2. Contracts have been updated to call interfaces for external interactions rather than implementations (example PR). This greatly reduces the compilation time and speeds up our development process.
      3. Removal of CustomGasToken logic in two PRs: 1, 2. The CustomGasToken functionality had not been governance approved nor deployed, but was included in the contracts in the monorepo before the decision was made to remove it.
      4. Changes to deposit transaction aliasing (PR): This change ensures that addresses which are currently aliased in a deposit will still be aliased after the L1 Pectra upgrade’s introduction of EIP-7702.

    All of the changes above have been audited.
    Minor fixes resulting from that audit are finalized and reflected in the report.

Fault Proofs Incident Response Improvements

This proposal was previewed here.

DelayedWETH

  • DelayedWETH.hold(...) (callable only by the Phase 0 Security Council Account) now executes an approval and transfer from the target account to the owner account instead of simply executing an approval.
  • We provide a version of DelayedWETH.hold(...) that does not require the owner to specify the target’s balance. These changes significantly reduce the complexity of the runbook for executing a DelayedWETH.hold(...) action.

OptimismPortal

  • OptimsimPortal.setRespectedGameType(...) (callable only by the Guardian) no longer sets the respected game type and the retirement timestamp at the same time. We provide a special reserved input value (type(uint32).max) that can be used to set the retirement timestamp.
  • OptimismPortal.checkWithdrawal(...) now asserts that a FaultDisputeGame was the respected game type at the time of creation rather than asserting that the game is currently the respected game type.
  • Combined, these two changes mean that changing the respected game type no longer automatically invalidates all existing withdrawals while maintaining the option to invalidate withdrawals if necessary.

AnchorStateRegistry

  • AnchorStateRegistry.tryUpdateAnchorState(...) is removed and the AnchorStateRegistry.setAnchorState(...) function is repurposed as the primary way for FaultDisputeGame contracts to attempt to update the anchor state.
  • The AnchorStateRegistry’s internal anchor state is now unified across all game types instead of storing a different state per game type. AnchorStateRegistry.setAnchorState(...) now verifies that the game is fully finalized and only allows the respected game type (as defined in the OptimismPortal) to update the anchor state.
  • Changes to the AnchorStateRegistry in total mean that the anchor state cannot become “poisoned” in a way that the state recognized by the AnchorStateRegistry is not recognized by the OptimismPortal.
  • Additionally, the AnchorStateRegistry is updated so that all OP Stack chains can share a common AnchorStateRegistry implementation.

FaultDisputeGame

  • The FaultDisputeGame is modified to support the concept of “bond refunding” whereby bonds are automatically distributed back to their original depositors if the game is invalidated either by the retirement timestamp or the blacklisting mechanism. This reduces the need to actively step in to redistribute bonds if necessary.

Supporting Documentation

Audit Reports and Findings

Offbeat Labs

Offbeat Labs completed an audit of this code and found 1 Low Severity issue which has been addressed.

Spearbit

Spearbit completed an audit of this code, but the report is not finalized as of the publication of this post. Spearbit found 1 Medium Severity and 2 Low Severity issue.

The Medium Severity issue did not represent any safety risk - a deliberate design decision was found to conflict with the updated L2Beat Stage 1 requirements that were published at the end of January 2025, after the component was designed. We are updating our design process going forward to take these new L2Beat Stage 1 requirements into account. We have since modified the design of the component to satisfy the updated L2Beat requirements.

DeputyPauseModule (Superchain Pause Improvements)

This proposal was previewed here.

DeputyPauseModule

  • The DeputyPauseModule is a new Safe Module that we are proposing to install into the Optimism Foundation Safe. The DeputyPauseModule allows the Optimism Foundation to assign a “Pause Deputy” private key that can be used to create signatures that authorize the use of the Superchain-wide pause.
  • The DeputyPauseModule allows the Pause Deputy private key to cause the Optimism Foundation Safe to execute a call to the DeputyGuardianModule account ONLY for the purpose of executing the pause function. The Pause Deputy has no other capabilities.

Supporting Documentation

Audit Reports and Findings

Radiant Labs

  • Report
  • Low/informational findings only, all addressed

MiloTruck (independent)

  • Report
  • Low/informational findings only, all addressed

Impact Summaries

OP Contracts Manager

The adoption of this proposal will improve the safety and reliability of upgrades. This should in-turn enable smaller and more frequent upgrade to occur, thus increasing the speed of iteration and improvement.

Fault Proofs Incident Response Improvements

IMPORTANT: If adopted and deployed, this proposal will cause a one-time invalidation of all pending withdrawal proofs created on L1. Users would be notified ahead of time to complete any pending withdrawals before the upgrade is executed and to avoid creating new withdrawal proofs that would not become executable in time. This aspect of the proposal does not place any ETH or ERC-20 tokens at risk and would simply require the user to submit a second withdrawal proof transaction on L1 in the worst case. This type of invalidation has already occurred in previous upgrades (e.g., when fault proofs were first introduced).

DeputyPauseModule (Superchain Pause Improvements)

  • Adopting this proposal will significantly improve the Optimism Foundation’s ability to quickly respond to incidents while simultaneously eliminating a major source of overhead during the upgrade process (re-signing many pre-signed pause transactions).
  • The Optimism Foundation Safe is expected to rotate the Pause Deputy private key on a regular basis (approximately every 3 months).
  • Usage of a private key in place of the existing “pre-signed” pause transactions does not change the existing security model. In either case, the Optimism Foundation retains access to secret material that can quickly trigger the Superchain-wide pause in case of an emergency.

Corresponding to this change, PR#46 adds a new Governing Policy for the Deputy Pause Module to the Standard Rollup Charter.

Precommitment Impact Review

This Upgrade Proposal does not impact any of the precommitments in the Standard Rollup Charter.

  • Collective Fee Take: no modification or onchain implementation introduced.
  • Governor/Servicer Role Separation: no change to role structures or authorization patterns.
  • Ossified GasLimits: no changes to ossified gasLimits
  • Direct Fee Margin Controls: no change to the relevant configuration structure and authorization. Some helper methods for get/set logic are added to the SystemConfig, but do not fulfill the overall intent of this precommittment.

Action Plan

If this proposal is accepted, the upgrade is expected to take place on April 7, 2025. In that case, mulitisig ceremonies will be coordinate such that the following transactions will be executed.

  • Upgrade transaction commitments

    All of the following data will executed by the relevant multisigs as a DELEGATECALL to Multicall3DelegateCall. If target and calldata in the runbooks prepared for executing the upgrade do not match, they should not be executed.

    1. To upgrade OP Mainnet, Soneium and Ink

      0x82ad56cb000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000026b2f158255beac46c1e7c6b8bbf29a4b6a7b76000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000164ff2dd5a100000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000003000000000000000000000000229047fed2591dbec1ef1118d64f7af3db9eb290000000000000000000000000543ba4aadbab8f9025686bd03993043599c6fb04039facea52b20c605c05efb0a33560a92de7074218998f75bcdf61e8989cb5d90000000000000000000000007a8ed66b319911a0f3e7288bddab30d9c0c875c300000000000000000000000089889b569c3a505f3640ee1bd0ac1d557f436d2a039facea52b20c605c05efb0a33560a92de7074218998f75bcdf61e8989cb5d900000000000000000000000062c0a111929fa32cec2f76adba54c16afb6e8364000000000000000000000000d56045e68956fce2576e680c95a4750cf8241f79039facea52b20c605c05efb0a33560a92de7074218998f75bcdf61e8989cb5d900000000000000000000000000000000000000000000000000000000
      
    2. To upgrade Base

      0x82ad56cb000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000026b2f158255beac46c1e7c6b8bbf29a4b6a7b760000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a4ff2dd5a10000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000100000000000000000000000073a79fab69143498ed3712e519a88a918e1f40720000000000000000000000000475cbcaebd9ce8afa5025828d5b98dfb67e059e039facea52b20c605c05efb0a33560a92de7074218998f75bcdf61e8989cb5d900000000000000000000000000000000000000000000000000000000
      
    3. To upgrade Unichain

      0x82ad56cb000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000026b2f158255beac46c1e7c6b8bbf29a4b6a7b760000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a4ff2dd5a100000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000001000000000000000000000000c407398d063f942febbcc6f80a156b47f3f1bda60000000000000000000000003b73fa8d82f511a3cae17b5a26e4e1a2d5e2f2a4039facea52b20c605c05efb0a33560a92de7074218998f75bcdf61e8989cb5d900000000000000000000000000000000000000000000000000000000
      

As this is a L1 contracts-only upgrade, no action is required by node operators.

To prepare infra and the community for withdrawal flow changes, we will be rolling out mainnet messaging starting next week.

Communication will include:

  • Comms: At least two rounds of public messaging on OP social channels notifying users of the changes to withdrawal flow, and of the impact on withdrawals during the upgrade window.
  • Contributors: Coordinate with bridge infra contributors, including maintainers of popular frontends to the standard bridge, to post banners notifying users of the changes.
  • Support: Prep developer support team for inbound support tickets about transactions that need to be reproven due to not finishing the first proof in time before the changes.
  • Bridges: Notify bridges and other infra contributors whose operations might be impacted due to the withdrawal flow changes.

If a critical security issue is discovered before upgrading, OP Labs will collaborate with the community to extensively communicate that the upgrade will no longer occur.

Emergency Cancellation

The optimistic mainnet releases will be activated on the above mentioned date. If there is a critical security issue found between approval and rollout, the Optimism Foundation and Security Council will work to coordinate an emergency cancellation.

Conclusion

This proposal outlines more robust incident response processes and technical improvements to contracts and their management. This network upgrade provides simplified processes, allows smaller and more frequent upgrades to occur, and enables more flexible and less disruptive ways to respond to potential incidents in the OP Stack fault proof system. We hope to gain your support for the implementation of these changes.

12 Likes

It’s clear that OP Labs and the Security Council will collaborate on emergency cancellation if needed, but it would be good to explicitly define who makes the final decision (Optimism Foundation, Security Council, or both?).

While the proposal clarifies most of the technical side, some governance-related question on this (probably not for you maurelian):

  1. Process:
  • Will there be public confirmation/logging of key rotations?
  • Is there a designated team within the Foundation responsible for managing this process?
  1. External delegation:
  • If OP Labs or another entity gets access to the Pause Deputy key, how will responsibilities be defined?
  • Will the Security Council have visibility over these decisions?

How much time users have to react before the upgrade?

These are all minor questions and I’m fully supportive of this proposal.

6 Likes

The SEEDGov delegation, as we have communicated here, being an Optimism delegation with sufficient voting power, we believe this proposal is ready to move towards a vote.

2 Likes

I am an Optimism Top 100 delegate [Delegate Commitments - #65 by mastermojo ] with sufficient voting power, and I believe this proposal is ready to move to a vote

1 Like

I am an optimism delegate with sufficient voting power and I believe this proposal is ready to move to a vote.

1 Like

GM @maurelian ! This link seems broken:

The World Foundation delegation, in its capacity as an Optimism delegate with sufficient voting power, believes this proposal is ready to move to a vote.

I am an Optimism delegate with sufficient voting power and I believe this proposal is ready to move to a vote.

Corrected link is here: optimism/docs/security-reviews/2024_12-DPM-RadiantLabs.pdf at develop · ethereum-optimism/optimism · GitHub

Sorry about that!

1 Like

I am an Optimism delegate with sufficient voting power and I believe this proposal is ready to move to a vote.

I am an Optimism delegate with sufficient voting power and believe this proposal is ready to move to a vote.

Emergency Cancellation
The optimistic mainnet releases will be activated on the above mentioned date. If there is a critical security issue found between approval and rollout, the Optimism Foundation and Security Council will work to coordinate an emergency cancellation.

It’s clear that OP Labs and the Security Council will collaborate on emergency cancellation if needed, but it would be good to explicitly define who makes the final decision (Optimism Foundation, Security Council, or both?).

The Optimism Foundation and Security Council together are the final decision-makers, as they are both blocking signers on the 2/2 L1ProxyAdminOwner.

Process:
Will there be public confirmation/logging of key rotations?

Because key rotations are an onchain action, they are naturally logged.

Is there a designated team within the Foundation responsible for managing this process?
If OP Labs or another entity gets access to the Pause Deputy key, how will responsibilities be defined?
Will the Security Council have visibility over these decisions?

The Optimism Foundation’s leadership team is responsible for managing this process, with specific incidence response playbooks delegated to the OP Labs security team. The Security Council will be informed of these decisions, though the current playbooks prioritize taking immediate action to favor safety over liveness, so this information/decision will not come in real time.

2 Likes

The DAB assessed this proposal and happily recommends its approval.

3 Likes

Could someone from OP Labs confirm if this is the correct report mentioned as made by Offbeat Labs? Thanks

Hi, BOB team here, we are one of the top 100 delegators.
In response to this proposal we are happy with the OPCM + Fault proof response improvement, however can you shed some light on the “DeputyPauseModule”?
Specifically,

  • Definition of the “pause” mean in the Superchain context? Does it halt all rollups or just pausing withdrawals?
  • What criteria trigger a pause (e.g., % of funds lost)?
  • How is it unpaused, and how does the Security Council work with chains to restore state?

Thank you

2 Likes

Yes, that is correct.

1 Like

It currently pauses withdrawals for all chains in the superchain, although there exist plans to make this more granular so that subsets of chains can be paused.

The pause will be preemptively, when a bug is identified which puts user funds at risk necessitating an upgrade.

It is unpaused by the security council can call the unpause() function on the SuperchainConfig contract.
I’m not sure about the meaning of “restore state”, but my understanding is that the Security Council would work with affected chains to ensure that it is safe to reactivate withdrawals prior to unpausing.

Appreciate the feedback Maurelian, We have some followup questions:

  • Do you have more granular guidelines or criteria of when they will pause all withdrawals?
    So for example, is a $1k or a $1mm hack? Or is the pause only triggered when they identify a bug in the OP stack code? To me, it’s not entirely clear what class of bugs in terms of likelihood of exploitation, impact and scope would warrant a pause.

  • On restoring state: this would come into play if the security council pauses withdrawals due to an issue at with an oracle provider. Let’s say an oracle provider and a DeFi protocol loose tons of user funds due to a hack or misconfiguration. The chain in question wants to hard fork and roll back their state to before the hack. How would the security council act?

Thank you

3 Likes

The following reflects the views of L2BEAT’s governance team, composed of @kaereste and @Sinkas. It’s based on their combined research, fact-checking, and ideation.

We voted FOR the proposal.

The new emergency procedure outlined in the proposal aligns with our Stage 1 requirements, so we don’t see a reason to be against it. Our research team reviewed the OP Contracts Manager to make sure the contracts were set up as intended and found no issues.

3 Likes

LobbyFi voted FOR since there was one lobbyist who bid for FOR option. His bid exceeded the threshold, which is at 10% of the instant buy which has been chosen for this proposal.