Leader Election Proof of Concept

May 31, 2023
Open
Open
Foundation Mission
  • Links

Foundation Mission (RFP) – Leader Election Proof of Concept

To take on this project, submit a proposal to this thread by June 28. Read more about Missions here.

  • S4 Intent: Progress Towards Technical Decentralization
  • Proposal Tier: Ember
  • Baseline grant amount: 30k OP
  • Accepting one or multiple proposals: Multiple
  • OP Labs or Optimism Foundation Sponsor: Mark (@tynesol)
  • Submit by: June 28th at 19:00 GMT
  • Selection by: July 13th at 19:00 GMT

Background

What is Leader Election?

Leader election is simply the ability to have different leaders that are responsible for creating the next canonical state transition in a distributed system. In a blockchain, leader election gives the ability for different block producers to produce blocks at different times. Leader election algorithms can be racy or race free.

A racy leader election algorithm is where many potential leaders are competing at the same time to be the leader, this is the case with proof of work. A race free leader election algorithm is when there is a single known leader at a particular point in time and there is no other way for a different potential leader to be the leader at that time, this is the case with Ethereum Gasper.

With Proposer-Builder Separation, the block builder (also known as a validator or a proposer) is virtualized in the sense where a network of block builders that are ultimately responsible for selecting the ordering of the transactions and then the proposer is simply responsible for signing off on the block. This turns the single entity that was responsible for producing a block at a particular slot into many possible entities that are all competing for building the most profitable block during the slot, so the raciness has returned because of MEV.

It is very difficult to understand the various second order effects of different leader election mechanisms for an op-stack chain. Leader election is desired as a feature because it can be used to make sequencing more decentralized. It does not guarantee that sequencing is more decentralized, so be careful when thinking about the coveted “decentralized sequencer”.

What is Decentralized Sequencing?

You could define a decentralized sequencer as 1) permissionless entry to proposer set, 2) some weighted probability of being elected leader based on a costly signal. Decentralized sequencing is a spectrum. Introducing PBS can slightly decentralize sequencing, but it will not give the property of censorship resistance because the single sequencer has the right to censor any block that they choose. Decentralized sequencing gives the property of censorship resistance, as if transactions are being censored then its possible for new sequencers to join the network and begin to include transactions that were previously being censored.

What is the MVP for leader election?

Allow the batch inbox to be a smart contract and check in the derivation pipeline that the transactions do not revert. This would allow developers to experiment with leader election mechanisms that are based on an L1 smart contract. This would require a few changes:

  • Gas estimation when creating the batch instead of computing the gas limit client side based on the intrinsic gas in the batcher
  • Batcher needs to be able to know when its the leader and when its not, so create a standard interface that all smart contracts used as the batch inbox must implement, so the batcher can submit batches only when it is its turn
  • Modifying the [derivation pipeline](https://github.com/ethereum-optimism/optimism/blob/edb6603feb5f49f2fdfe41f4b39c91695ed81ff8/op-node/rollup/derive/calldata_source.go#L107) to ensure that reverted transactions are not included, the receipts are necessary for this in the linked context
  • Remove the check that the tx.from is equal to the batch sender when the [batcherHash](https://github.com/ethereum-optimism/optimism/blob/develop/specs/system_config.md#batcherhash-bytes32) is version 1. The version is just the first byte, I don’t see a reason as to not just use bytes32(1) although there may be useful information that could be included in the bytes following the version byte
  • Pull the number of L2 blocks per channel into consensus and make it very easy to determine when channels open/close from the smart contract
  • A simple smart contract that implements a permissioned set of leaders and round robins between them as a proof of concept
  • Nice to have: add the batch inbox to the system config with a setter so it can be modified via governance and emit an event when it changes so the leader election mechanism can be dynamically changed by governance
  • Nice to have: a way for the fee recipient to be able to be set by the current sequencer, right now it is deterministic and always set to a predeploy

A contract that is a batch inbox should not read the full batch into memory, it should simply implement a fallback function and then do logic based on the msg.sender and be able to track channels. Once certain design patterns are proven out in the smart contract, they can be pulled into the derivation function similar to “precompiles” and then op-stack users can configure which sort of leader election mechanism they would like. Experimentation for leader election can now happen outside of the core protocol in parallel by anybody that wants to work on it.

How will this RFP help the Collective progress towards technical decentralization?

Leader election opens up the ability for multiple entities to act as a sequencer and produce canonical blocks. When there is only a single sequencer, there is risk that this sequencer can use its privilege to extort users through censorship or toxic MEV extraction. When there are multiple sequencers, the chain is as censorship resistant as the least censoring sequencer. Due to the complex second order effects of leader election, it is difficult to know for sure the ideal mechanism. This RFP is specifically designed to allow many possible leader election mechanisms to be expressed in smart contracts, so that teams can work on them in parallel outside of the core protocol. Then over time we can “enshrine” the mechanisms that work the best into the core protocol itself. The best mechanisms will be able to handle toxic MEV extraction and make it less of an issue for end users.

What is required to execute this RFP?

Minor changes to many parts of the codebase is required for this RFP to be implemented fully. Ideally, this code is behind a hardfork flag so that the code can still sync existing networks. See the “What is the MVP” section above for details

What milestones should be tracked to determine successful completion of this RFP? How should the Foundation measure progress towards this RFP?

  • Derivation pipeline changes as described in “What is the MVP”
  • Batcher changes as described above
  • Optimism system smart contract changes for the system config
  • Batch inbox smart contract that implements round robin
  • Running the local devnet with multiple batchers running, both using the same op-geth

The bullet points are the general chunks of work involved in the project and can act as milestones for development. Progress towards the completion of this RFP should be based on working code where working is defined as not production ready (not optimized, no metrics, not audited or thoroughly tested) but operating as expected based on enough test coverage to see it working and a final deliverable of a demo on the local devnet where it is obvious that multiple batchers are working in parallel.

How should the community measure impact upon completion of this RFP?

The community should measure impact of this RFP based on it removing one of the centralization bottlenecks from the system. The system is as centralized as the most centralized aspect. The lack of fault proofs is the most centralized aspect (a single party can rug the network) so we are working on that, but the lack of leader election will be the most centralized aspect of the system after fault proofs ship.

This also opens up the design space for people to experiment with leader election mechanisms, which will be very fun. We can imagine some experiments but guarantee people will come up with stuff we never imagined.

Application instructions

To apply for this RFP, please complete the form in the expandable section below and leave your response as a comment on this issue thread below. Submissions will be open until June 28, at which time the Foundation will review all submissions and select up to three individuals/teams to complete the work defined here.

Submission form

Copy the entire application below and leave a comment on this issue with your answers completed. A representative from the Optimism Foundation may reach out using the contact info provided to request more information as necessary.

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier

  • Alliance Lead: Please specify the best point of contact for your team
  • Contact info:
  • L2 recipient address:
  • Please list the members of your Alliance and link to any previous work:

Read more about Alliances here

What makes your Alliance best-suited to execute this Mission?

  • [...]
  • [...]

Please describe your proposed solution based on the above Solution Criteria (if applicable):

  • [...]
  • [...]

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each peice of work:

  • [...]
  • [...]

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

  • [...]
  • [...]

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

  • [...]
  • [...]

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

  • [...]

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

  • I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
  • I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant
  • I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual
  • I confirm that I have read and understand the grant policies
  • I understand that I will be expected to following the public grant reporting requirements outlined here

-- end of application --

  • More Information
  • Tag 1
  • Tag2
  • Tag3