Retroactive Public Goods Funding (Retro Funding) 6 was recently announced, and will award OP to contributions made to the Optimism Collective governance system. As part of Round 6, a set of Guest Voters will be invited to participate alongside Citizens. Similar to the experiment in Round 5, the goal is to learn more about different voter selection mechanisms as a way to create a foundation of knowledge that can be used when making decisions about Citizen selection in the future.
Research Questions
There are two primary research questions inspiring this experiment:
- Do selection methods that aim to select for specific characteristics (such as Web of Trust or Proof of Work) result in a significantly different group composition than random selection?
- Do randomly selected voters make resource allocation decisions that are significantly different from voters selected via Web of Trust / Proof of Work? Are we able to assess those differences to be better / worse?
The first research question derives from a hypothesis that selection methods are better than random chance at selecting for specific characteristics. We currently have two cohorts of voters that were selected with a specific selection method: Citizens that were selected via Web of Trust and Round 5 Guest Voters that were selected via Proof of Work. Without a control group, we cannot make any assumptions about the effectiveness of these methods in selecting specific characteristics. Randomly selecting a set of Guest Voters enables us to create this control group and use this group as a baseline against which to compare the two previously selected groups as well as future groups.
The second research question goes a level deeper to try and understand whether deliberately selected voters actually perform differently on the allocation task than a random group of people. Once again the current experiment can be thought of as the creation of a control group against which to compare Citizens. Having a control group is valuable as it allows us to draw conclusions about the effectiveness of the ‘treatment’. In this case, the treatment is ‘being selected via Web of Trust’. If the control group is found to vote in a similar way to Citizens, then there is no effect of ‘being selected via Web of Trust’ on the outcome, which is the Retro Funding allocation.
Experiment Outcome Measures
Research Question 1
Do selection methods that aim to select for specific characteristics (such as Web of Trust or Proof of Work) result in a significantly different group composition than random selection?
The following outcomes will be measured to address the first research question:
- Social graph modularity of Guest Voters
- Professional background / occupation of Guest Voters
- Engagement and alignment of Guest Voters with the Optimism Collective
These measures will be analyzed partially through data associated with Guest Voters’ Farcaster Accounts and linked Ethereum accounts, and partially through data collected in an onboarding survey. The outcomes will be compared with similar measures made for Round 5 Guest Voters and Citizens.
Research Question 2
Do randomly selected voters make resource allocation decisions that are significantly different from voters selected via Web of Trust / Proof of Work? Are we able to assess those differences to be better / worse?
The following outcomes will be measured to address the second research question:
- Voting behavior (time spent voting, number of metrics selected etc.)
- OP allocation patterns (allocation distribution curve, min and max allocation, median allocation, delta between allocation and ratings provided in impact attestations)
- The votes of Citizens and other stakeholders on which overall allocation was perceived to be ‘better’ that of the Guest Voters or Citizens.
Experiment Operations
In order to address the research questions above, a set of voters must be randomly selected. There are a few high-level requirements of this random selection mechanism:
- It mustn’t select the same person twice (i.e. one-person-one-vote)
- It must select from the general population of Superchain users
- There should be as little bias in the sample as possible
The first step of random selection is defining the population. Once the population is defined, a random number generator can be used to randomly select a set of individuals.
There are a few ways to think about defining the population. One is to make an announcement allowing anyone to register their interest, and then randomly select from the set that registered. However, this approach introduces a lot of bias into the sample because the population to be sampled from is skewed towards individuals that are highly engaged with Optimism.
A preferable way to randomly select participants is to define the population as broadly as possible, randomly select individuals and reach out to them with an invitation to participate. There will still be some bias in the sample, but considerably less than if the opt-in process happens prior to the random selection.
The goal is therefore to define a representative population of Superchain users, where there is a low probability that the same person is selected twice, and there is the ability to reach selected individuals with an invitation to participate. After reviewing several options, we decided to sample from the population of active Farcaster accounts. This is because:
- The Farcaster Protocol runs on OP Mainnet, so Farcaster users are users of the Superchain
- It is possible to reach any Farcaster user via cast or DM
- The population is large enough - all Farcaster accounts is ~750,000 users, with ~50,000 being active daily
- The population is likely representative of the larger set of active Superchain users - creating a social account is a common action and the set is unlikely to be skewed towards any specific characteristic
- While some users have multiple Farcaster accounts, the rate of user:account is probably lower than the rate of user:Ethereum account
Random sampling process
- Starting with all Farcaster accounts that were created…
- Filter out business accounts (we want people, not organizations)
- Filter out existing Citizens and Round 5 Guest Voters
- Filter out accounts that have had zero Farcaster activity in the last 3 months
- Run a script to randomly select voters
- Reach out to voters with the invitation to participate
- Run the script again to select new voters to fill the remaining unfilled slots until the target number of Guest Voters has been reached
What about Sybils?
While the probability that two accounts belonging to the same person are selected in a set of tens or hundreds of thousands of accounts is low, we must still consider Sybil-resistance in the experiment design.
All Guest Voters are required to participate in an onboarding process that includes a video call requiring active participation. Voters that don’t participate will not receive a voting badge.
Experiment Design
Retro Funding 6 will see the participation of a record number of Guest Voters. Approximately 100 Guest Voters will participate alongside roughly the same number of Citizens. The purpose of inviting such a large cohort of Guest Voters is to maintain experiment conditions that have a similar number of participants.
While Guest Voters will participate in the allocation task under the same conditions as Citizens, at the end of the round their votes may be counted differently.
Citizens will be able to review the aggregated voting results of Guest Voters and cast a final vote on whether to integrate the votes of Guests at a 1:1 ratio, or whether to diminish the aggregate voting power of the Guest cohort by 25%, 50% or 100%. This process will act as a fail-safe in case the system is captured by Guest Voters.