A reoccurring request by badgeholders has been to access more third party data on projects’ impact, instead of solely relying on information projects have provided themselves. For many offchain contributions, we’re missing standardized reliable data which can be surfaced to measure impact. The hypothesis behind this small-scale experiment is that attestations by governance actors on the impact of Governance Infrastructure & Tooling projects will support badgeholders in making more informed voting decisions in Retro Funding 6.
Overview
In Retro Funding 6, the Collective will experiment with collecting data on the impact of different governance contributions via attestations. The goal is to produce standardized verifiable data which provides value to badgeholders.
- Timeline: Impact Attestations will be collected from Oct 14th til Oct 23rd
- Scope Categories: Impact attestations as part of this experiment are limited to contributions within the “Governance Infrastructure & Tooling” category.
- Participation: Top 100 Delegates, Citizens and governance leadership functions will be asked to participate in issuing impact attestations. It’s essential that participants use addresses which hold relevant reputation, such as delegate or citizen role. For the output of this process to be meaningful, the experiment should have participation from at least 40 Citizens, 40 Top 100 delegates, and various governance leadership functions. Participants should not attest to the impact of their own projects.
Experiment Output
The aim of the experiment is to generate relevant insights about the impact of Governance Infrastructure and Tooling to support the Retro Funding 6 voting process. Insights derived from the attestation data will be served to existing badgeholders on the relevant project page. These insights will not be surfaced to guest voters, to not impact the experiment conditions of the voter selection experiment. You can see an early iteration on how this information could be surfaced to badgeholders below.
To measure the success of the experiment, badgeholders will be asked to rate the value this information provided in the ballot submission survey.
Issuing Impact Attestations via Impact Garden
Users will be able to generate Impact Attestations via Impact Garden to ensure the data gathered is standardized and enables comparison. Impact Garden’s work has significantly shaped the design of this experiment, you can read more about it here.
Impact Attestations have two core components:
- Reputation
- Qualitative data standardization
Reputation
- Farcaster account: we will leverage the Farcaster account of the rater to identify their roles within the Optimism Collective, both past and present. This is done by querying all attestations issued to any of their verified wallets.
- Bonus: Using the Farcaster username reduces the possibility for sybil attacks as we can leverage the social graph and information of the attestors, while creating a loop in which we enrich that same social graph with information on governance participation.
- Onchain attestations: Each Impact Attestation will reference the project being attested to onchain. By doing so, we continue to increase the onchain information available on the reputation of contributors to the Optimism Collective.
Qualitative data standardization
Every reviewer will be asked to answer 3 questions to ensure their views can be aggregated and analyzed in a comparable way. Measuring these metrics over time will also enable us to identify if the tool has improved or decayed and the continued relevance of it to the members of the Collective.
- Q1. How likely are you to recommend this tool to someone in your role or position?
This question is akin to the Net Promoter Score measurement, through it we’re looking to understand how satisfied members of the Collective are with this tool and the service it has provided them with in the past. We hope that this question will also provide insights into the usefulness of a tool beyond the dependency it has for our systems. - Q2. How would you feel if this contribution ceased to exist?
Through this question, we look to understand the dependency governance members of the collective have to these different tools to perform their day to day functions, or their role specific functions (Retro voting, etc). - Q3. Brief explanation for your rating
We have included an open ended question to allow raters to share additional valuable information on their answers to question 1 and 2. This information can surface particular benefits that have been derived and allow for more precise tuning for voters decisions.
Simple vs complex impact relationships
Governance infrastructure & Tooling projects are a good fit for this experiment, as these projects usually have a direct user relationship, which can represent their impact. One can measure the impact of a governance tool by understanding how much utility it has provided to actors within the governance system.
This is not true for all governance contributions. Measuring the impact of a Council, such as the Grants Council, is more complex, requiring multiple types of data that increase the complexity of the experiment.
A look forward
This experiment marks a first step into the frontier of generating standardized data to measure the impact of offchain projects. Below are principals which guide the experiment and possible future iterations:
- Rich reputation graph: It is essentially to understand who the users are that are deriving impact from a project and what their role is within the Optimism Collective. This requires the population of a rich reputation graph. The Collective already has made significant progress within this area, issuing attestations for relevant governance roles and building out builder information within OP Atlas.
- Broad participation: For the collected data to be high signal, reliable, and robust, the process requires broad participation of actors within the Collective.
- Integrated: signals on impact, such as impact attestations, should become integrated into the experience of using a project, rather than being an activity in itself. At its best, the user is not required to take any additional actions to signal impact. This aids in ensuring broad participation and even coverage.
. Enjoy this 
