Collective Trust Tiers were introduced in Season 4 as a way to allow teams to build reputation onchain and create a common standard for grant eligibility. Although limiting grant amounts based on a teamās reputation is still a valuable concept, the implementation of Trust Tiers has revealed some limitations. As such, the Trust Tiers program will see some significant changes starting in Season 6 and beyond. These changes are informed by the experiences of builders and the Grants Council, and adhere to the principle of iterative governance design and experimentation.
Trust Tiers - what we learned in Seasons 4 and 5
These learnings derive from a survey with five members of the Season 5 Grants Council and four builders, as well as several informal conversations with Grants Council members.
The Grants Council rated the concept of Trust Tiers as neutral, but rated Trust Tiers in their current form as not very useful. Here are the reasons given:
- Trust Tiers do not accurately reflect teamsā reputations. Some teams seem āovervaluedā and others āundervaluedā based on their Trust Tier. Specific limitations called out:
- Reputation gained in other ecosystems isnāt transferable to Optimism.
- There have been many edge cases where teams pivoted and lost their Tier, or trusted contributors left a project that retained its Tier.
- Trust Tier status gained through Retro Round 3 was called out as seemingly inaccurate or adding noise to the data.
- Trust Tiers are very hard to verify, making them difficult to use effectively.
- The Grants Council must rely on a projectās self-reported Trust Tier until the final stages of a grant application, when they verify the self-reported Trust Tier.
- This is because the operational overhead of checking each applicantās Trust Tier up-front is too high.
The builders surveyed were neutral on Trust Tiers, mostly understanding the concept and aware of the Trust Tier their project is in. The following limitations were highlighted:
- Teams arenāt always clear how to calculate their Trust Tier when theyāve had multiple grant types or they fall into a gray area.
- Trust Tiers may benefit from including other reputation markers beyond the team having previously received a grant.
Diagnosing the problems with Trust Tiers
The primary issue with the Trust Tiers is not that the reputation of teams doesnāt matter in grant decisions, but that the Tiers have not been an effective method of predicting whether a team would be successful at completing a Mission Request. In addition, manual verification of the Tiers is inefficient and may lead to confusion.
Therefore, our objective should be to build an alternative reputation model that can more effectively predict whether a team will successfully complete a grant, with the ultimate goal of reducing the number of incomplete mission requests and grants clawed back and increasing the successful outcomes of completed grants.
The initial implementation of Trust Tiers was based on subjective contributions to the Optimism Collective for each Tierās qualification. These criteria are composed of trust signals like āhas previously worked for the Optimism Collectiveā or āhas earned over a certain amount of OP in Retro Fundingā. Each Tierās set of criteria was a best guess at which signals might be relevant.
The next generation of Trust Tiers should be built bottom-up, using the underlying trust signals as training data to build a machine learning model that effectively predicts a teamās likelihood of success. This approach is more likely to yield Tiers that effectively segment teams by reputation. It may also reveal that a continuous trust score (e.g. a scale of 0-100) is more useful than dividing teams into levels.
Trust Tiers in Season 6
To re-build the Trust Tiers bottom-up, the first priority is to grow the number of trust signals available about projects. Only then can the Collective attempt to build a model or models that effectively categorize or score teams.
Since the Trust Tiers do not appear to be working in their current form, they will no longer be used in Season 6. The Grants Council will make grant decisions based on the information that is available to them about the teams applying.
Instead, the Collective will focus on developing the trust signals that underlie the Tiers, and record these as attestations. As part of this:
- All teams doing work in the Collective as part of Retro Funding or Mission Grants will be asked to create a persistent project profile, to which they collect attestations.
- Teams working on Foundation and Token House Mission Requests will receive attestations upon grant completion.
- Attestations will include the OP amount and season in which the grant was issued.
- Retro Funding awards will also be represented as attestations issued to the projects that applied. OP amount and Round are included in the attestation schema.
- The owners of user interfaces that include project profiles are encouraged to display relevant project attestations.
This is one of several initiatives the Foundation is bootstrapping in Season 6 to equip the Collective to be able to make data driven decisions. In future seasons as the project model gains more usage, the Collective can continue to develop the number and variety of trust signals available.
Please see the governance docs for more detailed information on how projects and people are represented in the Collective.