Retro Funding 5: Voting Rationale Thread

A place for badgeholders to share their voting rationale for Retro Funding 5.

Relevant resources

3 Likes

Firstly I want to acknowledge the effort and dedication of all applicants, for their contributions in advancing public goods in the space.

Category I was assigned is Ethereum Core Contributions

Voting Rationale

I decided to allocate the full 8 million OP available for this round to maximize rewards for the most important contributions to our ecosystem.
I used a top-weighted allocation method, making some changes to fit my perspective better. I gave priority to projects that hadn’t received RPGF before. For projects that had already received RPGF, I wanted to see significant progress. eg rewarding impact progress a level 10 to 20 moment since RPGF round 3, as I felt round 3 already covered the early stages of impact ‘1 to 10’.

Voting Experience

The user interface gets better with every round, and I appreciate all the work put into making the voting process simple and straightforward.

I’m grateful to be part of this process, in rewarding valuable contributors of the Ethereum ecosystem.
Looking forward to RPGF round 6

3 Likes

First off, here’s my old but still accurate disclosure. With that out of the way, here’s how I voted:

Total Budget

8mm OP (maximum)

Why? Projects in RPGF 5 are existential for Optimism ecosystem. See my argument for not reducing budget here.

Allocation

  • Ethereum Core Contributions: 50%
  • OP Stack R&D: 35%
  • OP Stack Tooling: 15%

Why? Although I personally believe there are a handful of projects in the “Ethereum Core Contributions” category that do not belong there, I chose to ignore that and weight the allocations based on how existential each category is to Optimism as a whole.

Allocation Method

I used a custom allocation method using my own spreadsheet, see here.

See my reasoning for applying deductions for projects with revenue and grants here.

Post-Submission Q&A

How much time did you spend on voting in this round (in hours)?

20 hours

Please rate the voting experience

7

I was plagued by an annoying bug which I spent hours trying to fix myself (thinking it was an issue on my side), would appreciate better testing next time. Though to be fair, the bug was fixed in ~12hrs (on a weekend no less).

Also “Allocation method” didnt end up being useful to me (apart from inspiration), as I used my own spreadsheet to create my weights. But glad I was able to import CSV!

Overall despite the bug, better voting experience than RPGF 4, glad we got back a little bit more control over our ballots.

How worried are you about detrimental behavior among badgeholders influencing the allocation of Retro Funding in this round?

4 (somewhat worried)

It is a worry, though I have no evidence that it’s happening nor did anyone attempt to bribe me.

Given the design of this round, how confident do you feel that OP rewards will be allocated efficiently to the most deserving projects?

3

Funding per project is capped, without taking into consideration the # of recipients. RPGF therefore biases towards average project size, meaning its beneficial when you have the smallest number of beneficiaries. Thus I think projects with smaller headcount will receive disproportionately higher rewards.

I also strongly believe that grants / revenue should be deducted, and think most badgeholders probably wont do that.

To what extent did the “Grants and investment” information influence your token allocation among projects?

7 (had a large influence on my token allocation)

I strongly believe that grants / revenue should be deducted, so I used this information heavily.

6 Likes

My voting rationale can be found here.

(Reflections on the review process are in the previous post, here.)

3 Likes

First – a requisite thanks to the OP team and fellow badgeholders for another smooth and impactful round of retrofunding! disclosure: i contribute to gitcoin, which developed the voting UIs (in partnership w agora on API and op on design.)

Voting Rationale
I decided to allocate 5M OP out of the 8M. I felt the lower overall quantity of applications justified this. I also wasn’t wowed by the quality of some applications. I would love to receive a higher quantity or quality of apps in future rounds and increase the budget.

Budget Allocation
Another disclosure: I did a ton of QA of the voting UIs and by creating a bunch of test wallets/ballots, I was exposed to each of the three categories. I submitted guest voter ballots to Eth Core Contributions (as QA) and OP Stack tooling (started as QA, but then was intrigued by the badgeholder conversation on this topic and re-did the ballot seriously.) My test wallets were 0xe406938368062CC09F00C65278d209DE5ac6Dc4C and 0x58338E95caEf17861916Ef10daD5fAFE20421005.

My budget allocation was informed by taking a look at the projects in those categories. I used the following:

  • 45% Eth core contributions
  • 35% OP stack R&D
  • 20% OP stack tooling

Ballot Allocation
In each category I felt that there were a small number of projects with high impact, a large category of projects that I was glad existed but weren’t world-changing, and a small number of high-potential projects that I didn’t want to cut funding from but didn’t justify a high amount of funding given the structure of retrofunding. I chose the impact groups allocation method as I felt that best matched my mental model. I really liked having the allocation groups in the UI and spent a bunch of time playing with them.

3 Likes

Category: Ethereum Core Contributions

Budget

I kept the budget at 8M OP and weighted more heavily to Eth core & OP Stack categories rather than tooling at this stage in the development of Superchain.

  • Ethereum Core contributions 40%
  • OP Stack R&D 40%
  • OP Stack tooling 20%

Ballot

My allocation was informed by the following

  • Protocol Guild membership is core devs, researchers, testers, protocol support & guild admin
  • Geth is “the other side of the diff” & is the majority execution layer client
  • Solidity team is not part of Protocol Guild (neither are Vyper or Fe)
  • Client teams (including Revm) & RIG are part of Protocol Guild, but I still wanted to provide some retro funding on top of Protocol Guild allocation
  • Grandine was open sourced in March 2024 and recently added to Protocol Guild
  • Account abstraction ERC4337 work is a key building block that is being used to onboard the world
  • Don’t know enough about the various libp2p libraries

Allocation

I started with top weighted curve as wanted Protocol Guild to receive the highest allocation due to their contribution to Ethereum core development. I then switched to custom so that I could modify the percentages.

The per project cap was 12.5% of total round allocation. I gave Protocol Guild the maximum possible. I asked for clarification if I could give higher but wanted to avoid my ballot being invalid. I would have liked to see Protocol Guild be given a higher percentage of the total round allocation.

I then allocated to Geth (as “the other side of the diff” for OP Stack and for being the majority client for the execution layer. This is on top of their share from Protocol Guild. There is an inordinate amount of pressure/stress as the majority client that the team have to carry. As a community we should recognize & reward this burden, whilst striving to remove this pressure with improved client diversity.

I similarly allocated to the Solidity team for their contributions to the language used in contract development for the application layer.

Next up was ERC4337 account abstraction, the building blocks they have created is what we will be onboarding the world with.

After that I allocated to execution layer teams, then consensus layer teams, plus EthereumJS and REVM. I didn’t differentiate based on client share (other than Geth), though I gave a lower amount to Grandine as they only open sourced this year and only very recently joined the Protocol Guild.

I allocated smaller amounts to RIG (they are also Protocol Guild members), Vyper for their contributions, Dappnode & Ethereum on Arm.

I allocated very small amounts to open source libraries such as the p2p libraries. Whilst small, these could make a difference to open source devs.

There was only a small number of projects I didn’t allocate to out of the 30 projects, as the quality was overall high.

Voting experience

Using custom was painful as after changing a percentage, there were seconds of delay before it would be updated. I had to manually modify percentages when I was under/over.

The per project cap should be higher when there are collectives/guilds involved.

Though I am cautious about the risk of double counting where Protocol Guild members are also members of client teams, research etc and would much rather a large allocation just to the Protocol Guild.

3 Likes

This round was really fun to be a part of. Very grateful to have been chosen to participate.

I will detail my experience as follows:

Pros:

  • Overall experience was quite frictionless
  • Ability to use “allocation method” as a starting point on the ballot was very helpful

Cons:

  • Struggled with saving the state after dividing the budget and had to refresh to get it to stick.

Assigned Category: OP Stack R & D

Budget Allocation

Total Budget: 4,000,000 OP

Ethereum Core Contributions: 800,000 OP (20%)
OP Stack R & D: 1,200,000 OP (30%)
OP Stack Tooling: 2,000,000 (50%)

I chose 4M OP as total budget as it seemed a good median between the min and the max. I didn’t want to give the min or the max as I imagine there will be many more rounds like this to come.

For the split I thought ETH Core as a base case needed at least 20%. The rest I divided between R & D and Tooling and gave the stronger weight to tooling. Like the case for ETH Core, I wanted to give a default to R & D and felt Tooling was most important as I think we are at a point where we the most friction in generated impact is a function of the difficulty onboarding both developers and end users as well as the tools available for doing so.

Allocation Method

I understood why all the projects were part of this section and saw benefits and impact from each. The most impactful projects seemed to have significantly more impact than those at the lower end so I chose a top weighted starting point and made individual adjustments from there.

I think the nature of R & D lends itself to generating smaller impact at the beginning and as projects gain momentum and adoption the impact snowballs. I was unable to reconcile / account for this issue in the voting unfortunately.

Key Takeaway

Note: I copied below content from my TG message in the general chat

Something I found challenging was trying to compare one project that received significant OP from earlier grants / rounds and another project that has not.

How can the project that has not compare on impact when they have significantly fewer resources, and arguably need the resources more to make said impact.

It makes me think of top sports teams that perform well so all the best players want to go there so they continue to perform well and it’s this self reinforcing cycle.

Consider a project that received OP prior to said asessment time frame and was able to grow / create impact with that OP prior to said time frame (time n = 1). For this time frame (time n = 2) their ability to create impact is still a function of growth / impact generated by prior allocation of OP (time n = 1).

e.g. brand awareness, network effects, economies of scale, etc…

Overall great experience! :smiley:

3 Likes

Hi Catjam / Meg

Thanks for sharing your reflections!

I just have to ask…

Did you really vote like this?

Because then we just uncovered a bug in the UI… Surely it should enforce that the percentages add up to 100%. :wink:

Hi all

As a voter, I really enjoyed participating in this round. I liked being able to go deep into a specific category but also have a say on some of the other important round parameters. It was (mostly) painless, apart from some last minute fiddling to my ballot. Kudos to the teams that worked on the interfaces! The more intimate groupchat experience was also a plus for me.

(I will save my thoughts on the effectiveness of the round from an allocation perspective for a separate post.)

How I voted

I used my lil spreadsheet to determine my overall budget and allocations. You can see my inputs, although I ended up tweaking things a little bit in the actual UI.

Ethereum Core Contributions

Many strong projects, although many also were well represented in RF3. I allocated 45% here.

OP Stack R&D

Probably the most diverse category, but I strongly believe that high risk - high upside R&D initiatives should receive outsized rewards. I also allocated 45% for this category even though it has fewer projects than Ethereum Core Contributions.

OP Stack Tooling

This was the weakest category IMHO (and also the one that I was assigned to vote on). I ended up allocating 10%.

Here was my comment in the groupchat:

  1. I have found Pairwise to be pretty useful in getting a sense of the mix of projects in our category and getting a sense of how I think they stack up against each other.
  2. One observation is that our category has a decent number of “side project” type contributions, with multiple submissions linked to the same person.
  3. RaaS providers definitely have an important impact on OP. However, I personally have mixed feelings about giving a large amount of RF for these types of projects since they already have strong business models and are not always strongly aligned with OP (or even ETH DA).
  4. In terms of metrics, I think “trusted stars” is probably the more useful indicator for this category, esp for projects that are more documentation-focused. When I star a repo, it means I want to remember it.
2 Likes

Oh gosh… thank you for catching! That was a typo, I promise the UI enforced 100% :slight_smile: Just edited with accurate allocations

1 Like

I am grateful to have a chance to participate as a guest voter. I was assigned to vote for the “OP Stack R&D” category.

Personal background: My project received RetroPGF in a previous round, so participating as a voter gave me a really different and valuable perspective. In particular, it was useful to see how other projects raise grant funding who applied to this round, as this was included in their applications. This is a case of clear benefit from having to do a round of civic duty, I feel it will improve my positioning as a future applicant too.

Category Allocations

Originally I started with roughly equal allocations and 5M, but after a lot of discussions with fellow voters and actually working through many of the submissions, I went with:

  • 3M total
  • 15% Core
  • 35% OP Stack R&D
  • 50% OP Stack Tooling

The main influence in the allocation was the nature of submissions in the categories. I felt there were more “big impact projects” submitted in the OP Stack Tooling category than R&D (the one I was assigned). One of my fellow voters advocated convincingly that the bulk of the round should go towards OP Stack ecosystem to better align with the theme of the round.

I came in with high expectations of the kinds of projects I would have liked to see in the R&D category, but there weren’t as many of them this round as I hoped, which prompted me to reduce the total allocation further. I suspect this is mainly an issue of timing, as technology is quite bursty in development and a lot of things are in-between tech generations right now.

We debated this in our voting workshop, and it seems clear to me that it’s not rational to decide on allocation for a round in a vacuum from the submissions available. These two things should influence each other.

Review Notes

I kept notes for every project I evaluated. I originally aimed to spend 3-5 minutes per project, but some projects took a fair bit more to evaluate. Recurring themes of things I evaluated against:

  • Does the project have compounding effects on other ecosystem components?
  • Is it a core/path dependency, or is it a supplementary dependency for success?
  • Is anyone using this, depending on it, or is it more of an informative result?
  • Have they received other grants, and how does their scale compare to perceived impact? (Minimal/reasonable/excessive)
  • Are they using prior funding to pump the same metrics they’re using to justify impact? (E.g. a defi project that got OP incentive grants used to incentivize TVL, then using TVL to justify impact.)
  • Are they working on something unique, or competing in a fairly busy group of overlapping-impact projects?
  • If it’s code-heavy, review the changes. Some projects are forks with minor changes, while other projects are entirely new things. Tried to load the diff during the impact period to get a better sense of what work of importance was focused on during that period.

Retrospective

  1. The categorization between “OP Stack R&D” and “OP Stack Tooling” seems way too fuzzy, got a mishmash of both in R&D… I think that division is just creating contention about how to allocate the ballots in general. For example, go-libp2p was in “OP Stack R&D” while other libp2p implementations were in “OP Stack Tooling”
  2. I liked the intent behind the attestations/testimonials, but it was tricky because raw numbers were misleading. Some projects had a substantial number of positive testimonials from names I did not recognize and did not correlate to the quality of the project. Seems easy to game at the moment, but we can improve this. “Trusted Stars” metric on Github is an interesting one, that could be applied here too.
  3. Many submissions were too abstract when describing impact. I wanted to see concrete examples of who exactly benefits from the project. For example, lots of things like “We made oracle technology to benefit projects who rely on oracles” rather than listing out actual projects using it, how many users benefit directly from that feature, how much economic activity is enhanced by it, etc. Ideally this would be caught in the badgeholder eligibility review phase, but I imagine more feedback/iteration editing/review cycles would help.
  4. Being assigned to a voting cohort to discuss over Telegram was great. It helped reduce confusion, and clarify our thinking around our approach.
  5. The voting software asked to estimate how much time I spent on the voting process the first time I submitted, then I went back and redid a bunch of reviewing and spent more time on it, and it never asked for revised time. :sob:

Wishlist

I kept notes about annoying parts of the review/vote process and some ideas of how it could be improved in the future (roughly ordered by ease of implementation):

  1. “Save for later” option (rename Skip?)
  2. Inline notes option (kept private)
    I used Obsidian to maintain notes in parallel but would be nice to have it integrated.
  3. Optional Confidence rating? (“Not sure” checkbox)
    There were several cases where the application was vague or required insider knowledge to evaluate better. I did my best but I would prefer to defer to people who are more confident in those cases (ie. reduce the influence of my allocation).
  4. Optional “Feedback to project” for how they can improve their application next time
  5. For code projects, link to commits/activity ranging the impact window, adds context of what was important during that period
  6. Would love a preliminary round where reviewers can ask questions/additional details from applicants, before the final review votes are given.
    This would probably need to happen in the badgeholder eligibility phase. I think this would improve the submission quality.
  7. Stretch idea: Ongoing ratings? Almost want to do “preliminary” ratings today which setup a vesting (maybe first year gets locked in), but ratings can be revised later when new context is received that adjust ongoing vesting
    This is a pretty big restructuring of how we do things today, but could be a valuable experiment.
1 Like

My Budget & Allocation

I opted to allocate the full 8M OP — consistent with the Optimism Foundation’s initial proposal, agreeing with their rationale.

My allocations were as follows:

  • Ethereum Core Contributions: 50%
  • OP Stack R&D: 30%
  • OP Stack Tooling: 20%

I was assigned to the OP Stack Tooling category and opted to use the “Impact Groups” allocation method — my view was that the groups allocation method was consistent with the voting approach (bucketing them into impact groups — “High”, “Medium”, etc.) and I didn’t feel I had sufficient nuance in my assessment of projects within the individual impact groups to warrant splitting rankings out further.

Reflections on the Round

I really like the concept of giving Badgeholders budget allocation authority
Love the premise of this — both on the group deciding on overall budget, and on the allocations to the three pools — these decisions directly (and massively) impact how much each project has the potential to receive, and they should be re-evaluated from first principles on an ongoing basis.

…but I wish we’d discussed/debated them more as a Badgeholder group.
I think the Badgeholder group would have benefitted from having more structured upfront discussions about these (very consequential) allocations as a group — there is so much important context on previous rounds, on the Optimism Treasury as a whole, etc. that could probably have been more explicitly surfaced and discussed with the Badgeholders to drive higher-context decision-making on this front. The “Helpful Information” sidebar in the UI was a good start, but given the importance of these allocations I think we could assign even more rigour / explicit points of discussion and knowledge-sharing on this next time.

1 Like

It is an honor to participate in this round of voting.

Personal Background:
I am passionate about learning more about the OP Stack and regularly engage with developers on the Optimism developer forum and Discord, discussing issues and experiences. One of my shortcomings may be a lack of deep expertise in one specific area, and my strength is my curiosity about all the submitted projects.

Budget
Total Budget: 8M OP
Rationale: I believe the scale of rewards is not a critical factor, and I aim to fully utilize the available budget.

Ethereum Core Contributions: 50%
Rationale: After reviewing the applications, I view this category as foundational. Although 50% is a relatively high allocation, this is what I arrived at after considering its relevance compared to other categories. This decision also took into account the encouragement of other categories.

OP Stack R & D: 30%
Rationale: In this category, I noticed some projects that frequently appear in conjunction with Optimism, whether on governance forums, developer documentation, or Optimism’s social media.

OP Stack Tooling: 20%
Rationale: I believe many projects in the second category are extensions of those in the first, while this category contains many projects that specifically serve the OP Stack. This most directly reflects developers’ attention to the OP Stack. Unfortunately, due to the growth stage or unmet review conditions, we haven’t seen enough projects in this category.

Ballot
I was assigned to vote for Ethereum Core Contributions.
How I allocated votes:

  • I attempted to further classify projects within this category, such as developer organizations, clients, and smart contract languages.
  • I researched their influence in their respective fields, considering market share, organization size, etc., and scored their impact accordingly.
  • I evaluated the relevance of each field. For example, I believe smart contract languages are more directly applicable to the Optimism network than most clients.
  • I encouraged diversity, as diversity has positive effects both for the future and historically. For instance, even if a leader in a field is twice as influential as the second place, their reward allocation would not be twice as large.
  • For research projects and individual projects within this category, I assigned lower allocations, mainly due to their relevance.

Other Thoughts in This Round:

  • The potential impact of budget allocation on the rewards projects can receive. For example, under my allocation, the top project in the third category would not exceed 8M * 20% * 12.5% = 200K.
  • I did not review which projects initially applied in this category… I hope we can see more projects here, or perhaps relax the review criteria.
  • For projects from the same organization or with the same applicants, I suggest providing a notice, especially when the organization applies across multiple categories.

Finally:
The overall experience was smooth, and the number of projects requiring votes was reasonable. I appreciate the insights and information shared by others throughout the voting process.

1 Like

(post deleted by author)

Graciassss To Badgeholders and Applicants. I want to express my appreciation to all the contributors who have worked hard to support the OP Stack and Ethereum Core ecosystem. In this Retro Funding Round 5, I have the opportunity to evaluate their contributions and allocate OP funds with careful consideration.

Total Allocation: 8,000,000 OP (Maximum)

I decided to allocate the full 8 million OP, as the contributions received in this round are crucial for the sustainability of the Optimism ecosystem. My allocation is focused on three main categories, with weights based on their long-term impact on the core technology that Optimism is developing.

Category Allocation

  1. Ethereum Core Contributions: 48% (3,840,000 OP)
    This category encompasses the critical infrastructure that forms the foundation of the OP Stack. I have given high priority to projects in this category because of their support for the core technologies that underpin the OP ecosystem. Additionally, I have allocated extra weight to projects that have not previously received Retro Funding but have shown significant progress since the last round.
  2. OP Stack Research & Development: 31% (2,480,000 OP)
    Innovation is key to the ecosystem’s growth. Therefore, I have allocated a substantial portion of the funds to R&D projects directly involved in the development of the OP Stack and in supporting protocol upgrades. These projects not only impact existing technology but also pave the way for major future improvements.
  3. OP Stack Tooling: 21% (1,680,000 OP)
    Tooling is one of the essential elements that is often overlooked but has a significant impact on improving the accessibility and usability of the OP Stack. I have allocated this amount to ensure that developers and users have better tools for developing and using our ecosystem.

Allocation Method

I used a weighted allocation method modified from a top-curve approach. This allows me to reward projects that I consider to have a significant impact while still providing fair allocations to other important projects. Projects that have shown noticeable improvements in impact since the previous round were given special consideration in my allocation.

Voting Experience

The voting experience this time was very good, with a user interface that made the process easier. I was also pleased to see the option to import a CSV file, which allowed for more flexible allocation management. However, there were a few minor bugs that need to be fixed, and I hope this can be improved in the next round.

I am honored to be part of this amazing process!!!