Retro Funding 5: OP Stack - round details

[0x666]

@system

Citizens will vote on the round’s OP allocation within the rounds voting experience. The Foundation is assigning a minimum OP Amount for the round of 2M OP and a maximum OP amount of 8m OP. Citizens will be able to vote for an OP amount within the range of minimum and maximum which they believe appropriately rewards the impact within the round. The round allocation will be decided by taking the median of Citizens’ votes.

citizen only has right to vote reduce,no rights to vote increase,isn’t this not fair?
please consider allow citizen to vote the range between (+/-)6M

[saulius]

@joanbp

I don’t know for sure, but I understand ‘turnout’ to be the number of applicants, not the number of accepted applications.

As I have understood it, based on discussions in the context of the Collective Feedback Commission, the original expectation was that ~300 projects would apply for RF5. The reality is closer to half of that.

If on top of that a large percentage of the applicants are found to be in-eligible, according to the round rules as they have been defined, then that is an extra concern, not the only one.

I think the key turnout is the number of total accepted applications. The number of applications reflects the round amount. Increasing the round would get more applications for sure, but I really don’t see that the original round amount doesn’t match the application number. It’s a free market, OP retro funding is very well-known, so most of the applicants who were happy with the original round amount submitted their applications.

My main concern is that limiting the amount after the applications are submitted will further signal limiting core funding. This sector is already challenging in terms of funding and has very limited possibilities to generate revenue. I really don’t see how decreasing the amount would help the ecosystem in the long term.

[v3naru_Curia]

Disclaimer: The views expressed here are my own and do not represent the Grants Council or any other governance entity I am involved with.

I think it makes sense to reconsider the allocation for Round 5 since fewer applications were approved this time. I wanted to share my thoughts on what might be a fair OP token allocation for Retro Funding 5. I did some number-crunching based on data from Retro Funding 3, and I thought it’d be helpful to share as a reference for others.

Here’s the Breakdown:

1. Looking Back at Retro Funding 3 (OP Stack Category):

• 146 projects received allocations in the OP Stack category during Retro 3.

2. Checking Out Resubmitted Projects for Retro Funding 5:

• I went through the list of projects that got approved for Retro 5.
• Found that 21 projects from Retro 3 have reapplied and got approved for Retro 5.

3. Allocation for the Resubmitted Projects:

• Let’s assume these 21 projects have continued to make a consistent impact since Round 3, so they’d get around the same allocation they got back then.
• The total allocation they received was 4,429,911 OP.

4. Allocation for New Projects Using Averages:

• That means 125 projects from Retro 3 didn’t reapply.

• For Retro 5, there are 58 new projects that didn’t participate in Retro 3.

• I’m thinking we can use the average (mean) and median allocations from those 125 projects to estimate what the 58 new projects might receive.

• From those 125 projects in Retro 3:

  • Average allocation per project: ~65,459 OP
  • Median allocation per project: ~49,689 OP

5. Estimating Allocation for the New Projects in Retro 5:

• Using the Average:
  • Total estimated allocation: 65,459 OP * 58 = 3,796,622 OP
• Using the Median:
  • Total estimated allocation: 49,689 OP * 58 = 2,881,962 OP

6. Calculating the Total for Retro Funding 5:

• With the Average:

  • Resubmitted projects: 4,429,911 OP
  • New projects: 3,796,622 OP
  • Total: 4,429,911 OP + 3,796,622 OP = 8,226,533 OP

• With the Median:

  • Resubmitted projects: 4,429,911 OP
  • New projects: 2,881,962 OP
  • Total: 4,429,911 OP + 2,881,962 OP = 7,311,873 OP

Additional Note:

Based on data we gathered from the on-chain application attestations, the total allocation for Retro Funding 3 allocated to the OP Stack was actually 12 million OP. This includes projects with overlapping categories. If we only count projects that were exclusively in the OP Stack category, the number is 2.72 million OP. These numbers are different from the benchmark number provided by the OP Foundation, which was ~6.5 million OP. This discrepancy might be due to differences in categorization or counting methods.

I’ve included the sheet here with all the calculations for those who want to dive deeper into the numbers.

So, What’s my takeaway?

Based on these calculations:

• Using the Average: Total comes to about 8.2 million OP
• Using the Median: Total comes to about 7.3 million OP

I think it makes sense to adjust the OP allocation to somewhere between 7.3 million and 8 million OP (due to the cap). This way, we’re aligning the rewards with the potiential impact and participation in this round, keeping things consistent with past rounds.

[joanbp]

Nice to see some number crunching.

In the spirit of putting our thoughts out where everyone can see them and discuss them:

Let’s assume these 21 projects have continued to make a consistent impact since Round 3, so they’d get around the same allocation they got back then.

This assumption might hold true if RPGF3 and RPGF5 both operated with a specific and comparable time span for the impact that is to be rewarded, say, if they both rewarded impact within the last 12 months. However, RPGF3 was open ended back in time - applicants were free to apply based on impact created over multiple years.

Also, it is my impression that not all returning applicants are careful to only report fresh, not-previously-rewarded impact or to argue clearly for why previously rewarded work has in the meantime become more impactful so as to merit new rewards.

For these reasons, I would actually expect careful analysis to show that at least some returning applicants should receive significantly less this time around than they did in RPGF3.

• That means 125 projects from Retro 3 didn’t reapply.
• For Retro 5, there are 58 new projects that didn’t participate in Retro 3.
• I’m thinking we can use the average (mean) and median allocations from those 125 projects to estimate what the 58 new projects might receive.

On the other hand, I would expect at least some of the new applicants to be high achievers.

It is worth noticing that the 21 returning applicants received a lot more than the average OP Stack project in RPGF3, meaning that the 125 not returning projects must have received less. I would not assume that the new applicants are more like the latter group than like the first without any evidence.

---

Who will pick up the baton next? Let’s just see where this can take us.

[Gonna.eth]

Disclaimer: The views expressed here are my own and do not represent the Grants Council or any other governance entity I am involved with.

@joanbp

This assumption might hold true if RPGF3 and RPGF5 both operated with a specific and comparable time span for the impact that is to be rewarded, say, if they both rewarded impact within the last 12 months. However, RPGF3 was open ended back in time

I believe this is the key missing in this discussion. Retro5 is looking to compensate for the impact of the last 6-8 months (Period from Retro3 up to Retro5). Retro3 had no time threshold so we can assume it was the impact since the project inception up to Retro3 or from Retro2 to Retro3.

In both cases, inception to Retro3 or Retro2 to Retro3 both are more than 6 months. And I believe this is why it is justified that the Retro5 cap is lower than 10M

@v3naru_Curia Your numbers report is amazing, mind adding the variable “impact time considered for rewards” and see what comes up?

[grapebaba]

I am very pleased to see such detailed statistics, as there are some projects that did not choose OP Stack in retro 3, such as go-ethereum, and there are also some projects whose application names are different from those in retro 3, such as test in prod, and some that were applied for individually in retro 3, which have now been changed to projects. In fact, the number of resubmitted projects is at least greater than 5M.

[v3naru_Curia]

Thanks for bringing this up, especially regarding the “impact time considered for rewards.” I think this is a crucial aspect to consider, as it adds depth to understanding the scope of contributions eligible for retro funding.

I agree that for Retro5, the period considered for the impact (from Retro3 up to Retro5) justifies a cap lower than 10M, given the relatively shorter duration compared to previous rounds. To further refine our approach in future rounds, I’d like to propose an additional requirement for applicants: providing the project inception date when applying, in addtion to their previous retro funding reward rounds. This will allow us to assess not only the recent impact but also provide a more comprehensive view of a project’s contribution over time, especially for projects that haven’t applied in previous rounds.

This adjustment would ensure a fairer and more transparent evaluation process, helping us align the allocation of rewards with the full scope of each project’s impact.

[ccerv1]

tl;dr - here is a Google Sheet you can copy to help set the amount of rewards per category.

Read on if you want to know why this might be helpful to you.

---

Dear voters,

As you know, you will be casting three types of votes in Round 5:

• First, you’ll propose the budget. You’ll propose the budget for the entire round, up to a max of 8M.
• Then you’ll decide how much of that budget should go to each category, eg, 40% Ethereum Core Contributions, 25% OP Stack R&D, 35% OP Stack Tooling.
• Finally, you’ll score and allocate rewards to projects in one category randomly assigned to you.

While the third decision will likely take the most time to perform, the first and second decisions could be even more consequential in terms of determining how much funding projects receive from the round.

You will be presented with this initial screen:

Resist the temptation to simply allocate the default ⅓ to each category!

There are 79 total projects in the round and the most popular category (Ethereum Core Contributions) has 33/79 projects (42%), whereas the least popular (OP Stack Tooling) has 20/79 projects (25%). If you believe each category is equally important, then allocating 42/33/25 would be better than 34/33/33.

But perhaps you believe some categories are more impactful than others. Or you think the top projects in some categories should receive much more than an average project. If that’s you, then good news, I’ve created a little tool to help you vote your values!

---

Here it is: RF5 Voting Simulator

---

The tool is nothing more than a humble spreadsheet. Copy it into your personal Google Drive and play with the assumptions.

For each category, you decide:

• How many projects you think should receive retro funding
• How much the top project in that category should receive
• How much the median project should receive
• How much the bottom project (that receives any OP) should receive
• If you like power law or linear distribution curves

After making your settings, the graph will update automatically and you’ll get a recommendation of what percentage to allocate to that category.

Repeat this process for all of the categories.

Behind the scenes, all that’s happening is some regression to try and fit a curve to your three data points. The fit probably won’t be perfect – the “modeled” reward sizes give you a sense of how close you are to your “target” rewards. You’ll probably need to play with the numbers a bit to get the right approximation. And if you put weird numbers in you might get a curve that violates the rules of the round.

Of course, the eventual results will be a function of how everyone else votes. But hopefully this little tool can help you vote more optimally and save you some time. Enjoy!

[grapebaba]

It seems there are some update, Ethereum Core Contributions category right now has 30 projects, OP Stack Research & Development has 29 projects.

[OPUser]

Thank you for sharing this. Discussions based on metrics are always a step in the right direction.

However, one important factor missing here is the time frame considered. RPGF3 was not time-bound, and when rewards were allocated, at least to me, this was a crucial aspect. Some projects had been live for over a year, publicly available, operational, and generating impact. Round 3 was the first time these projects participated in RPGF. eg,65,000 OP for two years of contribution less than half of that, under 30,000 OP?

This round is time-bound, as others have already mentioned, and it makes it difficult to make an informed decision based solely on past data, although it does provide a point of reference.

I understand that building in public is challenging, and countless hours of work cannot be sustained on charity alone. Looking back, the primary goal of RPGF was to bridge the gap between impact and reward. My intention isn’t to contradict your points but to offer a new perspective. Personally, I’d prefer to allocate 10% more rather than risk allocating 2% less.

Another point to consider is the treasury. While we have a significant amount of OP set aside for the next few rounds, without impacting the treasury, if we continue spending without considering revenue, we could potentially hurt our long-term sustainability.

Between October 2023 and August 2024, the current round’s time frame, Optimism’s revenue is approximately $11.37M ( + $1.25M from Base)

Source

Round 4: 10M OP
Round 5: Between 2-8M OP
Round 6: 3.5M OP

We can calculate the inflow/outflow ratio based on these figures.

Optimistically, we could run approximately 80 more rounds, each with 10M OP, just from the treasury (with 799.99M OP remaining from the initial RPGF allocation), without relying on revenue. However, our current focus is narrow, limited to the Superchain ecosystem. It would be unfortunate if we couldn’t extend this experiment beyond Superchain to support open-source contributions, public goods, and potentially common goods as well.

There are many ways to approach this, and different strategies could be implemented in future rounds but I believe I am branching out from the discussion point so holding my train of thoughts here. However, I wanted to emphasize that, while it’s important to review past rounds to assess rewards, we should also try to look into future.