New RPGF3 Distribution Disparity Data

[alexcutlerdoteth]

Heya Folks -

@Jonas suggested I share Carl’s latest post on the RPGF3 distribution data here to get more Badgeholder’s eyes on it. Key takeaway is that the top sequencer revenue generators received a whopping 1.5M OP of the 30M of RPGF3. That is 5% to the top builders who are creating demand for blockspace which is what will fund public goods.

I’ve been quite vocal in flagging that sends the absolute wrong signal to the ecosystem’s most critical constituency and if not mitigated quickly will likely have a chilling effect on ecosystem growth.

Carl Cervone ☕️

[Valueinvestor.eth]

first of all, thanks for your contribution to the ecosystem have been tremendous so want to flag this upfront.

As a Top 1% Optimism user (In terms of transactions, volume and actual daily usage of OP as a replacement for all my TradFi activites I agree 100% with you. Clearly, OP should want to attract and retain users like myself.

I get the feeling that OP governance is just a bunch of twitter VC and I was not familiar with the whole badgeholder thing, but who are these badgeholders? And why am I not one of them? Clearly I should be on the basis that I have used this chain for 1. longer 2. more often 3. in greater volume than almost all wallets.

Am not a dev but when I saw the list, I couldnt recognize half of the top ones.

[MinimalGravitas]

It’s an interesting point, but perhaps that data would benefit from the inclusion of a column listing how much value has been used to incentivize transactions. A project which can generate 10 ETH worth of sequencer fees just by paying 1 ETH’s worth of incentives, is quite different from one that we give 50 ETH to for incentives that goes on to generate 20 ETH back to the pot.

One explanation might be that badgeholders felt these projects had already been compensated by other grants / incentive programs. [From the second tweet]

I’m not suggesting that total previous grant amounts (which were included in the RetroPGF applications) are relevant here, just the amounts that were given out to users for making transactions.

If users are paid out more in OP than the sequencer gains back from their transactions, then it does it seem especially meaningful to count as sequencer revenue being a net benefit in itself?

In my opinion it’s not too surprising that it wouldn’t correlate with RetroPGF allocations. Once the ecosystem has progressed a bit more, and dApps are running without OP incentives for use, then this metric might match the RetroPGF allocations more closely? That would be my guess anyway.

[ccerv1]

Thanks @alexcutlerdoteth for moving this to to the forum instead of the bottom of a Twitter thread. Let me share the analysis here and some additional thoughts.

The distribution

Just ran the numbers and the top 20% of RetroPGF projects in terms of sequencer fee contribution earned a combined 1.5M OP (5% of the pool).

ERC 4337 (AA) was the one project that stood out. The remaining 15 protocols all received <100K OP.

You can see the actual top 16 projects here, sorted by token rewards.

PSA - anyone can access the underlying data or fork my specific analysis notebook here.

Some thoughts

What’s to make of all this?

One explanation might be that badgeholders felt these projects had already been compensated by other grants / incentive programs or have their own built-in value capture mechanisms. However, the fact that many of the leading OP protocols appeared on a relatively small number of ballots (ie, <50 out of 130+ voters) suggests that these projects may simply have been overlooked by the average badgeholder. That’s the simplest explanation.

There’s an version of Hanlon’s razor attributed to Goethe that goes like this:

Misunderstandings and lethargy perhaps produce more wrong in the world than deceit and malice do. At any rate, the latter two are certainly rarer.

(h/t miyazono)

So the simplest explanation would be that the average badgeholder “misunderstood” the direct contribution these protocols were making to Optimism’s economic engine and/or was too “lethargic” to do their own research on 600+ projects.

How do we correct for this?

I’m glad this kind of analysis is prompting some important conversations.

But the important question is: is the kind of distribution the collective actually wants to see?

My strong hypothesis is that most citizens would say no. The top protocols in terms of sequencer fee contribution on OP Mainnet should not be getting a “peanut butter spread” distribution somewhere in the middle of the pack.

OK, so what does a more optimal distribution pattern look like??? Then, one level deeper, what’s the right split between defi vs quests vs identity vs consumer use cases? And, finally, if citizens think use case X is most important, but when the application + review process is finished we only see one project standing, do they stick to their guns and give a large amount to one project or do they update their preferences?

FWIW, I’ve been banging this drum since after RetroPGF2:

Overcoming bias

Also buried in a tweet reply is this exchange with @jackanorak

The encouraging thing here is the apparent pattern indicated by the flat 100ks across that people simply didn’t know how to distinguish between different apps or attach value to their contributions.

In response, I took the above projects and ranked them in terms of sequencer fee contribution (hiding the project name). Then I came up with four simple formulas for determining token distribution formulas.

We could test this – and it would be interesting to see what options are most favored by voters depending on how the options / amounts are presented.

For all the mission proposals that seek to improve governance, I’d personally like to see more work go into this kind of stuff

[alexcutlerdoteth]

@MinimalGravitas

It’s an interesting point, but perhaps that data would benefit from the inclusion of a column listing how much value has been used to incentivize transactions. A project which can generate 10 ETH worth of sequencer fees just by paying 1 ETH’s worth of incentives, is quite different from one that we give 50 ETH to for incentives that goes on to generate 20 ETH back to the pot.

One explanation might be that badgeholders felt these projects had already been compensated by other grants / incentive programs. [From the second tweet]

I’m not suggesting that total previous grant amounts (which were included in the RetroPGF applications) are relevant here, just the amounts that were given out to users for making transactions.

If users are paid out more in OP than the sequencer gains back from their transactions, then it does it seem especially meaningful to count as sequencer revenue being a net benefit in itself?

In my opinion it’s not too surprising that it wouldn’t correlate with RetroPGF allocations. Once the ecosystem has progressed a bit more, and dApps are running without OP incentives for use, then this metric might match the RetroPGF allocations more closely? That would be my guess anyway.

I don’t think we should over think this. 95% of of the RPGF budget was allocated to individuals and projects that produce little or no impact on sequencer revenue and onchain activity was the single least predictive factor in allocation amount. The factor most critical to the sustainability of RPGF was one of the least substantial in both total and relative weight of allocations. I think debating the role of VC or grant funding should be secondary to first achieving a distribution that at the very least better represents tangible onchain impact.

[MinimalGravitas]

@alexcutlerdoteth

I don’t think we should over think this.

To be fair though, you did create a thread to think about it! I’m just trying to explain why only looking at one side (the value generated for the sequencer by transactions) then we’re missing half of the story.

Just to use Velo as the example, if they generated 30 ETH worth of fees that works out to about 25,000 OP. Which isn’t a huge amount compared to the OP incentives that were being paid out to incentivize those transactions. I can’t remember exactly how much from the last grant was for incentives but something like 1.5M out of a 4M total grant - it doesn’t exactly matter but you get the idea, there’s a couple of orders of magnitude difference at least).

@alexcutlerdoteth

I think debating the role of VC or grant funding should be secondary

Sure but like I said above, I’m not talking about general funding. I’m just referring to the amount of OP that came from the OP collective and was used in effect to pay users to make those transactions.

Taken to the ludicrous extreme we would reward a dApp that just pays users 10 OP for every 9 OP worth of gas they wasted. That would generate huge transaction numbers, but would be a net loss to the collective coffers and probably wouldn’t get a huge amount of RetroPGF awarded to it by badgeholders!

And again, I do think that this metric of generated revenue should match up more closely with RPGF rewards in the future, but for now it doesn’t seem very important unless you also include how much we’re paying for it.

Hope that makes my point a little clearer?

[Jonas]

Thanks for taking this discussion to the forum @alexcutlerdoteth!

Want to share some perspective on this:

1. Rewarding onchain builders for their impact is one of the goals for RetroPGF and surfacing the data on how RetroPGF 3 has perfomed in this aspect is super valuable! These types of observations are the right way to inform learnings from the round and iterations of future rounds. I’ve seen a lot of takes on this topic on Twitter, taking this to the forum allows citizens to have visibility and engage in this discussion.
2. Rewarding builders who create demand for blockspace has been in scope for RetroPGF 3. Strong feedback has been that there wasn’t enough focus by badgeholders on rewarding this type of impact. This aligns with a bigger learning in RetroPGF 3 that the wide scope of the round makes the voting harder for badgeholders and prevents the program from effectively driving developer behavior, as it’s hard to understand what is in scope. Introducing multiple types of rounds with focused scope has been a very popular request from badgeholders and will be part of the next iteration of RetroPGF!
3. The Citizens’ House should explore analysis of the impact of onchain builders beyond generating sequencer revenue. If RetroPGF rewards for sequencer revenue generation exceed the sequencer revenue itself, which is the case for most builders participating in RetroPGF 3 on a 6 month timescale, then the system becomes easily attackable. Combining more sophisticated “impact vectors”, such as repeat users, users which are likely unique humans, etc. will help with setting incentives in the right way. Recommend diving into @ccerv1‘s work on impact vectors here.

Overall, I think the discussion around how to measure and reward the impact of onchain builders is exciting, especially because this is an area with a lot of data we can leverage, and there’s much to explore in how to set the right incentives!

@ccerv1

We could test this – and it would be interesting to see what options are most favored by voters depending on how the options / amounts are presented.

Love this idea! Some early prototyping on how to design metrics driven voting interfaces has kicked off here

[alexcutlerdoteth]

I understand where you are coming from, I just don’t want to miss the forest for the trees here. Governance Grants do not exist to drive transaction volume and almost none of them (to my knowledge) directly incentivize transactions as core KPIs. They exist functionally as subcontracts to individual teams or builders to achieve ecosystem goals or intents (i.e. “Spread the Optimistic Vision”). Just speaking for Velodrome, we have never directly incentivized transactions. There can of course be second or third order effects that could impact transaction volume from the programs these protocols administer, but that is just where I think we risk distracting from the core issue here which is that just 5% of the RPGF rewards went to those driving significant demand for Optimism blockspace . It is far more important imo to ensure that balance is addressed before we dig too deeply into some of the subsequent nuances as right now the picture painted is that one of the lowest ROI activities a builder can do right now is build a worldclass app that attracts significant usage.

[ccerv1]

I will be the first one to agree that one metric rarely if ever tells the whole story! But I also suspect the simplest explanation is the most plausible one here, ie, badgeholders didn’t have much knowledge of sequencer fee impact.

IMO, the type of logic badgeholders should be trying to employ is:

1. How much of the pool should go to the top protocols? 5%, 20%, 95%, etc.
2. What kind of token distribution do you want to see across the top protocols? peanut butter spread, power law, winner takes all, etc.
3. How do you create a first version of a distribution based on one or more verifiable impact metrics? ETH fees over last 6 months, ETH fees from a subset of the most active users, etc
4. How do you determine what other information is relevant to “correct” the distribution? past grants / incentives, VC funding, etc

The power of this kind of approach is we have a rationale for why we are funding project X more than project Y. We will probably be incredibly bad at it, but hopefully we get better each round.

Right now, we have no way of explaining why Sushi (a defi protocol that I assume most badgeholders have heard of) generated 1.5 ETH in fees and got 100K OP and Holonym (a zk/identity protocol that most badgeholders probably have not heard of) generated 3.3 ETH in fees and got 65K OP.

[alexcutlerdoteth]

Exactly right. And many such cases:

This protocol reported 13 total transactions on Base and Optimism and received 38k in OP. Functionally, zero sequencer impact earned about 38% as much RPGF as a Velodrome, Synthetix, or Kwenta. There is no way to explain why here, but this is the message that was sent to builders.

[katie]

To me, it seems obvious that the projects contributing to sequencer fees, which is how RPGF will be funded going forward, should receive the majority of the RPGF rewards. I was disappointed with the amount of projects with little to no impact that received rewards this round. That being said, this was only the third round and I’m hopeful that we can make significant improvements going forward.

[mariorz]

I have to say that I agree with @MinimalGravitas on this one. It doesn’t make a lot of sense to categorize the “top protocols” by sequencer revenue without considering how much OP those protocols have distributed from previous grants.

Would you agree that the metric used to define “top protocols” be: sequencer revenue - OP value distributed?

It seems clear that not considering OP incentives distributed would only self-perpetuate a set of existing protocols.

[jackanorak]

@mariorz

Would you agree that the metric used to define “top protocols” be: sequencer revenue - OP value distributed?

In fact I do not, and neither should anyone else, for the reasons @alexcutlerdoteth has already outlined:

@alexcutlerdoteth

Governance Grants do not exist to drive transaction volume and almost none of them (to my knowledge) directly incentivize transactions as core KPIs.

They’re serving other goods; transaction volume is beside the point here.

Just as @Jonas is encouraging people to look at additional impact vectors beyond just sequencer revenue (which is why I am in some respects skeptical of CSR), I would encourage people to look past simplistic framing and instead train their eyes on the actual data @ccerv1 is presenting here.

The story is straightforward: badgeholders are not proportionately rewarding what is currently only thing holding up this program for the long term.

[mariorz]

Perhaps I’m misunderstanding and you can expand, but just to use as an example, has Velodrome distributed OP to its users as a form of incentive? If so, this would clearly affect the protocol’s sequencer revenue. It seems obviously flawed to use sequencer revenue as a metric without considering how much of it is artificially generated by OP distribution from previous grants

[jackanorak]

They aren’t incentivized to transact.

In the case of user incentives, you would have to deduct all the other types of value that are being generated from the same grant to arrive at a measure of transaction volume relevant for such a measure, one that relies on an impossible-to-calculate counterfactual.

This is why I’m reminding people that there are many types of value offered by these grants, value that, because now both you and @MinimalGravitas have mentioned it, Velodrome has offered in unmatched magnitude. I’m not sure what is causing such a focus on this; it’s setting back discourse.

And, again, this is beside the point – @ccerv1 is showing and @Jonas is acknowledging a blanket deemphasis of sequencer revenues in the last RPGF. This doesn’t have to be a litigation of Velodrome’s case - there are many, many others here.

[mariorz]

What are they incentivized to do then?
For example, if they are incentivized to provide liquidity, the liquidity increase automatically impacts sequencer revenue in the form of extra arbitrage transactions.

Dont want to single out Velodrome, but more generally, it seems wild to disagree with the idea that we should consider if 1 ETH of sequencer revenue by any protocol was the result of paying 5 ETH’s worth of OP to incentivize its usage.

To be clear, I’m not disagreeing with the use of sequencer revenue as a signal for future grant distribution, so I agree with @ccerv1. What I’m saying is we have to control for how much of that revenue is the result of artificial incentives from previous OP distribution, otherwise, the result will be the self-perpetuation of a set of protocols.

“Show me the incentives and I’ll show you the outcome…”

[alexcutlerdoteth]

You are welcome to read about grant programs here, but as Jack has said there has been no direct incentivization of transaction volume.

But, I again think you are missing the forrest for the trees here. Optimism is in fierce competition for worldclass builders with a seamingly endless number well funded Layer2s. RPGF and the idea that impact=profit is at the core of Optimism’s economic model and value proposition. The only way the system works is if we massively increase activity on these chains and the core philosophy behind RPGF is we do that by ensuring impact = profit.

Despite this, 95% of rewards went individuals and projects with little to no directly attributable onchain activity that drives sequencer revenue. Meetups, private events, part time members of goverance, protocols that were live three weeks, protocols that reported 13 total transactions, individuals who reported 0 impact metrics, all earned at a rates comparable to the top projects and builders in the ecosystem. The ones actually creating demand for Optimism’s blocksapce often at great personal risk.

Is sequencer revenue alone a perfect measure? Absolutely not. Would have weighting it higher in the distributions even if it meant ignoring second or third order effects of VC or grant funding on impact metrics led to better outcomes? Absolutely.

In the end here the biggest issue we face is that RPGF3 sent the message that if profit is the goal (which is what the system is intending to incentivize), building large expensive impactful applications on the OPStack is one of the lowest ROI ways to go about it… and that is big big problem for the future of the chain and program.

[alexcutlerdoteth]

@mariorz

What are they incentivized to do then?

To be clear, grants are designed to incentivize the key outcomes (i.e. Intents) as defined by the collective. For example: Collective Intents: Season 5. The Collective says “we want to achieve X, show us how you can use OP to do it”. Protocols then effectively just administer the programs on the Collective’s behalf.

[Gonna.eth]

I want to implement CSR for developers and try to bring it into RFGs so OP distribution by TX is fair. Still waiting for a few approvals.

[DRAFT] Contract Secure Revenue (CSR) Mission Request

Delegate Mission Request Summary: Option A: Research on Implementing CSR with Optimism Sequencer Conduct in-depth research to explore the technical aspects and feasibility of integrating Canto’s Contract Secured Revenue (CSR) directly with the Optimism Sequencer, providing valuable insights to inform future development decisions. Option B: Research CSR System with OP Stack Using OP Tokens Investigate the integration of a CSR system with the OP Stack, leveraging OP tokens for compensation, wit…

[mariorz]

@alexcutlerdoteth

Is sequencer revenue alone a perfect measure? Absolutely not. Would have weighting it higher in the distributions even if it meant ignoring second or third order effects of VC or grant funding on impact metrics led to better outcomes? Absolutely.

I’m not talking about second or third-order effects though, or VC funding. If the collective is handing out OP for a protocol to distribute to its users, we should discount the value of that OP from the sequencer revenue generated by such a protocol.

Again, it seems wildly illogical to argue that a protocol should receive grants based on it generating 1 ETH of sequencer revenue if the collective paid out 5 ETH to incentivize that revenue.