Rev's RPGF3 Retro

[revmiller]

Introduction

I’m a big fan of RPGF and believe it’s a groundbreaking crypto experiment. If we create a sustainable way to fund public goods, it’ll empower teams to tackle our biggest challenges long-term.

After RPGF3, I reflected on the process and have some comments to share in categories:

1. Badgeholders
2. Projects, Application & Voting
3. Other Considerations

Disclosure: I’ve been a Badgeholder since RPGF2, nominated by the Optimism Foundation. I did not submit my project during RPGF3, and I’ve disclosed my conflicts of interest. Having read all RPGF3 feedback posts from Badgeholders, I’m grateful for the insights and acknowledge others’ feedback, giving credit where it’s due. The feedback shared in this forum post aims solely for improvement purposes—I genuinely want the RPGF experiment to succeed.

Badgeholders

• Clarity in commitment
  Badgeholders should acknowledge their realistic time and effort commitments - and pass on responsibilities / nominate someone else if they can’t commit fully. Yes, we need to clarify the ‘Minimum Viable Badgeholder Commitment’ (worth noting: @tjayrush on “Minimum Expected Behaviour” and @Eagle_Cloak accountability standards.

• Understanding Commitment
  A high-quality RPGF contribution demands significant time & effort. Here’s a rough breakdown of what it takes to make a substantial impact:

  • Pre RPGF Onboarding (3-7+ HRS)
    Reading RPGF docs, forum posts, learning from past RPGF experiences, engaging with other Badgeholders, and participating in productive discussions.
  • RPGF Badgeholder Coordination (5-10+ HRS)
    Attending RPGF meetings (~1hr/meeting), workshops, and gatherings with other Badgeholders in groups & 1:1, both online and IRL (Devconnect was fun!).
  • PRGF Project Discovery & Analysis (5-100+ HRS)
    Reviewing projects with varied impact evaluation strategies, spending time on deeper project impact analysis, reviewing funding from previous RPGF rounds, and considering projects’ impact on the overall Collective, Ethereum, and public goods landscape.
    Key points:
  1. I assume the majority of Badgeholders reviewed some RPGF projects and relied on Lists for others, while a few analyzed all 643 projects to the best of their ability, striving for fair impact evaluations.
  2. From my experience, each project took me 4-30 min, with some projects requiring hours. I spent substantially less time on what seemed like opportunistic projects (low-quality article/video mentioning OP) and other one-time events. Most of my time I focused on a) smaller yet highly impactful projects and b) breakthrough projects, within my expertise area.
  3. I’ve seen a several projects that haven’t delivered much tangible value since the RPGF2 yet reapplied. In light of this, I respect folks who’ve honestly assessed their impact between RPGF2 & RPGF3 and chosen not to reapply, while also appreciating other Badgeholders reminding of the importance of considering previous allocations(especially @Gonna.eth with his RPGF2 List).
  4. Depending on Badgeholder’s availability & commitment level, they might either compile lists for their ballot with relatively modest analysis (3-7hrs), OR do their best to make calculated decisions based on deep review process (30-100+ hrs).
  • Impact Allocation Iterations (2-10+ HRS)
    Reviewing existing allocations, experimenting with impact evaluation formulas (among many others: @ccerv1 here, here; @mel.eth here; @dmars300 here, @joanbp here, engaging in feedback sessions with other Badgeholders, and possibly interacting with projects for better evaluation (asking clarifying questions, requesting more data).
  • Ballot Submission (1-4+ HRS)
    Compiling allocations (esp if you’ve used Spreadsheet), double-checking for errors (at scale of 400+ projects in my case), and making through potential tech challenges.
  • Reflection & Feedback Sharing (1-10+ HRS)
    Continuing the RPGF process beyond ballot submission by reflecting on the experience, reading other Badgeholders’ thoughts, participating in Collective’s RPGF retrospective, and sharing constructive feedback while acknowledging others’ contributions.

  Overall, the RPGF process might take anywhere between 20 to 140+ hours, akin to a part/full-time monthly commitment, depending on each Badgeholder’s unique circumstances.

• Independent Decision-Making
  Delegating decision-making shouldn’t be a default. While trusting experienced Badgeholders might be reasonable, individual analysis is crucial. Like many other Badgeholders (Katie comment; Thomas on group think), I found Lists hepful for discovery, but counter-productive for allocations. Moving forward, if we’d keep the Lists, let’s focus on impact analysis & comments, and remove OP allocation to avoid associated biases.
  Note: this issue could also be resolved with “Advanced RPGF Hub” (read ‘Bold Experiment Idea’) that includes a comments/feedback dashboard ft. each project.

• More Transparency Needed
  There’s a need for increased transparency among Badgeholders, especially concerning their conflicts of interest—including in Lists. Specifically, regarding individual/team projects associated with Badgeholders creating these lists. I found it inappropriate to see Lists from Badgeholders featuring allocations to their individual/team projects without sufficient transparency. Adding disclosures in the future & removing OP allocation will solve this.

• Mindful Allocations
  Allocations need careful consideration. It’s not wise to give large amounts of OP to low-impact projects, even if those amounts seem small compared to the total RPGF size. Requesting more info from projects helps, but mistakes happen. Still, it’s crucial to vote decisively even with a small OP amount that covers people’s efforts. Some projects had little impact, but I allocated sub 500-1k OP to some projects based on my rough calculation of associated costs with their impact. Errors are inevitable, no matter what. Projects can specify their private & RPGF fundings in the next round, including estimate costs associated with impact—this will help Badgeholders make better decisions, bridging the gap between impact and profit. Overall, I believe it’s better to allocate less and adjust in the next round than to overallocate, which might incentivise bad actors to exploit RPGF – that’s not okay.

• Leveraging Expertise
  I support @zakku’s idea of highlighting Badgeholder expertise by allocating % of total projects to be reviewed deeply by Badgeholders within their expertise category, followed by impact statements and their comments. This could enable particularly busy Badgeholders to a) offer high-quality contributions where needed and b) utilize their knowledge and strengths most effectively. This approach would lead to multiple Badgeholders reviewing the same set of projects, sharing their insights and impact statements. Other Badgeholders may still opt to review most or all projects and share their comments as well.

• Recognizing Biases
  This is crucial, and I’d repeat it again: every Badgeholder should thoroughly understand and study the biases associated with RPGF. Carl has shared a great post that I highly recommend reading and learning from.

• Rewarding Badgeholders
  I echo the sentiments of many Badgeholders, including @lefterisjp in his thoughtful feedback post - being Badgeholder could equal to part/full-time monthly commitment for some, requiring anywhere between 20-140+ hours. Considering the expected increase in commitment for future rounds, there should be a sustainable method to fairly compensate each Badgeholder for their contribution.

BOLD EXPERIMENT IDEA

Advanced RPGF Hub

This wouldn’t just be a knowledge hub; it would also serve as a social platform where Badgeholders, project teams, and aspiring applicants could have dedicated place for coordination, impact sharing, evaluation, justification, Q&A, and more.

This will solve the siloed experience that Badgeholders and project applicants face when trying to communicate across various channels. I’m sure many Badgeholders have likely received tons of DMs on Discord, Telegram, X, etc - from applicants seeking their votes.

An advance RPGF Hub, featuring a knowledge base, integrated social features (potentially with Farcaster?), enhanced discovery processes with advanced filters (in agreement with MinimalGravitas point), feedback loops, comments, and Q&A could exist with tiered access levels: token-gated for Badgeholders/project applicants, and read/comment-only for the public or aspiring RPGF projects. It could also incorporate upvotable public and/or Badgeholder notes.
Some folks have suggested similar ideas: Michael here; cheeky-gorilla here; and others.

On this note, I highly appreciate Jonas for his time and efforts in consistently reaching out to Badgeholders before, during, and after the round - checking in, sharing feedback, and being incredibly helpful. Jonas even followed up with me when I hadn’t included any projects in my ballot at that time, as I was working on them separately outside of Pairwise/Agora, etc. I strongly believe that a Badgeholder Hub would address this issue and save everyone’s time. Ultimately, we require a better and simpler coordination tool to keep everyone informed about the process, progress, challenges, feedback, and more.

---

Projects, Application and Voting

• Greed should NOT be tolerated
  Many projects applied for funding but didn’t make a tangible impact on Optimism collective since RPGF2. It’s okay not to apply if you haven’t made an impact. Applying and pretending to have an impact is NOT okay.
  On this note: I respect folks like 0xzenodotus that’re conscious of their impact and make their intentions & reasoning clear, quoting: “We made the decision not to apply for Round 2 and instead worked tirelessly and put our best efforts toward creating impact— knowing we would shoot for Round 3”. This should be the way, and greed should not be tolerated, which leads me to the next point.

• Kickback-like “slashing mechanism”
  Mechanism similar to ETHGlobal’s model, but applied for rule violations as well. Projects stake tangible amount of ETH / OP / else for both verification and the duration of RPGF, reducing amount of spam and discouraging violations of rules; it will be reimbursed after project submission.
  I’ve seen similar idea shared by Michael here - and absolutely support this.

• Clarity on Impact
  It wasn’t always clear the overall impact of the project and the non-rewarded one (impact between previous and current round) - I assume this might potentially confused especially new Badgeholders & led to unfair allocations. Shout out to Gonna for kindly reminding Badgeholders about the importance of analysing past funded impact.

• RPGF funds transparency
  Adding a section where projects will share how they’d allocate RPGF funds could increase accountability and distinguish projects planning to give back to the Optimism ecosystem. It’d be cool for projects to specify a custom % of OP allocations (0-100%) they’re planning to allocate back to Optimism - as part of their product-led growth on Optimism / Superchain. I loved seeing projects that have stated or simply will (as part of their commitment) redistribute RPGF funds back to community. Among others, kudos to @brian_rabbithole & Rabbit Hole team for making it clear (”Any retroactive funding will be re-distributed through the Optimism Gateway”). On another note, it’s okay to say “the funds will be used to compensate the work done and/or cover future R&D/growth” - just be transparent about it.

• Highlight OP Native & Superchain Exclusive Projects
  We should give more credits to OP Ecosystem heroes, cherish most dedicated supporters of Optimism’s vision, mission, and values. Moving forward, let’s highlight both OP and Superchain Native projects - ideally, with a searchable filter/tag/color-palete.

• Ask projects to measure their impact
  I absolutely agree with Cris here & ITU Blockchain here - it’d be great for projects to share how they measure their impact and what was the costs associated with that impact.

• Disclose funding sources
  It’s crucial to disclose private/VC funding in RPGF applications for fairness. I’ve shared this point repeatedly since RPGF2 — funding amount should be disclosed, even broadly. I’m glad many folks (Thomas here; jpnmemelord here; Lefteris here) agree. I was surprised RPGF3 lacked a private funding section. Kudos to projects that disclosed despite this.

• Make project pages dynamic
  RPGF project page could be more dynamic, allowing for upgrades (with version history) and comments (Notion style). Dicaso shares about Badgeholder’s commenting ability on applications and cheeky-gorilla point on Twitter’s like “community notes”.

• Clarity in Individual Disclosures
  Individuals involved in both individual and team RPGF projects should provide clear disclosure. Ideally, they should outline their involvement and their intended allocation of RPGF funding. Along with other Badgeholders (like Lefteris here; jpnmemelord here), I found this confusing. If an individual’s impact is primarily associated with a team project, is it fair to apply as an individual? It raises questions when multiple founding team members of a project create individual applications citing their impact on projects X and Y while already receiving RPGF rewards from those projects. It’s essential to establish clear guidelines for this scenario in the future.

• Imroving Security Measures
  Many projects were entirely new to me, and among them, quite a few appeared weird, lacking even basic SSL. Implementing a badge system to indicate applications that underwent a basic security review or, alternatively, refraining from accepting projects that don’t meet these requirements would be beneficial. I understand the challenges posed by the growing number of projects, yet it’s critical. A single malicious website could lead to significant losses for Badgeholders. Let’s proactively prevent any opportunities for bad actors.

BOLD EXPERIMENT IDEAS

It’s time for a serious reassessment of the RPGF application and filtering process.

While reviewing projects, I noticed many with low or no impact. Discussions among Badgeholders and RPGF participants have revolved around the projects’ quality and filtering. I strongly believe we should shift our focus from quantity to quality. Though it may mean rewarding fewer projects, the chosen ones offer the highest value to Optimism/Superchain. Whether it’s increased txs leading to sequencer fees (check analytics here) or projects fostering more projects (ETHGlobal report), they generate a significant impact.

Over a couple of RPGF rounds, every impactful project will receive recognition, bridging the gap between impact and profit. For instance, RPGF3 saw many high-impact newcomers like Zora and Mirror (yes, both VC-backed) - contributing significantly to the space. Repeating Badgeholders likely noticed them and allocated higher OP amounts compared to RPGF2 projects reapplying.

Think about the Badgeholder nomination process—you express your interest but can’t simply apply; you must truly deserve it. This means contributing meaningfully to Optimism or the public good space—whether for the Optimism Foundation, existing Badgeholders, or top RPGF recipients to nominate you. This process makes sense. Why not apply the same for RPGF applicants?

Imagine if impactful projects that earned their place through previous RPGF rounds nominated 1-2 other projects for the next round. They could vouch for their impact on Optimism Collective and the broader public goods ecosystem, sharing comments and helping them onboard, extending a warm welcome.

Considering the expected growth in RPGF rounds, this model could lead to quality-driven project growth, filtered by the community of public good impact makers. I can hear arguments, citing against decentralization or a gated experience. The reality is, if you’re genuinely impactful, people will notice. Whether it’s in RPGF4 or later, deserving projects will eventually get their due recognition (esp with data driven approach). And for those who claim they lack funding or resources to sustain themselves until then, it’s often an excuse, easier to rely on instead of being relentlessly resourceful.

Let’s do some simple math:

If we want a quality RPGF project to nominate and provide guidance for 1-2 other projects, assuming we take the top 400 RPGF recipients from this round:
A) 400 projects nominating 1 project each = 800 projects in the next RPGF
B) 400 projects nominating 2 projects each = 1200 projects in the next RPGF

This feels like a healthy way to grow. With a long-term perspective, we’ll gradually add a couple of thousand projects in a quality-first, sustainable manner. That’s how we’ll succeed.

Another interesting experiment:

What if we allowed all applicants to upvote their favorite impactful projects among all submissions?

Imagine if each project had the ability to upvote/downvote and/or assign a 0-10 impact score. Admittedly, biases might be associated with this approach, but it could serve as an intriguing experiment for community-driven moderation among RPGF projects. It’d also make sense to integrate a Sybil resistance solution and enable the broader public goods community to provide comments/feedback.

Yes, on one side, this might lead to group think, etc. However, on the other side, this could enable projects to debate openly their ‘impact score’, disproving others if necessary, by sharing evidence or additional data about their impact (assuming the RPGF project page will be dynamic).

---

Other Considerations

1. Allocation of OP for RPGF and Sequencer Fees

Among polynya comments, i found #3 “I’d like to see all OP allocated towards RPGF be burned” & #4 “RPGF will instead be awarded through a share of sequencer revenues” - particularly interesting. These points are worth exploring & discussing altogether.

2. The frequency of RPGF should be questioned.

With a constant increase of projects, allocation amounts, and responsibility placed on each Badgeholder, it’s critical not to rush RPGF. Otherwise, we might risk failing what could be one of the most exciting and critical experiments in impact x public goods funding.

Personally, I vote for a yearly RPGF - in consensus with Thomas here, among others. Ideally, it’d be great to see RPGF taking place late in Q4 each year. This schedule would allow for 10 months dedicated to providing impact, followed by 2 months for evaluation and reward distribution.

At the start of the year, the Optimism Collective could signal the impact goals—for example, an increase in projects building specific use cases or raising awareness about something Optimism/Superchain related. Similar to RFPs, we could collectively create Public Impact Requests (‘PIRs’) and jointly determine the most important Collective’s impact aspects for the upcoming year.

During the 10 month period before voting, efforts could focus on raising awareness, encouraging projects to explore these impact verticals, and aiding their growth and success on Optimism/Superchain.

Final Thoughts

I’m looking forward to seeing the results of RPGF3, conducting a retro with the Collective, and aim to further RPGF’s success. I’m deeply grateful for RPGF3 and everyone involved—it was a fantastic, memorable, positive-sum experience. I hope this feedback contributes to enhancing the RPGF experience. Thank you!

[glehen.eth]

I sometimes read the texts to try to participate, but it’s too much reading, is there a summary of RPGF3 activities?

[Eagle_Cloak]

I absolutely applaud @revmiller for the time and effort in synching the feedback so eloquently.

One note under Badgeholders - ‘More Transparency Needed’. I agree that it is critical that Badgeholder must disclose their interests.

I’m also reminded of a Roman saying, “Who guards the guards?” i.e., how can the OP ecosystem ensure that Badgeholders commit to the responsibilities outlined in the post?

To help address this, how about creating an Internal Badgehodler Oversight Committee(s), made up of Badgeholders, that tracks Badgeholder’s progress and makes quarterly reports only visible to Badgeholders?

For example, the committee develops processes and procedures to track the Badgeholder involvement, and gives recommendations. If the Badgeholder receives 3 negative marks on three separate occasions, the Badgeholder Oversight Committee removes the person’s badge.

These are some ideas. Something like an Internal Badgehodler Oversight Committee would take more fleshing out, but hopefully, it makes sense.

Cheers,
EagleCloak