RetroPGF Round 3 Feedback Thread

Detailed experiment, only suggestion is posting clear impact driven KPIs with learning every month for evaluation of success and new experiments and introspection.

4 Likes
  1. There will be a new set of badgeholders, made up of existing badgeholders and new additions. More details on that coming soon
  2. No, all projects who want to participate will need to sign-up for round 3
5 Likes

It would be helpful to have guidelines on what constitutes a separate application if the work is done by the same parent org but involves different set of contributors.

8 Likes

great Hope that it will come true!

4 Likes

Hello fellow badgeholders! I wanted to start a conversation for feedback from this round so we can share our experiences and make improvements for the next round.

Too Many Projects

The biggest piece of feedback from the last round was that the volume of projects applying was overwhelming, but somehow this round we ended up with 6 times the amount of projects, so it feels like that piece of feedback wasn’t heard. Imo there needs to be some kind of barrier to entry for applicants, and not put the responsibility entirely on badge holders to review such an insane amount of projects, most of which are out of scope.

Lists

Lists are a great idea in theory but not in practice, which is the case with many things in governance. There’s no level of expertise required to make a list, so we’re seeing a large amount of lists that include low impact projects which then highlights them to other badge holders. This is harmful because low impact projects receiving funding takes away from high value projects receiving funding that have actually contributed to the ecosystem in significant ways. There have been some useful aspects of lists, like I have personally looked at some of the technical lists. But overall, I think lists are more harmful than helpful.

Voting Experience

The voting experience this round has been slightly better, but I have a few pieces of feedback I shared with the agora team before voting started that I think could make it better
-have the ability to mark a project as seen or “don’t put in ballot” option
-have the ability to allocate percentages as well as dollar amount
-get rid of the lists function

I was also under the impression that there were going to be tools incorporated into the voting experience, like what happened to Pairwise?

Summary

Really my biggest piece of feedback is that we need a way to filter projects applying so we don’t end up with over 600 projects again, most of which are low impact. I realize this is still very early stages for retro PGF but I am really disappointed in how this round is going and I’m hoping we can make some big improvements going forward.

I would love to hear the opinions and experiences of others so we can hopefully work together to make the experience better for badgeholders and applicants as well. Thank you for all of your hard work!

28 Likes

We do see the pairwise tweet :hatched_chick: which categorizes the different projects but not sure :thinking: if they filtered anything out based on quality. Pretty sure it includes everything that passed the review process.

There are also some interesting things happening in the OP discord chat :speech_balloon: for RetroPGF discussion surrounding a few projects that are in this round who are being called out for certain actions as a result of grant funding received from Optimism governance in the past.

I do remember some talk about Pairwise being used as a tool to help build lists. Not sure about the level of integration into Agora UI for round 3 but definitely wanted to point this out in case you might have missed it.

https://www.twitter.com/pairwisevote/status/1728096861167169559?s=46&t=_8dkepIOA6H5XhasVM4F9g

7 Likes

If I had to give a single positive bit of feedback it would be that amount of effort being expended, the care people are taking in making votes, and the high level of discourse is really nice.

If I had to give a single negative bit of feedback it would be that it seems incomprehensible to me why there was not a question on the application form concerning VC funding. Clearly, this should be included. Especially if you’re asking for how much funding a project has received from grants. (And make it more clear that this only meant grants from the Optimism community, as I’ve seen someone say – if that’s the case.) I’m not making a comment on whether or not VC funded projects should be included, just that you must ask for the number if you’re going to ask for grants. It’s very unfair otherwise.

13 Likes

I feel you Katie.

As a RPGF apolicant, i have seen that few badge holders completely discarded The Optimist media and just added it back in their list cause i raise the interest.

Same feedback was given: Ooops we miss you out of the 600 projects

I feel that smaller projects are passing below the radar due to this.

11 Likes

I think it’s impossible to expect every badge holders to go through 600+ project and vote on them. It’s also quite impossible for a lesser known project to be voted on by enough badge holders for quorum. Last round was more fair for smaller or lesser known projects as there was no minimum vote requirement. The result would probably be only the most well known projects are funded that get the lion’s share of the OP, while the majority of applicants get nothing or struggle to reach Quorum. I know there is 1 week left but for now it’s not a great look.

Our project brought in 25k coinbase users and is being overlooked while some project that created game that no one plays can potentially be getting hundreds of thousands of OP.

8 Likes

For RetroPGF3 I’d like to see:

  • lists without values.
  • vote of zero not count for quorum
5 Likes

On Lists
Lists create recency bias by elevating whatever the creator decides to highlight. I have seen projects listed incorrectly where they do not belong, while others are completely neglected when deserving. A reputation system, or perhaps the ability for badgeholders to comment directly on applications would be far more constructive. The sheer number of applications (over 600) means that any list created will be prone to error.

On Ballots
There are plenty of deserving projects that have no visibility. I think the quorum is quite arbitrary and it is strange to see some meme projects and small blogs getting more ballots than heavy hitters in the industry. Many projects have resorted to shilling their applications on Discord and X. I think this is enigmatic of a wider visibility issue. I have heard people say that its early and all votes haven’t been casted but if you’re a smaller project you face the risk of not meeting quorum and their is an oppurtunity cost to doing nothing… which is probably why there has been so much feedback

6 Likes

Just to coin in here. I haven’t been on the forum in a bit because I’ve had a really hard year and the grieving process has taken a toll.

I agree with the points that RPGF needs improvements, but I also think it is important that we make strides toward the change we want to see instead of waiting for the change to happen.

As an applicant for this round, I am discouraged by the current results but I have faith we may meet quorum and that others will get the recognition they deserve as well, even if it’s not this round.

One thing I’d like to point out, it takes courage to apply to any round or for any type of funding. For others it’s easier, for those with charisma and marketing skills it’s also easier.

Raising funds is simply not easy for everyone, but RPGF is meant to be accessible, not within arms reach and floated like a carrot.

We didn’t apply previously as we already don’t feel like we contribute enough compared to popular projects and known brands. Even though we dedicate 99% of our time and resources to developing open source and publicly accessible goods and services. (1% spent touching grass)

We made the decision not to apply for Round 2 and instead worked tirelessly and put our best efforts toward creating impact— knowing we would shoot for Round 3.

Now, RPGF feels skewed and personally, a bit like a popularity contest.

Since we started our project we’ve put every dime we have toward building for the public and in public. It really does feel like at some point we should of tried harder to just raise capital and sell out. Instead, we have debt from borrowing and have actual overhead because our office space is open for the public and anyone can use it to access technology. And, we don’t ask for any money for providing a safe space to grow.

I’ve lost track of the number of people we’ve helped within our community or green pilled with what it means to build in Web3. But, none of that has added up to anywhere near the ‘impact’ supposedly created by other projects.

Not sure, how many projects can claim their users are actually real? or that they interact with real people on a day to day and inform them of what Web3 really is? What is being built with blockchain and cryptography.

Not, ‘incentives’ in the form of token handouts because woah APR is high, let me deposit and farm :fire:

Not sure how yield farming is really considered proof of impact aside from sequencer revenue getting compounded and then shovelled back into the farms. But MoAr TxNs. Unreal.

What about bank statements? Proof of Expenses etc.

Edit: Project didn’t make quorum.

4 Likes

If contributions, regardless of their size, could be recognized and rewarded, that would be excellent. I don’t believe there should be contributions that are ignored. Currently, it seems that we only acknowledge major contributions.

5 Likes

Hi, I’m Stalim from South Korea :star_struck:
(Right, as you know, it’s the nationality of BTS)

Let me briefly explain why I became interested in Optimism

By chance, I was rewarded by RetroPGF 2 when I translated the official article into my native language. (OPTIMISM TRANSLATOR)

Actually, I didn’t know how it worked, and I didn’t know why it was compensated
But it felt good to be rewarded, because it was a reward that I contributed even a little to Optimism.

After that, I found a way to contribute to more optimism, but I’m working for a company, and I’ve found myself much lacking blockchain technology…

So whenever I have time after work, I wanted to contribute to Optimism by writing a translation, and I did so.

As I contributed more and more and more, and as I participated in the community, I learned how retro projects work.

So I ended up participating in this thread. Nice to meet you again,

Let me tell you what I think about Retro3
(Please know that this includes an appeal to support the project I am involved in)

1. Badge holders, please pay attention to small and precious projects.

Many people don’t know as much about blockchain technology as they think.
So have to start with the small one.
have to learn something as part of that, but it’s a little hard to read in English.
So translation is small but important
If you look at the number of votes now, the big projects have already exceeded 17 votes, because they have great funding, marketing, and technology.
But other small projects or underfunded projects seem very difficult to get a vote on
Increasingly the so-called ants (like me) will have less participation, and then the community will not be activated and again it will be their own league.

2. Please look at the power of the community.

If you look at our Discord, I think it’s much more active than other projects
I think it was possible because the OP Community Contributor Project (NERDs, translators, ambassadors, TechNerd) did a great job in that role.
And there’s also an article on Optimism’s official doc explaining them.

3. Badge holders who haven’t already voted, please vote. The participation rate is too low.

Voting is both a right and an obligation.
Please take a look at 600 projects.

4. still trust the badge holders.

I’m sorry for writing in a rambling way.

> I am a proud Optimist translator. :saluting_face:

Optimism Translators

OP Community Contributors

6 Likes

I also thought this was a great idea, but now it seems to be having a negative effect. It’s allowing low quality projects to get into ballots. The volume of applicants also prevents voters from digging into the details. For example, I noticed that two members of Ethereum Honduras are reporting the same work (sounds like double reporting), and one of them even reported onboarding himself as an artist as “impact”.

Their work consist of deploying NFT contracts with one holder. Mis microrelatos (MICRR) Token Tracker | Optimism, minting POAPS (not on Optimism), and transactions between each other’s wallets which I’m not sure if I can share here despite it being reported on their application - don’t want it to be interpreted as doxxing :woozy_face:

This impact is being claimed on both applications here:

Randomness Ceremony

FilosofiaCodigo

Both of these applications are already appearing on ballots and lists by @ccerv1 and @brichis - no shade to them, just pointing out how easy it is to slip into lists, then ballots based on name recognition and titles like “Randomness Ceremony” which I initially related to the KZG Ceremony.

Both of these applicants are core of Ethereum San Pedro Sula and Ethereum Honduras, Which is a part of EthLatAm, which has overlap with EthKipu and SEEDLatAm.

6 Likes

I find it pretty reassuring that others are struggling a bit with the number of projects, was worried that I was missing a trick somewhere!

I do like the lists feature, at least in terms of getting collections of valid projects from people who’s expertise in an area you trust. It doesn’t really work in terms of the OP amounts when you’re adding different lists to your ballot, but I think this can be improved for next time with a simple addition: when you import a ballot there could be an option to assign a certain amount of OP to that list, then when the projects are pulled into the list, the ratio of OP per project set by the list builder is maintained, but the actual values are based on dividing up the total you assign by that ratio.

Another feature I’d appreciate for next time is a set of filters based on the project’s characteristics, in particular the amount of funding already received would be useful… I don’t need to waste time looking through DeFi dApps that have already received millions and millions of OP/$, so a way to just remove those from the set to evaluate would be really useful.

Finally, I wonder if nest time we need to rethink the 4 categories, as some of the shoehorning of projects’ impact into these seem… shall we say ‘tenuous’ at best!

15 Likes

As I mentioned a month ago, before the start of the voting period, I have some concerns around the high quorum required for this RetroPGF Round. While it helps prevent collusion among badgeholders, it might be doing more harm than good by making it very challenging for projects with small to medium-sized contributions to the Optimism Collective to get retroactive funding. The large number of applications complicates this even further.

Jonas previously mentioned here in the Forum that “the goal should be for all applications to meet the quorum”, but that seems pretty hard to achieve with one week left until the end of the voting period.

To illustrate this point, I spent some time today making a visual overview of how many RetroPGF 2 recipients already reached the quorum in RetroPGF 3:

While the top recipients of RetroPGF 2 have no issues getting above or near the quorum, it gets progressively more difficult as we approach the average recipient of the last Round.

But ensuring that RetroPGF remains accessible for smaller projects that have a positive impact on the Optimism Collective is extremely important for this experiment to succeed. At the end of the day RetroPGF is not just altruism, it’s also a critical growth strategy and key differentiator to other blockchains.

Right, we want developers and others to start building projects that benefit the Collective knowing that they’ll get retroactively funded for the positive impact they have had. Without a high degree of certainty that RetroPGF Rounds take place regularly and that past impact is fairly rewarded (be it 5K OP for a small contribution or 100K OP for a large contribution), we can’t expect talented individuals to commit tens or hundreds of hours on working for the benefit of the Optimism Collective.


I also want to point out that I had been hoping to see someone creating a List of completed Ecosystem Project Ideas, as the OP Labs and Optimism Foundation team earlier this year published a number of project ideas in the Ecosystem Contributions board they believe would be “strong candidates for RetroPGF”.

Out of the 55 Ecosystem Project Ideas and Draft Project Ideas, I was able to identify 10 which were completed and also applied for this RetroPGF Round. I put together a Google Sheet here with some more information and would greatly appreciate if a badgeholder was to create an official List out of it.

15 Likes

We would be baffled :hushed: if the efforts we put in since the last round were not rewarded in round 3 due to the fact that as soon as round 2 ended we amplified our focus :person_in_lotus_position: on the Optimism collective ten fold.

The amount of work that we were able to achieve between rounds was strenuous at times and honestly we could have used more support from round 2. Our project is mainly funded by RPGF and Gitcoin grants.

We are also happy to say that our senior developer was able to get surgery for an emergency appendix operation thanks :pray: to RetroPGF we received. So it literally kept the project alive.

It will be very interesting to observe these final days and see exactly :+1: what badge holders decide for round 3.

2 Likes

Full article here nanobro.eth | RetroPGF3 design can be improved

shorter version below.

Ballots - The major problem

In order to be eligible, projects need 17 ballots from badgeholders. This is likely the major flaw here.

Origin of the number 17 has no explaination. Many badgeholders and contributors have raise concern in Round3 design forum since before start.

Many projects trying to get their voice heard. Here is some notable one.

Numbers aren’t looking good.

While 11% of badgeholders doesn’t sound big. Reality is not all of them interested in your category.

This becomes a problem when projects have confidence in our work but aren’t certain whether they will receive reviews from 20 people.

At the end of the day, all badgeholders are working really hard and should earn respect.
But some undiscovered projects likely not get enough review. And it will ruin OP.

Some improvement options?

Some screenshots above are also good options.

And here another one. Learned from retroPGF3.
We can’t be so sure how many badgeholders are tech orient. How many are education orient.
We can set proper criteria (e.g. 25% threshold) and let badgeholders select their expertise.

For example

  • 50 badgeholders wanna review 165 OP Stacks.
  • 40 badgeholders wanna review 472 End user UX.

We get this.

We can see that End user UX need more badgeholders to review.

Badgeholders in each category might divide their work and cross-check with other badgeholders in the same categories.

  • Considering the reviewer-to-project ratio, we can take prompt action and plan ahead before any issues arise.
  • This approach ensures that all projects receive reviews.
  • We can adjust the number of badgeholders in proportion to the number of project applications.
  • Projects eligible for multiple categories should submit work for each part separately.
    • RetroPGF should allocate OP for each category before commencement.
      • Possibly allowing badgeholders to vote for percentage allocations.

And, as always, Stay Optimistic! :red_circle:

11 Likes

@nanobro I’m glad that you posted this screenshot by alpacasRule, it explains very well how many of us feel!

Retropgf2 data: 195 projects, most badgeholders tended to distribute their votes among 20-40 projects, with the median badgeholder allocating their votes among 30 projects.

Why did we decide the arbitrary threshold of 17 votes minimum while knowing this data from the previous round?

6 Likes