Retro Funding 5: Application Review Process

[system]

The Retroactive Public Goods Funding (Retro Funding) application review process is applied to decide which applicants are eligible to participate in retro rounds and is used to enforce application rules and eligibility criteria. This process is run by a selected group of Citizens during a 2 week period following the project sign up process. For a full overview of the application review process see Retro Funding: Application Review Process.

Reviewer Expectations

Any Citizen who participates in the review process should meet the following expectations

1. Time Commitment: As a reviewer you should expect to commit a minimum of 10 hours to participating in the review process. There’s a need for you to have ongoing availability during the process running from Sept 6th - Sept 20th.
2. OP Stack Knowledge: You should have high level familiarity with the OP Stack and its components to enforce the eligibility criteria. Please make sure you understand and are comfortable with enforcing the eligibility criteria before you sign up.
3. Communication & Synch calls: Stay attentive to messages, announcements, and action steps provided by the Lead Reviewers. Attend the onboarding call and attend regular reviewer synch calls.
4. Stick to the rules: Thoroughly review applications according to the application review checklist. You need to provide a written explanation for your review. Diverting from the rules to implement your personal values is unacceptable and will result in removal from the review process.
5. Conflicts of Interest: You are required to declare projects you have a conflict of interest with when applying to be a reviewer. If you discover a possible conflict later on, you’re required to contact a Lead Reviewer to be removed from reviewing this project. A conflict of Interest is defined as “Organizations where you expect any portion of funds to flow to yourself or any projects from which you derive income”.

Reviewer Selection

Citizens will be selected as Reviewers via a combination of an opt-in process and random sampling. This allows for a smaller group to engage in the process, while still being fairly selected via random sampling.

1. Opt-in: Citizens can opt-in to participate in the application review process until Aug 31st 19:00 UTC. To opt-in please fill out this form (https://deform.optimism.io/reviewer-sign-ups) and be sure you have read the Reviewer Expectations.
2. Random Sampling: 30 reviewers will be randomly selected from the citizens who have opted-in to review. Selection will take place on September 2nd and citizens will be notified via email about the outcome of the random selection.
3. Lead Reviewers: Diego, the Citizens’ House Gov NERD, will act as a non-voting lead reviewer, responsible for resolving questions, collecting votes, managing the timeline of the process and facilitating collaboration. One additional non-voting lead reviewer will be selected by Diego to manage and facilitate the process during European working hours. You can add your application to be a lead reviewer in the sign up form.
4. Reviewer Groups: Instead of randomly assigning reviewers to cases, there will be 6 reviewer groups with 5 reviewers each. Each reviewer will be randomly assigned to a group. This design aims at reducing the operational and technical complexity, as well as allow for more collaboration among reviewers.

Application Review Process Overview

1. Application is submitted via retrofunding.optimism.io
2. Application is assigned to reviewer group (see “Reviewer Selection”)
3. Rule Violation Check: 2 reviewers, from the relevant reviewer group, review the application for rule violations. If both reviewers approve the application, it qualifies for the round.
4. Full Review: If at least 1 reviewer rejected the application, the remaining 3 reviewers in the reviewer group are asked to submit their review. If the majority of reviewers approve, the application qualifies for the round.
5. Appeal: If a majority of reviewers rejected the application, the applicant can hand in an appeal. The appeal will then be reviewed by 5 reviewers from a different reviewer group to the one that did the initial review. The reviewer group decides if the application is rejected/approved by majority vote.
6. Application Edits: Edits to a projects application are only allowed in cases where the application form itself is misguided or deceived applications, bugs caused input errors or the application category was selected wrong. The Foundation decides which applications are allowed to be edited. Relevant edits are made following the Full Review and are considered within the Appeals process.

Application rules

Below are rules for which violation will result in the removal of an application from a Retro Funding round.

1. False statements & deception - false claims about your contributions, past impact or funding & grants are not allowed.
2. Hateful Content - No racist, sexist, or otherwise hateful speech, no discrimination as defined by the rules of engagement
3. Deceiving badgeholders - Malicious content that could cause harm or unintended consequences to users.
4. Fraud & Impersonation - Claiming to be a brand or person you are not. The Grant owner must be directly affiliated with the project; the funds must go to the project.
5. Advertising - Using Retro Funding application to showcase something you are selling like a token sale or NFT drop
6. Outside of Retro round scope or not meeting eligibility criteria - Contributions that do not have a clear relationship to the scope of the retro round, or do not meet eligibility criteria
7. Spam - Applications containing spam, such as irrelevant answers, plagiarized content, broken or unrelated impact metrics and contribution links. Applications in languages other than English*.
   • This will help simplify the process as English is the working language of the majority of Badgeholders. Please ensure you translate any content that’s part of the application.
8. Duplicate applications - Multiple applications from the same individual, project or group which apply for the same impact.
   • Members of contribution paths (Council Members, Ambassadors, NumbaNERDs, SupNERDs, TechNERDs, Translators) or Councils can’t submit individual applications for their work within the relevant workstream, as each workstream will apply as a project.

Review Checklist [Draft]

Reviewers will be asked to go over this checklist when performing the initial review of an applications. Please note that this is an early draft.

1. Spam: Does the application contain spam, such as irrelevant answers, plagiarized content, broken or unrelated impact metrics and contribution links? Is the application in another language than English?
2. Eligibility: Is this application Eligible, following the eligibility criteria of the round? [Reference category specific criteria checklist] 2.1 Is this application eligible for another category, different to the one they have selected?
3. Optimism Grants: Has the application correctly reported the grants they have received from Optimism (including Token House, Citizens’ House an Foundation grants)?
4. Venture Funding: Has the application correctly reported their venture funding based on public information?
5. Pricing: Has the application correctly reported their current pricing? See by checking the pricing on the projects website.
6. Impersonation: Does the application have a verified Github repo which belongs to the organisation they claim to be? Do any linked artefacts not belong to the project?
7. Future promises: Does this application make promises about work they will do in the future?
8. Advertising: Does this application actively promote a token or NFT sale in their application or on their website?

Outcome Overview

• Total Applications reviewed: 149
• Initial Review Approved: 62
• Initial Review Rejected: 77
• Appeals: TBD
• Appeals Approved: TBD
• Appeals Rejected: TBD
• Total Applications Approved: TBD
• Total Applications Rejected: TBD

You can find the results in this spreadsheet here

[Jonas]

Updates to RF5 Application Review Process

The sign up to become a reviewer has been closed with 11 total submissions. This is much lower compared to previous rounds, which can partly be explained by the requirement to have relevant knowledge of the OP Stack. The application process for Retro Funding 5 has closed with 145 submissions.

As a consequence of the lower participation rate, the following updates will be made to the process:

1. 2 reviewer groups: There is roughly equal distribution across the three categories, while early observations show that the OP Stack Tooling & OP Stack R&D categories have higher rate of non-eligible applicants. As a result, there will be one group dedicated to reviewing Ethereum Core Contributions and one group dedicated to reviewing the OP Stack Tooling & OP Stack R&D categories.
2. Appeals by the same group: The project appeals will be reviewed by the same reviewer group. This ensures that reviewers have sufficient context on the categories eligibility criteria to make informed decisions. While appeal reviews by a different group are overall desirable, the goal of consistent decision making within the application review process takes higher priority.
3. Jump seat reviewer: One reviewer will be on standby in case another reviewer drops out.
4. Removal of “Rule Violation Check” step: To increase the efficiency of the process, all applications will be immediately reviewed by 5 reviewers, instead of an initial check by 2 reviewers. Considering the low number of applications, the reduction in reviews by this optimisation is likely not worth the additional time required to complete it.

[sam.ng]

When is the deadline for receiving an email to know whether the application is approved or not?

[Jonas]

The initial phase of reviews has been completed

• Total Applications reviewed: 149
• Initial Review Approved: 63
• Initial Review Rejected: 85
• Pending: 1

You can find the results in this spreadsheet here

If your project has been rejected, you can hand in an appeal by Thursday 18:00, Sept 19th. Below you find a guide on how to hand in an appeal.

1. Find your project in Charmverse: You can find a link to your project page in Charmverse here, this has also been shared with you via email.
2. Log-in: Identify who on your team is able to submit an appeal. To do so, please refer back to the original application and locate the ‘Authors’. Any one of these authors will have the ability to submit an appeal for your project by logging in with their Farcaster account.
3. Rejection reason: Review the rejection reason by clicking on the comments of reviewers (see screenshot below)
   
   
4. Submit appeal: Locate and click on the appeal button in the bottom right hand side of your screen. In order to view the reasons why your application was rejected, click on the comment box on the right hand side of your screen.
   
   
5. Appeal outcome: Your appeal will be reviewed and you will be notified of the outcome on Tuesday, Sept 24th

If you experience any issues, please jump into the #retrofunding-discussion channel in the Optimism Discord

[Jonas]

The Review process has completed!

• Total Applications reviewed: 149
• Applications approved: 79
• Applications Rejected: 68
• Applications Withdrawn: 2
  You can find the results in this spreadsheet here