Huge thanks to @danelund.eth for publishing an An internal operating procedure designed for season 6. He committed to designing the S6 internal operating procedures, and completed this task flawlessly. Thank you, Dane, for your dedication and exceptional work. Your efforts have set a high standard for our council, and we are immensely grateful for your contributions.
Grants Council Internal Operating Procedure
The internal procedures in this document govern the activities of the Season 6 Grants Council. The document sets forth the processes the Review Teams will follow in order to screen applications and conduct a formal review of proposals submitted for grants.
Review Teams
In the rules below, the Review Teams shall be the Superchain Review Team, the Optimism Mission Review Team, and the Audit and Special Mission Review Team.
The Milestones and Metrics Team will be a standards team and will not provide a graded review of proposals, other than as specified below or in the charter, in the regular course.
Cycle Review Procedure
Each Cycle will have two periods, a Submission Period and a Decision Period.
The Submission Period is designed to accommodate the submission and fair review of applications. This period is designed to enable an initial review of submissions in advance of the Decision Period.
The Decision Period is designed to facilitate efficient, sound and diligent grants decisions.
The Cycle Procedure is set forth in detail in the sections that follow.
Standard and Procedures for Review Teams
Each of the three Review Teams will conduct three stages of review within each cycle: an Intake Review; a Preliminary Review; and a Final Review.
Intake Review Standards and Procedures
The first form of review for a given proposal will be an Intake Review. The Intake Review will be a sufficiency review, conducted to eliminate proposals that do not adhere to minimum assessment standards from the later review stages of a given cycle.
The first reviewer assigned to a proposal shall take one of two actions after an initial review of the proposal, either confirm that the proposal meets the minimum standards for review or reject the proposal for substantive review. A reviewer may reject a proposal for substantive review on one of the following grounds:
- The proposal has been marked ready for submission but has not been completed;
- The proposal lacks sufficient detail for the reviewer to address each item set forth on the relevant review rubric;
- The proposal breaches the Grant Policies or Rules of Engagement for the Optimism Collective;
- The proposal has a clear lack of clarity or effort that it would be premature to review for substance;
- The proposal is not aligned with the mission request objective;
- The proposal request is above the author’s tier;
- The author does not answer questions within 48 hours of intake;
- The OP distribution is not clear or developed;
- The project is not deployed and deployment is a critical feature of assessing the merit of the project; or,
- The application is trying to circumvent grant restrictions set forth by the Optimism Collective
- The application is not for OP Mainnet (Except Intent 3B applications)
Reviewers shall have the discretion to select any of the above criteria as grounds for rejection.
Preliminary Review Standards and Procedures
If a reviewer determines that a proposal passes the Intake Review, a Preliminary Review will follow. The same reviewer will review the proposal and assign a score based on the relevant Review Rubric for the line of review.
A reviewer may revisit a proposal that has been updated during the Submission Period to update the score based on any new edits. It is the responsibility of the proposer to notify the relevant Review Team that an edit has been made, and edits may only be considered if time permits during the cycle.
Final Review Standards and Procedures
During the Final Review, the Milestone and Metrics and / or Lead will provide a list of proposals that exceeded the cutoff score specified in the Review Rubric for each line of review. The proposals will then be assigned to two other reviewers on the relevant Review Team to be reviewed in order of Preliminary Review score (highest to lowest). Each reviewer will review the proposal and provide questions or feedback to the proposer. Based on the volume of applications, the Lead will set a date at which proposals will be considered final and no further changes will be considered. At this point, each reviewer will finalize scores.
Each Review Team will review the list of proposals ordered by an average of the two highest scores received (between the one Preliminary Review and two Final Reviews). Each reviewer will then have the opportunity to agree with the final ranked list or object to specific rankings. If there is an objection to the specific ranking, the relevant proposal will be reviewed by each reviewer on the Review Team and the proposal will receive the average of score of all of the reviewers (including any updated score from the preliminary reviewer).
Once the Review Team agrees to the ranked list of final scores, the team shall vote on the number of proposals (n) to be named finalists. The number shall be confirmed by a simiple majority of the Review Team. Proposals ranked 1-n shall be named finalists for the cycle.
Other Review Procedures
- Intake Review
The Milestone and Metrics Team and / or Lead shall endeavor to assign proposals for Intake Review chronologically, with the earlier submitted proposals being reviewed first.
If a proposer challenges the reviewer’s Intake Review determination, the determination will stand for the relevant cycle; however, the Milestone and Metrics Team will consider the objection and review whether the proposer gave due consideration and a reasonable ground for rejection and document the result. The Milestone and Metrics Team may choose to make the results of the objections and determinations available to the community in advance of any future elections.
If a proposer does not pass Intake Review, no substantive review will follow in a given cycle unless the proposer affirmatively reapplies in a subsequent cycle. The Milestones and Metrics Team or Lead will notify proposers that their proposals did not pass the Intake Review by the end of the Decision Period.
The Milestone and Metrics Team and / or Lead shall track which reviewer manages the Intake Review in a given cycle and endeavor to assign a different reviewer for Intake Review in a subsequent cycle.
- Preliminary Review
Reviewers are encouraged to provide feedback to projects; however, priority should be given to projects that [are within 2 points of passing to the Final Review]. Reviewers are not required to provide feedback during the Preliminary Review, but the Milestone and Metrics Team should endeavor to assess how frequently reviewers provide comments during this period.
- Final Review
Proposals that proceed to the Final Review in one cycle will be considered in the Final Review of each subsequent cycle unless the relevant Review Team votes to require the proposer to reapply for further consideration. Requiring reapplication should be reserved for circumstances in which some core aspect of the application would need to change for the Review Team to view the proposal as being competitive.
Feedback should be provided to each proposal that reaches the Final Review. If a proposal reaches the Final Review in a given cycle by default, feedback may be reiterated or updated based on the degree to which the proposal is updated or remains the same as it had in the prior cycle.
- Timing
In general, for each cycle, the Submission Period will last one week and the Decision Period will last two weeks. The Council may amend the duration of each period by a simple majority vote.
- Review Rubrics
At the beginning of each cycle (not later than the end of the cycle’s Submission Period), each Review Team will publish a rubric for the relevant lines of review that apply for the cycle. Generally, a line of review refers to a mission or other request for grant proposal that applies to the Review Team.
Rubrics should articulate the categories of review for a line of review, the number of points a category may receive, and the factors that differentiate scores within each category. Each rubric should specify a cutoff score for a project to reach the Final Review.
The Lead may designate reviewers to manage the drafting and proposed revisions to a given rubric. Rubrics and amendments shall be approved by a simple majority vote of the relevant Review Team.
Team Procedures
- Superchain Team
The Superchain Team will consist of the three reviewers with the highest vote count in the reviewer election. A reviewer may elect to step down from the Superchain Team at the beginning of the Season. If a reviewer steps down, the reviewer with the next highest vote count will automatically become a member of the Superchain Team and the departing reviewer will become a member of the Optimism Mission Team. The Lead will confirm any changes on the governance forum for the Optimism Collective promptly.
The Superchain Team may adopt Review Rubrics or amend its own procedures (to the extent that they do not conflict with the Grants Council Season 6 Charter or these Internal Operating Procedures) by a 2/3 vote.
- Audit and Special Mission Review Team
At the beginning of each Cycle, the Lead shall coordinate with the Lead of the Developer Advisory Board to designate a reviewer from the Developer Advisory Board.
Any Special Mission shall be designated a Special Mission at the beginning of a cycle by the Lead, provided a simple majority vote of the Optimism Mission Review Team votes in favor of the designation.
- Milestone and Metrics Team
The Milestones and Metrics Team will be responsible for assessing and tracking milestones for grants that have reached the Final Review as well as finalists from prior seasons. The Milestone and Metrics team will designate a visible space for the community to view its determination on milestones.
Prospective Proposals
In order to be considered as a finalist, a proposal must include critical milestones. Critical milestones are intended to show good faith effort and a baseline of success toward the goals articulated in the grant proposal.
The Milestone and Metrics Team will confirm Milestone Guidelines for assessing milestone sufficiency at the beginning of the Season by majority vote of the Team. The Milestone Guidelines will be aimed at creating discrete, trackable milestones for each successful proposal.
At the beginning of each Final Review for each cycle, the Milestones and Metrics Team will review each proposal selected by each Review Team for Final Review to determine whether the proposal’s milestones meet the standard of sufficiency set forth in the Milestones Guidelines.
In order to be eligible to become a finalist, a proposal must receive a favorable vote of a majority of the Milestones and Metrics Team members verifying that the proposal’s milestones meet the standards of sufficiency.
Past Proposals
The Milestones and Metrics Team shall make recommendations to the Foundation or Optimism Collective as to whether or not to proceed with the remainder of a grant from a prior Season. In general, the Milestones and Metrics Team shall not recommend proceeding with a project unless all critical milestones have been achieved (per the vote of the Team).
The Milestone and Metrics Team is generally responsible for voting on whether or not milestones have been achieved. In circumstances where the Team determines that it would require expertise beyond the capacity of the Team, the Milestone and Metrics Team Lead may submit a milestone for consideration of the Team that originally voted to designate the proposal as a prior finalist. In such cases, the majority vote of the relevant team shall be considered determinative.
Standard procedure
Proposals for the continuation of a grant should be submitted to the Milestones Hub.
In general, proposals will be reviewed only after a project confirms that all relevant critical milestones have been completed. However, the Milestone and Metrics Team may consider requests to review individual milestones in the case that a project requires feedback on a milestone to continue.
Absent an approval for a greater amount of time, critical milestones will be considered past due after six months from the delivery of the grant for Experiments proposals and twelve months from the locking of the grant for Builders proposals.
Milestones and Metrics votes shall be recorded.
Other procedures
In cases where a project has failed to complete its critical milestones in the allotted time (either the six or twelve month defaults for legacy Experiments and Builders or the time allotted on the specific proposal, whichever is greater), the Team will default to recommending against proceeding with any further steps of the grant.
In the case where a proposer makes a proposal to change the substance (or extend the time) of a critical milestone, the Team may consider the change only if (a) the Team determines that there is substantial good reason to support the request and (b) at least 66% of all of the Council reviewers vote in favor of the proposal.
For milestones pertinent to the Audit and Special Missions Team, the Team shall wait for the Developer Advisory Board to confirm the completion of the critical milestones prior to confirming completion to the Foundation.
At any point, if the Council becomes aware of a matter that should be recommended for a grant policy violation, the Council may revoke its recommendation to move forward with a grant.
The guidance in this section yields to any contrary rule, guidance, or vote from Token House or the Foundation.
Reviewer Metrics Accountability
The Milestone and Metrics team will adopt any specific metrics and processes as mandated by the charter.
Official Record and Rule Changes
The Council will maintain a tracker of proposals considered, finalists named, milestones for finalists, and the progress / completion of milestones.
On a bi-weekly basis, the Council may review the effectiveness of its internal procedures and make proposals for improvement. The Council can amend the internal procedures by a simple majority vote of the members of the Council; and, each Team may amend the internal procedures that pertain to the specific Team (insofar as the amendment does not conflict with the Season 6 Charter or these Internal Operating Procedures) by a simple majority vote of the Team. Any amendment to these Internal Operating Procedures must be recorded as a comment to this post and reflected on the Landing Page. It is recommended that the Team defer the effectiveness of an amendment until the next review cycle, unless the amendment creates a critical improvement to the review process.
Rules of Decision
If one or more Team members abstains from a vote, a vote will pass by the simple majority vote of the remaining members, provided there are more than one voting members remaining. If only one member of a Team votes, the result will be to take no action on the proposed matter. For instance, if the matter being voted on relates to whether or not to include an application in the final review, the result will be not to include the application in the final review.
Any Council-wide votes must include members of each Team to achieve a quorum.
The Lead may correct any clear errors in the Internal Procedures or resolve any conflicting provisions. For resolving conflicts, the Lead may submit a question about the best resolution to a vote of the Team members, the results of which may be recorded on the Landing Page.