Milestones & Metrics Council Season 7 Retrospective Report

Elected Representatives 💼 Council Communication Threads
1

1. What is your assessment of the impact KPIs that were set in your Budget Proposal at the start of the Season? Have you made progress towards, or achieved, these milestones or KPIs? If not, why?

  • Goal: <5% of quality applicant drop off due to milestones issues
    • Assessment: This season’s grantees’ milestones were focused on TVL based and do our understanding, there were no issues regarding milestones submissions that led applicants to drop off. A more relevant outcome of the season was likely to look at S5 Cycle 19 and Cycle 22 grants that reached their 1 year lockup due date this season. Of all S5 grants, M&M reviewers reached out to each project numerous times and no projects were forfeited due to lack of attempted communication. Further, no grantees’ grant decisions ended up going to a token house appeal vote.
  • Goal: <1 week turnaround from Grant completion to grants delivery
    • Assessment: Grants that ended up being approved for completion were turned around in less than a week. Some grants that had incomplete or unfulfilled milestones had some back and forth that took longer than one week. Going forward, between grant completion and fund delivery, there was at times, substantial wait times (additional 1 week). We will look to shorten this time in the coming seasons.

2. Impact assessment - how well did your team perform in relation to success metrics, impact per OP allocated, or other pre-defined measures that are relevant for your Council or Board?

  • Metrics: Helping aid increase in TVL on OP and Superchain in line with Grants Council: We were able to successfully coordinate with grants council throughout the season and assist in their grant selection process. No delays or hindrances to the grants council’s decisions and workflow arose from our actions. This season, we ended up managing a 10m OP multisig to execute S7 grant deliveries. After successful onboarding and opsec training, the weekly cadence of transactions had little issues.
  • Other Goals: Projects to be clawed back due to misunderstood timelines <5%
    • We communicated deadlines to teams months in advance and there were no issues here. It’s worth noting that there were some issues with timelines for when teams would actually receive funds post 1 year lockup. In the future, we can be more proactive to ensure fund delivery timelines on our end are clearer and more consistent.

3. What are the major problems you ran into over the course of the Season?

As this was the third iteration, everything was pretty smooth and there were no major issues. Being that as an independent council, this was the first iteration, figuring out the different roles and increased team size had a slight learning curve. After the start of the season, one member switched out with another due to personal reasons, and the offramping and onramping went smoothly as well.

4. What are possible solutions that could be explored next Season?

We can be more deliberate in emphasizing the expectations and workload for the council at the start of the season. A reserve operational budget and transition process was in place, so the transition of reviewers went very smoothly.

5. What improvements to the team’s mandate would you suggest for next Season?

Adding in upkeep of a grants claiming tool is something we took on, and should be made deliberately next Season. Adding a live metrics dashboard and using data from teams like OSO could also enhance grantees’ experiences.

4 Likes
2

Thanks for putting this together @Juanbug_PGov!

On the first goal of “<5% of quality applicant drop off due to milestones issues”: How many grants were voluntarily forfeited by projects? My understanding is that sometimes projects decide to forfeit their grant as they know they haven’t met, or will not meet, their milestones - are these numbers reflected here?

It’s really exciting that the M&M Council was able to safely manage 10M OP in the first community-managed multisig!

In terms of helping the Grants Council accomplish their goals, I think it could be useful to collect an NPS score from the Grants Council and Developer Advisory Board next Season.

1 Like
3

Hey @lavande, thanks for the message. In the past, team’s have dropped off due to disagreements between milestones which prevented them from getting approved for a grant by the Grants Council. This season, things ended up being significantly clearer on this front, as the only milestone was TVL based, and critical to the nature of the grant being approved on the first place, hence, there were no “last minute” issues with milestones that caused this drop off.

Therefore, the goal of <5% of quality applicant drop off due to milestones issues we believe was reached.

For projects that voluntarily forfeited grants due to not meeting milestones after the applicants are approved, those numbers are NOT here. The reason why this was not a goal this season was that this metric is very closely correlated with token price, something we can’t predict. As token price falls or gains, teams will give more or less priority to specific grants, and it makes sense that the forfeiture rate follows as well.

This data, or in other words, applicants that did receive grants and were approved, but later did NOT finish all milestones:

  • Season 5: Cycle 19 and Cycle 22 from S5 had their 1 year deadline during this Season 7. Of these projects, 31/149 Builders/Growth projects forfeited at least some funds due to not completing their milestones.
  • Season 6: Has not been finalized yet, and will be finalized in Season 8.
  • Season 7: No projects have dropped off so far due to not hitting milestones, and a few have fully completed all goals and have been approved for secondary funding tranches.

For future seasons, it might make sense to include a forward looking “accepted proposal completion” rate, and yes, as you mentioned, a NPS score to help the Grants Council.

Thanks for the comment and feedback!

1 Like