Season 8 and 9 Milestone and Metrics Council Selection

Elected Representatives đź’Ľ
,
1

In Season 8, we’re excited to experiment with alternatives to elections for non-political contributor roles.

Different contributors play different roles within the Collective. Elected members serve as representatives, entrusted with governance responsibilities that would otherwise be held directly by tokenholders. Elected representatives often make decisions on behalf of tokenholders and, in the process, are essentially “writing policies” for the Collective. Representatives should be elected as their work is political in nature (e.g., Grants Council and the Developer Advisory Board).

Other contributors do not act as representatives, but rather take actions necessary for the Collective to achieve its goals. However, these are not actions that tokenholders could or would otherwise do themselves. We refer to these contributors as civil servants: a contributor that executes important functions for the Collective. Civil servants’ work is not political in nature, so we will begin to experiment with non-political methods of selecting members for these roles in Season 8.

The members of the Milestones and Metrics Council act as civil servants. The Milestones and Metrics Council executes (enforces) the grant policies and completes the important actions of measuring grant performance to help the Collective achieve its goals. In Season 8, we will experiment with a non-political process to select members of the Milestones and Metrics Council.

Season 8 and 9 Experiment

For Season 8 and 9, we will conduct an initial experiment with M&M to select contributors via a non-elected model. The goal is to begin to test different models by which the Collective can:

Select members to fulfill non-political roles (those that execute functions) based on competencies (expertise and experience) rather than political means (elections)

In this experiment, the Milestones and Metrics Council will not be elected. Instead, a set of selection criteria, as outlined below, will be ratified by the Token House. That selection criteria will be used to determine a list of individuals / addresses that are eligible to be on the Council in Season 8 & 9, since all Councils will now operate on 12 month terms.

Any eligible member (person meeting the ratified selection criteria) may submit an M&M Charter and run to be Lead during Voting Cycle #39a. The Charter will specify the number of members needed in Season 8 & 9. To determine members from the full list of eligible addresses, the Foundation will randomly sample from the list of eligible addresses. Imagine there are 7 eligible addresses, but there are only 3 members as outlined in the approved Charter. The final 3 members will be randomly sampled from the set of 7 eligible addresses. Eligible addresses must opt-in to participate in the sampling.

This process could be thought of as a stratified sampling approach where in a subset of qualified members are selected from the full list of qualified candidates. Sampling is a temporary mechanism to be used only until we collect enough information on attestations to be able to incorporate a rank order into the selection mechanism itself.

:1234: Note on the random sampling - we will use R, a software environment for statistical computing, to compute the randomness. To make sure the random selection is reproducible, we will set the seed which allows the R script to give the same result with this seed each time. To ensure the output isn’t tampered with, we will use the first 3 integers that appear at the Ethereum block hash at height 22575600 as the seed.

Transitioning Councils to non-elected models should have no bearing on the rewards received by members. Leads will continue to submit Council Operating Budget requests as in previous Seasons and the contributions made by civil servants are of equal importance to the Collective as those made by representatives. Importantly, all members will still be accountable to governance via the Representative Removal proposal type outlined in the Operating Manual.

Selection Criteria

To be eligible to be an M&M member in Season 8-9, candidates must meet 2 of the following 3 requirements:

Analytics Skills Reputation in Collective Strong Operational Security
Previously served on the Milestones and Metrics Council Community contributor (above wannabe level) Completed an operational security training provided by Opsek
Completed a relevant analytical Foundation Mission Any governance role in Season 6 or 7 *please note, in the future, other opsec trainings will be eligible

:recycling_symbol: The first iteration will be imperfect. The initial criteria is intentionally restrictive to ensure qualified candidates are selected that can execute on the M&M’s mandate and maintain the highest level of security practices.

Lead Selection:

  • The Lead must satisfy the same criteria as all other members. After the eligibility criteria is ratified by governance, any eligible member may run to be the Lead by putting forward a Charter in Voting Cycle #39a. The author of the Charter selected by governance will become the Lead. Selection of the remaining members, as specified in the Charter, will be completed immediately after Voting Cycle #39 and the final member list will be posted as a comment to this forum post.

:police_car_light: If the selection criteria is not approved by the Token House, an election will be run in a subsequent voting cycle, which will delay the launch of Season 8.

What does this mean for delegates?

  • This constitutes a Reflection Period proposal type, subject to Token House vote at a 30% quorum and a 51% approval threshold.
  • By voting yes on this proposal, you are approving the outlined selection criteria for Season 8 and 9 Milestones and Metric Council members. Only folks who meet this criteria will be eligible to run to be Lead by putting forward a Charter draft in Voting Cycle #39a. You are not voting on whether or not you think we should run this experiment.
14 Likes
2

Hmmm this sounds a bit weird. Randomly sampled by the foundation from the list of “eligible people” sounds like moving even more power to the foundation and away from token holders / delegates.

Why?

Due to lack of an answer here I will be voting against this. I don’t think it’s a good idea to move more power away from the token holders and to the foundation. Optimism is already pretty centralized around the foundation. It should strive to move away from that (if it want to be a DAO).

14 Likes
3

“Civil servants’ work is not political, so we’ll use non-political selection methods…”

Eligibility #1: previously sat on the very political M&M Council
Eligibility #2: held a political governance role in Seasons 6 & 7

So the plan is “no politics, but only if you’ve already done politics."
This is counterintuitive and ends up narrowing the talent pool to a pretty short list.

5 Likes
4

I feel like i’d want to get a few sample outputs of councils before outright voting for this to get a sense of what we might be looking at

5 Likes
5

Like many others here, I want to express my negative view of this initiative. Here’s why I’m against this new proposal:

  1. There is no basic mechanism for delegate influence — no direct voting.
    Sure, that’s how some positions were filled in democratic Athens 2,500 years ago — but only those prone to corruption.
    In this case, there’s no corruption risk, so it’s unclear what we’re trying to achieve, aside from fueling endless debates about fairness.
  2. The participation criteria clearly suggest that it’s meant for the same familiar group of people. Newcomers will have almost no chance to get in.
  3. Extending the term to 12 months only worsens an already unfair selection process — or rather, the lack of any real elections.
6

Thank you for engaging in this discussion!

While we acknowledge the initial set of criteria is limited, this is meant to be a first iteration and we decided to err on the side of ensuring eligible candidates are qualified and can execute on the M&M’s mandate.

This means the initial list of eligible candidates is smaller than we’d expect in the future, when we have a more robust set of attestations and data about which attestations are good signals. While imperfect, this as an important experiment to ensure we can select for non-political roles in the Collective in the future.

This will help alleviate the common concern that elections devolve into a popularity contest. As the criteria is still subject to governance approval, and the random sampling method will be publicly verifiable. This approach does not shift any more power to the Foundation.

Given questions about what a sample output look like, based on the above selection criteria, the following folks are eligible:

  • Jun
  • Mahesh
  • Takeshi
  • Varit
  • Chom
  • Chain_L
  • Lau

The following folks meet the criteria, but are not eligible for the reasons provided below:

  • All current Cohort A and Cohort B Security Council Signers (can’t be on two councils at once)
  • Carl (can’t be on two councils at once)
  • Mel.eth (resigned from current M&M)
5 Likes
7

i’ve generally been pretty fully on board with a lot of stuff that’s been proposed and am typically in favor of experimentation at this stage of the Collective’s development

as such I’m not going to dispute the merits of this proposal in principle but practically speaking i feel like there are few enough eligible people here where voters could easily form opinions about who would be more effective than others, and it’s not obvious to me that there’s a lot of actionable information that would come out of this particular setup

that said the downside of running this experiment on milestones and metrics is fairly limited so yeah if this is intended to trial progressive steps in this direction (and would like to hear more of the long-term thinking and what’s expected to be learned), i can vote for this

1 Like
8

Having a clear list of eligible candidates makes the process much easier to follow. I appreciate that becoming a nominee no longer requires a long forum post. Moving to a criteria-based selection with a simple opt-in is a solid improvement.

That said, after reviewing the list of candidates, I still prefer to vote on nominees rather than leave the final selection to randomness. The pool isn’t large enough to create uncertainty about who’s best suited for the role, though perhaps that’s just my perspective as someone deeply involved in governance.

One open question: If this vote doesn’t pass, does the process revert to the previous system, or will the Foundation iterate based on feedback? I don’t want to block experimentation with governance design, but I do want to ensure the most competent contributors are the ones managing the Milestone & Metrics multisig and tracking grant performance.

3 Likes
9

One open question: If this vote doesn’t pass, does the process revert to the previous system, or will the Foundation iterate based on feedback?

Looks like on the live proposal it says “If the selection criteria is not approved by the Token House, an election will be run in a subsequent voting cycle, which will delay the launch of Season 8.” So maybe more time to figure out.

–

From my POV, I feel like this is a change worth experimenting with. I think the weakest piece of all grant processes is the performance phase, post-award, and that we should be aggressive at the risk of trying to break things to find models that serve the Superchain best – I do not see this as a one-way door and feel like this encourages future iteration based on what happens here.

In this spirit, I’m supportive and will vote yes.

My caveat here is that I would like to see how we can qualify via data how effective folks are that hold these positions, rather than subjectively trying to interpret it. This latter point I think is something all councils (not just this one) could benefit more from.

4 Likes
10

While I support incorporating competencies into the role selection process, I feel the current approach is too opaque. It seems optimized to keep the same group of contributors in charge, rather than fostering an open, merit-based system.

Many of the criteria appear to rely on “has been part of this before,” which isn’t a strong indicator of capability, especially since the initial set of roles was filled through election, not on merit.

I believe a reshuffling of certain roles and an infusion of new talent are crucial for the next phase of the Collective. This is particularly important given that many delegate and operator roles haven’t been refreshed in quite some time.

In practice, I would like to see a process where:

  1. Current and past operators are reviewed (perhaps with input from those they’ve worked with), and
  2. New contributors, such as guest voters and top 100 delegates, are allowed to participate in the M&M Council process after a basic skills validation.

This way, the system truly becomes an open-merit-based process.

11

I am curious to Gonna’s question here about if it reverts to previous system or will foundation create a new proposal based on feedback?

12

If the selection criteria is not approved by the Token House during this current Voting Cycle, an election will be run starting in Voting Cycle #39a alongside the other Councils and Boards, which will delay the launch of Season 8.

The Foundation will iterate on all the feedback received and may put forward an alternative selection process for M&M in Season 10.

1 Like
13

We believe the election process enables a more level playing field for the Council selection process. It is hard to design a very objective and fair selection process, and we have some concerns with the Selection Criteria as is:

Analytics Skills → We agree with this is beneficial for a Milestone and Metrics council, however do not believe the subpoints listed actually serve as examples of this. “Previously served on the Milestones and Metrics Council” does not indicate someone positively contributed to the Council before, and it leads us to not bringing in new people and ideas. Also, “Completed a relevant analytical Foundation Mission” is relatively subjective. In an election allows folks to provide examples of their Analytical skills, rather than using a relatively subjective criteria.

Reputation in Collective → We think the intent of this makes sense, however a better framing might be “Consistent Positive Contributions” to the Collective or something since Reputation is subjective. Again, an election would allow people to provide examples of fulfilling this criteria.

Strong Operational Security → We assume this exists due to the fact the person would manage key materials as part of serving on the Council. If this is indeed the rationale, we believe it should be always required (which is not necessary in the current framing where the person only needs to meet 2 of the 3 requirements).

1 Like
14

The metrics idea sounds good on paper… but in practice, I don’t see it as a step in the right direction I agree with most of the comments above, especially Luuk’s

The weirdest part of this whole thing is that there is clearly an expectation that we should all just vote yes on Foundation proposals. The fail state is rarely described in proposals liek this, and apparently not even planned for since if it fails it will delay Season 8.

I thought we were working towards decentralization, but this whole proposal process feels like a step backwards so I will be voting against.

4 Likes
15

Echoing some thoughts and countering some:

  1. Agree that experimenting with a different process is worthwhile, but

  2. The chosen criteria do not ensure quality

  3. Arguments about “decentralization direction” and “tokenholder power” are incomplete without also addressing the countervailing problems of voter apathy and quality of tokenholder-made decisions (since many tokenholders don’t make voting their full time profession, which means there is often lack of context and in some cases domain expertise). I’m not saying we can resolve this tension here but I wanted to flag lest we lean too heavily on simplistic arguments

2 Likes
16

Yeah I think this is in part where I’m bumping up against major concerns. Incumbency isn’t merit. If you want capable people filling nonpolitical roles, someone needs to assess their capability.

17

ah shoot wish i’d read this. why would running this election in parallel with the others necessarily delay season 8?

1 Like
18

A delay seems unnecessary. It’s not as if a newly seated Milestones & Metrics Council would have anything to do until the first wave of grant recipients.

19

After carefully evaluating the proposal, we are supportive of initiatives moving toward onchain attestations and believe it’s best to minimize popularity based voting.

That said, several elements of the current design need to be addressed before we could fully support the experiment. First, the nominee pool is too small. Random selection from such a limited sample undermines the entropy that lends legitimacy to sortition and also risks delaying the Season 8 and 9 calendars if we must re-roll. As said here:

This methodology does not ensure the strongest candidates end up on the council.

The narrow eligibility criteria may unintentionally exclude highly qualified contributors. In pursuit of objectivity and automation, we may be risking reverting to a more centralized process in which external experts are barred simply because for example they have not previously served on a council. Upon a less critical second look, we also note that pure randomness could select nominees who decline to participate, leading to further re-rolls.

We suggest considering the following aspects to iterate the current proposal:

  • Grant the Council lead greater authority, veto power over the final list, or require each nominee to secure two out of three approvals (from the Foundation, the lead, and a Top 100 delegate). This way, the lead would have greater ownership of organizing the Council’s working group. This is a valid example to explore.
  • We believe it would be valuable to broaden eligibility to attract new talent. As suggested here:
  • Require all nominees to pass an OPSEC check
  • Instead of unweighted random selection, explore a weighted, “mined” sortition model that preserves unpredictability without sacrificing deadlines. As said here:

We support the overall idea of the experiment, but this proposal could benefit from further refinement, address remaining concerns and isn’t worth rushing, so we will vote AGAINST.

2 Likes
20

Thanks for the continued discussion around this proposal, it is good to see the community engaged in an active discussion about an important topic.

A lot of feedback relates to doubts about the specific criteria being robust enough to select for top candidates and/or that it may unfairly advantage incumbents. This is admittedly a limitation of this first iteration as we currently have a limited set of attestations or objective data to leverage and we need to continue to collect data about which attestations/data serve as the strongest indicators of performance. We will continue to build towards a more robust graph of attestations and upcoming elections will further build our understanding of which attestations are useful when evaluating candidates. Having more robust data about this will also allow us to expand selection criteria to rely less on previous participation and move towards more open-merit-based criteria (which was an acknowledged tradeoff in this first iteration.) We would love to see continued discussion about the type of selection criteria delegates would be supportive of and/or find more useful.

We would like to clarify some of the feedback about the use random selection, or “sortition,” as a small component within this proposal. Importantly, this is not a sortition experiment. In fact, we acknowledge that the use of sortition here is a sub-optimal and temporary mechanism to be replaced in future iterations.

Sampling is a temporary mechanism to be used only until we collect enough information on attestations to be able to incorporate a rank order into the selection mechanism itself.

We have experimented with sortition before (see here and here) - in instances where we used a sample to approximate the full population in a way that is robust to manipulation and has individual fairness properties.

The goal within this proposal is simply to (temporarily) use random selection to determine a set of final members from the list of eligible (and opt-ed) in members without relying on the Foundation or a process that mimics an election (application process.) This experiment is meant to test the use of an attestation-based selection mechanism (similar to what we did with the Collective Feedback Commission members), which ultimately will not use random sampling.