Curia Forum Score Integration: Measuring Delegate Engagement Beyond Voting

[v3naru_Curia]

Curia Forum Score Integration: Measuring Delegate Engagement Beyond Voting

We’re excited to announce that CuriaLab is integrating Forum Score into the OP Governance Analytics Dashboard. Forum Score is a feature designed to quantify and showcase a delegate’s engagement within the OP collective governance discussions. By measuring contributions beyond traditional on-chain voting, it offers a holistic view of a delegate’s contribution on governance.

What is Forum Score

Forum Score is designed to provide a quantifiable measure of a delegate’s activity and engagement in DAO governance discussions. It captures off-chain contributions, such as forum discussions and comments, ensuring that constructive participation is recognized. By aggregating diverse metrics, it offers a more comprehensive evaluation of delegate engagement and contributions.

Why Did We Build It?

Governance is more than just voting; it is shaped by discussions, proposals, and deliberations. However, off-chain interactions often go unnoticed. Forum Score surfaces this data, offering DAOs a more complete assessment of delegate participation beyond voting records. By recognizing diverse forms of engagement, it enhances transparency and provides a clearer picture of each delegate’s contribution on governance.

---

How is Forum Score Calculated?

Forum Score aggregates several key metrics from governance activities. The calculation involves three main components, with all data points first converted into percentile scores for fair comparison across users.

Note: We only include data from users who have visited the forum for at least 10 days to ensure a stable, representative sample of active participants.

A. Proposal Score

This score measures the contributions of a user in creating and discussing proposals for the DAO in the forum.

Metrics Included:

• Proposal Initiations: The amount of proposals a user has created.
• Proposal Discussions: The amount of posts a user has created under a proposal thread.
• Proposal Likes Received: The average amount of likes a user received from their proposals and posts within a proposal thread.

Relevant Proposal Categories:

• #58 Technical Proposal: Protocol Upgrade
• #59 Technical Proposal: Reflection Period Proposal
• #46 Retro Funding Mission
• #69 Governance Fund Missions
• #82 Feedback

Formula Component:

Proposal Score = (Prop_Initiated * 0.5) + (Prop_Discussed * 0.3) + (Prop_Like_Rec * 0.1)

B. Engagement Score

This score measures the contributions of a user to the overall forum.

Metrics Included:

• User Topic Count: The amount of topics a user has created.
• User Post Count: The amount of posts a user has created.
• User Likes Received: The amount of likes a user received from all their posts and topics.

Formula Component:

Engagement Score = (User_Topic_Int * 0.7) + (User_Post_Count * 0.4) + (User_Like_Rec * 0.2)

C. Activeness Score

This score measures how active a user is in keeping up with the forum.

Metrics Included:

• User Days Visited: The number of days a user has visited the forum.
• User Time Read: The amount of time a user has spent reading a topic or post.

Formula Component:

Activeness Score = (User Day Visited * 0.07) + (User Time Read * 0.06)

D. Normalization and Final Calculation

Normalization Factors

All data points are first converted into percentile scores, ensuring fair comparisons across users. These percentile values are then weighted according to their importance.

• Max Score
• Sum of Weights × Max Score

Overall Formula:

Forum Score = ((Max_Score * 1) + (Proposal Score * 1) + (Engagement Score * 0.5) + (Activeness Score * 0.5)) / (Sum of Weights * Max_Score)

---

How to Link Your Forum Score

Delegates can integrate their Forum Score with their governance profiles to provide a more complete view of their participation. The process is as follows:

1. Connect Your Wallet: Visit optimism.curiahub.xyz and connect your delegate wallet.
2. Link Your Forum Account: Enter your forum username.
3. Verify Your Identity: Sign a verification message to confirm your forum handle.
4. Post the Generated Signature on this thread:
   • Copy: Copy the generated signature text.
   • Post: Navigate to the forum and paste the signature text.
   • Verify: Complete the verification process.

---

We need your Feedback!

We want to ensure Forum Score accurately reflects meaningful participation in governance. Your insights will help refine the methodology and improve its effectiveness.

We’re eager to hear your input on how we calculate Forum Score:

• Metrics & Weights: Do the current metrics and their assigned weights (Proposal, Engagement, and Activeness Scores) reflect meaningful contributions?
• Forum Categories: Should we revise or expand the selected proposal-related categories?
• User Experience: How can we enhance the process of linking forum accounts to governance profiles?

Your feedback is essential to fine-tuning Forum Score and making governance participation more transparent and rewarding. Let us know your thoughts and contribute to building a more transparent and stronger governance ecosystem!

[GFXlabs]

@v3naru_Curia

We’re eager to hear your input on how we calculate Forum Score:

• Metrics & Weights: Do the current metrics and their assigned weights (Proposal, Engagement, and Activeness Scores) reflect meaningful contributions?

One of the learnings from similar experiments is that you will get what you measure for. It would be good to already be thinking about how to incentivize more quality of discussion as well as quantity.

This is really hard, unfortunately! So a good first step would be to at least provide less incentive to crank out a large number of low-effort, low-impact comments or farming likes. Perhaps consider either a hard cap on the user post count and user likes received, or some kind of decreasing returns as they accumulate more.

[0xDonPepe]

Not related to forum activity, but it would be nice if attendance at governance calls were also included in the delegate score. @alexsotodigital is already doing a good job posting recaps of the calls. Adding a section with attendees wouldn’t be much of a hassle, and then we could pull the data from there into Curia.

[alexsotodigital]

Hi there!

First of all @v3naru_Curia, I celebrate this initiative as I see a lot of potential in “ensuring that constructive participation is recognized”.

@v3naru_Curia

Governance is more than just voting; it is shaped by discussions, proposals, and deliberations. However, off-chain interactions often go unnoticed.

I broadly agree with this statement. And I think it reveals a more fundamental question:

what quantitative metric helps us to best measure the value provided, even if it is qualitative?

Speaking of Metrics & Weights, I have a couple of triggering questions:

• I see that more weight is given to the ’ Proposal Initiations’ than to the Proposal Discussions of a previously created one. I assume that this is because creating a proposal is difficult (if you are looking for a certain viability of being approved) but… won’t this encourage fragmentation in the forum (everyone trying to be the initiator)? What if it were required that the proposal has been accepted for it to be counted?
• I see you are starting to use the forum to verify users (with the ’ Public Forum to Wallet Link Verification’), could this not be misinterpreted in the Forum Score as a hot topic on the forum? I mean, how to differentiate automated and/or technical posts from quality comments?

Anyway, curious to see a first version of this to identify more angles.

May the force be with you.

[shashisenses]

Nicely written. Detailed info. Congratulations

[Pumbi]

Hey @v3naru_Curia! Congrats on this initiative!

I just successfully linked my profile with my forum account. However, since I’m not a delegate, the Curia hub doesn’t seem to fully recognize me as a user—I can’t view my public profile.

Any solutions for this? It would be great if the tool could also measure the participation of contributors without the restriction that they must be delegates.

[v3naru_Curia]

Hi everyone, thanks again for all your valuable feedback. I’ve consolidated some of your comments and our thoughts below:

@GFXlabs

One of the learnings from similar experiments is that you will get what you measure for. It would be good to already be thinking about how to incentivize more quality of discussion as well as quantity.

This is really hard, unfortunately! So a good first step would be to at least provide less incentive to crank out a large number of low-effort, low-impact comments or farming likes. Perhaps consider either a hard cap on the user post count and user likes received, or some kind of decreasing returns as they accumulate more.

This is a great point. We agree that implementing diminishing returns on post count and likes received makes sense—especially if we also factor in who’s giving the likes (for example, filtering out likes from newly created accounts that might be bots or spam). Additionally, we’re exploring ways to assess the quality of engagement using AI, so our focus isn’t solely on quantity but on the substance of the discussion. Expect further iterations soon.

@0xDonPepe

Not related to forum activity, but it would be nice if attendance at governance calls were also included in the delegate score. @alexsotodigital is already doing a good job posting recaps of the calls. Adding a section with attendees wouldn’t be much of a hassle, and then we could pull the data from there into Curia.

This suggestion sounds interesting—we’re open to exploring it. Could you share the file you’re using to track attendance so we can see how best to integrate this data?

@alexsotodigital

Speaking of Metrics & Weights, I have a couple of triggering questions:

• I see that more weight is given to the ’ Proposal Initiations’ than to the Proposal Discussions of a previously created one. I assume that this is because creating a proposal is difficult (if you are looking for a certain viability of being approved) but… won’t this encourage fragmentation in the forum (everyone trying to be the initiator)? What if it were required that the proposal has been accepted for it to be counted?

We assign more weight to proposal initiations because starting a proposal requires significant effort and thought compared to merely adding to an ongoing discussion. That said, we recognize the potential risk of forum fragmentation. One idea under consideration is to require that a proposal or thread receives input from a set number of verified accounts before it counts toward the initiation score. We’d love to hear your thoughts on this approach.

@alexsotodigital

I see you are starting to use the forum to verify users (with the ’ Public Forum to Wallet Link Verification’), could this not be misinterpreted in the Forum Score as a hot topic on the forum? I mean, how to differentiate automated and/or technical posts from quality comments?

Yes, verification posts will count toward your Forum Score; however, their weight is intentionally kept very low compared to core categories such as:

• #58 Technical Proposal: Protocol Upgrade
• #59 Technical Proposal: Reflection Period Proposal
• #46 Retro Funding Mission
• #69 Governance Fund Missions
• #82 Feedback

We’re also open to feedback on which forum categories the community finds most impactful, so we can adjust weights accordingly.

@Pumbi

I just successfully linked my profile with my forum account. However, since I’m not a delegate, the Curia hub doesn’t seem to fully recognize me as a user—I can’t view my public profile.

Any solutions for this? It would be great if the tool could also measure the participation of contributors without the restriction that they must be delegates.

Currently, Curia Governance Analytics supports the top 1,000 delegates by voting power. However, we’re actively working on an independent leaderboard and broader forum analytics to recognize all contributors. You can check out our Forum Score Leaderboard MVP here.

---

Thanks again for all of your insights and suggestions as they help us refine our approach to making governance participation more comprehensive and transparent. Looking forward to hearing more thoughts and ideas from the community!

[alexsotodigital]

Hi @v3naru_Curia!

I bounced some points wanting to keep the ball rolling.

@v3naru_Curia

Could you share the file you’re using to track attendance so we can see how best to integrate this data?

I clarify that the current recaps of the community calls are not tracking attendance; and I find it difficult to start doing so… especially because the user-name in the Google Meet could be different from the user-name in the forum; also that the attendance to the call (with camera off) does not show how ‘attentive’ they really are to what is happening in it.
Keeping an eye out for anyone who can come up with a way to do this.

@v3naru_Curia

One idea under consideration is to require that a proposal or thread receives input from a set number of verified accounts before it counts toward the initiation score.

It comes to mind; In order for proposals to move to a vote, four of the top 100 delegates must give explicit approval on the discussion thread (with the mythical text ‘I am an Optimism delegate with sufficient voting power and I believe this proposal is ready to move to a vote.’). If so, the proposals are then reflected in the Voting Cycle Roundup.
So the mention in those threads could validate that the proposal was put to a vote (which is an important milestone that could be considered in the Forum Score).

@v3naru_Curia

verification posts will count toward your Forum Score

What I meant was that I anticipate that (if there is a trend towards using the forum as a verification mechanism) it is possible that some bots could rank very high in the Forum Score (for having posts with ‘a lot of interaction’ comments), isn’t it? Although, I also assume this could be corrected with the use of AI, as you mention above. Just to highlight again the ‘quantity’ vs ‘quality’ issue.

---

I guess I’m stating the obvious, but I hope it inspires others with more sophisticated suggestions.
Anyway, happy to keep bouncing ideas off with you all.

[SEEDGov]

We see this as a positive initiative. Any step toward increasing participation and fostering more discussions in the forum is a step forward.

Currently, there is a gap between the votes cast and the feedback provided by delegates in the forum. To put this into data, in our delegate participation report from Season 6, we analyzed the number of feedback entries in the forum for each vote, specifically considering feedback from delegates in the respective proposals. As seen here:

@SEEDGov

Rethinking Delegate Expectations: Best Practices for Delegates and Delegations

Based on the data presented in this paper, there is an opportunity to improve the connection between the delegates who cast votes and those who provide a rationale and feedback for their choices. To put it into numbers, only 14.3% of votes include reasoning, and 8.4% offer feedback per proposal. We want to emphasize that active participation in the forum and the presentation of rationales are key factors that enrich discussions and improve decision-making.

On the other hand:

@v3naru_Curia

We agree that implementing diminishing returns on post count and likes received makes sense—especially if we also factor in who’s giving the likes (for example, filtering out likes from newly created accounts that might be bots or spam). Additionally, we’re exploring ways to assess the quality of engagement using AI, so our focus isn’t solely on quantity but on the substance of the discussion. Expect further iterations soon.

In line with this, quantitative measurement will help recognize those who actively participate in the forum. However, when it comes to assessing the quality of feedback, these types of measurements present challenges. As previously mentioned in this post, there is a risk of incentivizing engagement farming, potentially sidelining meaningful discussions. That’s why we are particularly interested in what parameters you plan to address to mitigate the emergence of bots that artificially boost likes.

@v3naru_Curia

Engagement Score = (User_Topic_Int * 0.7) + (User_Post_Count * 0.4) + (User_Like_Rec * 0.2)

Regarding the Engagement Score, we believe that the number of likes is one of the most easily farmable factors, so its lower weight in the calculation makes sense.

@v3naru_Curia

Currently, Curia Governance Analytics supports the top 1,000 delegates by voting power.

We also think starting with the Top 1,000 is a great way to provide visibility to delegates outside the Top 100 who actively participate, ensuring they receive greater recognition.

[joanbp]

@alexsotodigital

I bounced some points wanting to keep the ball rolling.

@v3naru_Curia

Could you share the file you’re using to track attendance so we can see how best to integrate this data?

I clarify that the current recaps of the community calls are not tracking attendance; and I find it difficult to start doing so…

I do not wish to be tracked. Those who are there will know who were there. That should be enough.

(Taking note of who says what is something else, of course. But people should be allowed to simply be there and listen as they like, or not, without being held accountable for it. Others might choose to watch a recording. To each their own.)

[alexsotodigital]

Hi @joanbp

I am very grateful for your comment, because it makes me integrate this perspective that I understand perfectly. I especially love the different levels of privacy that you imply.

I will take it into account going forward.

But this triggers the question:

How can we recognize and compensate an off-chain contribution from a person who wants to maintain a certain level of privacy?

peer-to-peer attestations?

[joanbp]

Thank you.

To answer your question: I don’t know.

I guess, a place to start is to think about what should be compensated, and what shouldn’t. What does it mean to compensate? What is implied in this word? This question touches on deep topics around intrinsic vs. extrinsic motivation and different levels of self-direction vs. control.

And then, the next step might be to realize that there is so much we can’t adequately measure - especially when we want to respect people’s privacy, but for many other reasons as well. We do NOT want to try to move all of our lives onchain and out into the public. Lots and lots of contributions go unrecognized all the time because they are not public and loud and legible.

Peer-attestations may help in some contexts. But it is a hard problem. We need systems that recognize that most work happens offchain. And should happen offchain.

It is a bit like recognizing, on a personal level, that most of your life is invisible to me. I would do you great wrong if I thought that you are only what I can see. Or if I told you to document everything that is you. Or if I said that only what you show me has value. Right? That would be absurd.

We need to be willing to recognize the worth of things we can’t see. At least not directly.