[DRAFT] Improving Governance Accessibility through Community Participation Analytics and Hivemind bot for member support

[danielo]

Mission Proposal Example (for illustrative purposes only)

S4 Intent: Governance Accessibility (Intent 4)

Proposed Mission: Accessibility analytics and community knowledge hivemind bot

[Proposal Tier](Collective Trust Tiers - #2 43): Ember

Please verify that you meet the qualifications for your Tier: I am a new community member that has not worked with or for the Optimism Collective before

Baseline grant amount: 37,600 OP

% of total available Intent budget: 1.766%

Alliance: TogetherCrew (powered by RnDAO)

Alliance Lead: MrJackalop (Danielo)

Contact info: @mrjckalop (telegram and discord)

L2 recipient address: 0x75ab74D82Ddc235F74680ddf42701DdFA8bf5041

Please list the members of your Alliance and link to any previous work:

This alliance has received grants and completed work for Aave, Aragon, Celo, MetaCartel and more. The following proposal builds on top of this previous work.

This Mission is led by the following alliance members:

• Danielo (contact person) - Instigator at RnDAO and CoLead at TogetherCrew. Previously, Head of Governance at Aragon, 8 years experience in Organization Design consulting (clients include Google, BCG, Daymler, The UN, and multiple startups), and visiting lecturer at Oxford
  University.
  Twitter: https://twitter.com/_Daniel_Ospina
  LinkedIn: https://www.linkedin.com/in/conductal/
• Katerina - Co-lead TogetherCrew. Ph.D. using social network analysis. Since 2016 she is co-instructing a graduate course on data analytics for HR at Northwestern University. She has also co-organized the Learning in Networks sessions at the International Conference of Social Network Analysis (2018 - 2020), and previously advised a people analytics company on social network metrics.
  Twitter: twitter.com/katerinabohlec
  Linkedin: linkedin.com/in/katerinab
  Github: katerinabc · GitHub
• Tjitse van der Molen - Contributor at TogetherCrew. MSc degree in systems biology, focussed on large scale biological data analysis. Currently, he is obtaining his PhD in dynamical neuroscience where he is using graph theory and advanced statistical methods to study how neurons self organize into interaction networks. He is applying the same techniques to study community member interaction networks.
  recent project: analyses for this paper Functional neuronal circuitry and oscillatory dynamics in human brain organoids | Nature Communications
  Code and data are openly available in this github: GitHub - KosikOrganoid/Intrinsic-activity-code: Example data and code for analyses in paper
  Github: TjitsevdM · GitHub
• Cyrille Derché - Contributor at TogetherCrew. Bsc. Computer Science. Ex-Accenture, Co-founder and CTO of SaaS company helping medical device manufacturers deliver product data + documentation to healthcare professionals (handling of sensitive data). Builder of products, processes, and teams.

Please explain how this Mission will help accomplish the above Intent:
Accessibility is not only about the end result (e.g. how many people vote) but about understanding the conditions and root causes that lead to said result, so we can improve upon them. Accordingly, this proposal will provide Optimism with:

1. Analytics Dashboard: a mechanism to assess Governance Accessibility and a holistic understanding of Optimism’s social network across platforms => informing continuous improvement

2. Hivemind: leverage the same data pipelines for a “hivemind bot” using TogetherCrew’s expertise in ML to enable community members to easily query accumulated knowledge in Discord and Discourse to answer complex questions (e.g. has a proposal for topic X already been funded? how can I submit a proposal? What does the community feel about topic Y?). This removes knowlede-gap barriers to involvement and enable community members to participate more effectively with less time investment.

Analytics:

• Method: we go beyond basic engagement metrics, as those fail to capture which groups are encountering barriers to participate and which groups aren’t. In addition, it is prone to be gamified and mistake spam for valuable participation. Instead, this proposal leverages Organisational Network Mapping with “interactions data” captured from community platforms to more precisely understand the dynamics of governance accessibility across the community channels as well as the evolution over time of participants’ engagement and resulting community health.
• We leverage TogetherCrew’s existing infrastructure: Discord data pipeline and databases. This development was funded by previous grants from Aave, Aragon, Near, Celo, MetaCartel, and Polygon. We will expand upon it with a Discourse integration to include proposal-related conversations.

HiveMind:

• Method: The Hivemind bot enables members to query the community’s collective memory with advanced questions capability. A multi-agent LLM system is configured to summarise conversations and breakdown complex queries into simpler tasks that can then be resolved and aggregated by scanning the summarised community data.
• Equally, regular summaries of discussions are also generated as a valuable by-product.
• The Hivemind bot will leverage the data infrastructure already developed for the Analytics (Discord and Discourse data collection) to provide answers. The interface will be a bot made available in Optimism’s Discord (potential subsequent expansion to other platforms).

This proposal also includes a budget to cover the maintenance costs of the analytics infrastructure for the next three months (after deployment) and a forecasted usage budget for the Hivemind bot’s LLMs infrastructure. Should usage exceed this budget, the service will be discontinued and a subsequent proposal will be made for the community to decide.

For additional context see
4min: Dashboard overview TogetherCrew overview - YouTube
45min: in depth presentation to MakerDAO including Q&A (not part of a grant request): Tool House Rock #10 | TogetherCrew - YouTube

What makes your Alliance well-suited to execute this Mission?

• For the past 12 months the core of our work has been the study of Community Health and the development of frameworks and tools to assess it meaningfully. Governance Accessibility is a key part of that so we have already built significant data infrastructure and done work to separate meaningful metrics from less relevant ones.

• We’ve partnered with researchers at UC Berkley studying governance analytics in particular, so we can separate the wheat from the chaff and ensure not just a lot of indicators but the most relevant ones.

• We’ve years of experience in web3, and decades of experience in analytics, ML, and social sciences. We’ve also built a company handling sensitive medical data, have worked with lawyers from LexDAO to validate compliance of our data management practices, and consulted for dozens of clients from startups for fortune 500s and DAOs.

• This has included the delivery of this public goods framework: Tooling & Utilities nominations for RPGF2 - #29 by davisshaver

Please list a critical milestone. The critical milestone should be a measure of whether you’ve made best efforts to execute what is outlined in this proposal or not. If you fail to achieve your critical milestone, your grant may be clawed back.

• milestone 1: Deployment of TogetherCrew’s bot in Optimism Discord ⇒ dashboard goes live (early insights on community health are generated)
• milestone 2: Integration with Optimism Discourse ⇒ Inclusion of Discourse data on the dashboard (live insights on governance accessibility)
• milestone 3: Deployment of HiveMind bot in Optimism’s Discord ⇒ Community members enabled to query collective memory and troubleshoot accessibility

How should Token House delegates measure progress towards this Mission: These should focus on progress towards completion. Including expected completion dates for each is recommended.

• Successful deployment of discord bot - week 0-3 (dependant on Optimism’s discord admin)
• Analytics dashboard live, providing insights for community managers to improve Community Health including Governance Accessibility - week 0-4 (dependent on completion of the above)
• Successful integration of Discourse data pipeline and dashboard live including multiple data sources - week 6-12 (margin given due to unknowns about the volume of data and consequential infrastructure adaption needs)
• Integration of HiveMind bot with Optimism’s data and configuration - week 1-3
• Development of HiveMind bot Discord interface and deployment in Optimisms’s Discord - week 3-6

How should badgeholders measure impact upon completion of this Mission? These should be focused on performance and may be used by badgeholders to assess your Mission’s impact in the next round of RetroPGF.

• Community managers usage of analytics dashboard
• Number of queries to Hivemind bot per week by Optimism members
• Satisfaction with Hivemind bot outputs (the bot will collect this data too)

Breakdown of Mission budget request:

Costs Breakdown
Item	hours	OP per hour	total
Discourse integration	80	70	5,600
Data schema and comms patterns ethnography	30	70	2,100
Analytics scripts update	50	70	3,500
Integration into dashboard	60	70	4,200
Technical project management	40	90	3,600
UX Design	50	70	3,500
LLM multi-agent system configuration	60	90	5,400
Comms materials/documentation and onboarding for Optimism	20	70	1,400
Discord bot interace	30	70	2,100
Marging of error	20	70	1,400
Infrastructure design, development, deployment	40	70	2,800
Contributors Total			32,800
Infrastructure costs (data)			3,600
Infrastructure costs (AI)			1,200
Expenses Total			4,800
GRAND TOTAL			37,600

I confirm that my grant will be subject to clawback for failure to execute on critical milestones: Yes

I confirm that I have read and understand the grant policies: Yes

I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant: Yes

I understand that I will be expected to following the public grant reporting requirements outlined here: Yes

Attachment

Dashboard images (non-exhaustive)

Advantages of this proposal

• Both analytics AND collective memory query ‘Hivemind’ bot for accessibility. Also more costs effective as reusing the same data pipelines
• More than the sum: the bot can query the analytics as well, enabling it to use both qualitative and quantitative data to answer questions
• Research-backed metrics and recommendations: it’s not enough to have data but the key is knowing where to focus. We’ve already prototyped a recommendation engine (see images above) and this proposal would allow us to expand it further
• Cross-platform analysis (Discord analytics already built and ready to deploy, Twitter analytics funded by Web3 foundation and currently in-development, Discourse and voting analytics to be added via this proposal)
• Added value for moneyBuilding on top of previous proposals for other ecosystems thus able to provide 3x value for money to Optimism (Discord, Discourse, Twitter, and voting analytics + hivemind bot)
• *Reliable: this team is part of RnDAO and has been delivering research and analytics reports for a year in this area. Additionally, our Discord bot has already been greenlighted for safe deployment in the Optimism ecosystem.
• Compliance: our data management practices have been developed in collaboration with IP and GDPR lawyers from LexDAO
• Advance queries: multi-agent LLM model enables the bot to go beyond basic FAQs and instead operate as a “research assistant”, breaking down questions into subtasks and using different methods (combining qualitative data and quantitative analysis from the analytics) to provide better answers.

[vonnie610]

Hi @danielo

I’m Vee from the Optimism Foundation.

Your proposal is great; however, it should really be two proposals, one for the analytics dashboard and one for Hivemind. A few projects are working towards an FAQ bot, so it would be easier to compare them if it was separate. Additionally, grant requests should be denominated in OP (not the USD equivalent of OP, as 1 OP = 1 OP).

Your proposal mentions quite a few platforms and I wanted to better understand what permissions you will need from Discord, Discourse and Twitter/socials.

Personally, I would like to say how excited I am to see RnDAO getting involved in the Optimism ecosystem. I personally think the way y’all do community analytics is incredibly innovative and insightful.

[lavande]

Hi @danielo! Wanted to make sure you were aware of the Optimism Season 4 Pitching Sessions to help find the 4 delegate approvals you’ll need by this Wednesday at 19:00 GMT for your proposal to move to a vote.

These sessions are happening in Discord on Monday, 26.06 2pm ET / 6pm GMT / 8pm CET and Tuesday, 27.06 11am ET / 3pm GMT / 5pm CET.

You can sign-up here!

[danielo]

Thanks Vonnie for the feedback!

I’d argue that it makes more sense to combine them, as building separate infrastructure for them would increase costs by about 50%

Additionally, we already built the infrastructure for Discord data pipelines and Community Analytics and are offering that for free with the proposal. Then we use the OP requested to add the Discourse data pipeline, expand the metrics to include Governance analytics and then can connect the Q&A/Hivemin bot to all of that. On top of reusing the same data pipelines, community members will have the ability to query the analytics from Discord too!
There are UX benefits but, more importantly, our bot is not just a simple LLM model, but actually a multi-agent system that can answer more complex queries (think about it as a research assistant), so the bot can combine both qualitative and quantitative data (thanks to being connected to our analytics). For example, we can ask “who can I ask for feedback on my proposal” then the bot can take into consideration who has been active in Governance or recommend members who are “vital members in our analytics” (members who are well connected across the social network and hence have broader context).

It’s a case where the whole is more than the sum of the parts

As for the permissions, the discord bot needs to be able to read messages and message history and send messages (also likely “use application commands” so users can engage with it). The admin can decide which channels to give access to and which not, so it’s fairly safe, especially compared to other bots out there asking for admin permission. We already did a lot of work figuring out how to narrow our permission management to the bare essentials as we conducted an internal review on security, so Optimism also benefits from this previous work.
As for Discourse, just the ability to read the data (the exact permissions would be figured out through the grant work)

[danielo]

@katie @lefterisjp @polynya @mastermojo @Griff @linda @ceresstation your greenlight to go to a vote would be most appreciated!

[ceresstation]

Personally I think this is a super reasonable grant amount for an idea that could really simplify some of the community support process. I’d leave it to others in the ecosystem who are closer to that function to chime in on whether or not there are other projects that this would conflict with, but @danielo has made enough meaningful contributions to the ecosystem that I’d trust him to work closely with any other ongoing initiatives as well.

I am an Optimism delegate with sufficient voting power , and I believe this proposal is ready to move to a vote.

[spop]

Good to see further work being done in terms of analysing behaviour - I wonder what levels of granularity this could go to? I would be more interested in sentiment if accessible and also degree of engagement (this would apply to community as well as representatives). It’s what we started for Gitcoin last year https://delegate.gitcoin.co/ (sadly now it is much more focused on voting activity thus reductionist in nuance)

[danielo]

Hey Simona
sentiment is indeed part of it. The tool would use sentiment as a basis to calculate which messages then to analyse for conflict. so it’s kind of like a stepping stone and already included (and we’re exploring different ways to visualise it)

And the hivemind bot can also query on-demand said messages and the messages graph too, so all kind of on-demand analyses are possible

Degree of engagement is already built, both as an overall score and also as a network graph which is even more powerful

[Griff]

I am an Optimism delegate with sufficient voting power , and I believe this proposal is ready to move to a vote