Proposal: Automated DAO Proposal Editing by SimScore.
Build a Discourse Plug-In to convert Forum Discussions into Clear Proposal Revisions with Community-Sourced Justifications.
Rationale
A challenge that DAOs have identified is the need for governance experimentation particularly related to AI assisted governance. The SimScore Solution is a Discourse Plug-in which analyzes Forum Replies with the SimScore API converting Discourse discussions into clear proposal revisions and comprehensive Justifications.
Key Benefits of Automated Proposal Editing with SimScore.
Quality Improvement
By systematically analyzing diverse community perspectives, SimScore helps refine proposals beyond what any individual contributor might achieve. The statistical approach ensures that valuable insights aren’t lost in the noise of numerous comments, while also providing clarity on genuine consensus patterns across different viewpoints.
Throughput Acceleration
Instead of waiting weeks for proposers to manually process dozens or hundreds of comments, SimScore automates the analysis of community feedback. This can reduce proposal refinement cycles from weeks to days or even hours, dramatically increasing governance throughput.
Enhanced Transparency
Each suggested change includes a clear justification linked to specific community reply. This creates an auditable trail showing exactly how community feedback influenced the proposal, reinforcing trust in the governance process.
The SimScore Solution
We (SimScore) propose a system that provides a sophisticated, data-driven approach to analyzing community feedback that operates through multiple integrated components:
1. SimScore / Constrained AI Analysis
Data Collection: The Discourse plugin collects all forum replies and discussions related to a specific proposal.
SimScore API processes this data through three distinct analytical dimensions:
Priority Rankings: Calculates a mathematical “Consensus Point” from all written ideas, then measures each individual response’s similarity to this consensus (expressed as SimScore %)
Conceptual Space Mapping: Positions each response in a two-dimensional space to visualize clusters and relationships between ideas
Pairwise Relationships: Creates a complete similarity matrix showing how strongly each response relates to every other response
Note: The SimScore API is complete and ready for deployment. DescriptionGithub
Validated Consensus Identification: The system requires evidence across multiple dimensions before confirming community consensus:
High-priority feedback
Multiple moderate-priority responses that cluster together
Strong pairwise relationships between responses from different authors
d. Constrained AI Implementation: AI serves as an interpreter of consensus patterns rather than a primary synthesizer. It is strictly constrained by SimScore’s statistical baseline to prevent misrepresentation of community views.
2. Discourse Plugin
Side-by-Side Proposal Editing: The system integrates directly into Discourse’s edit section, showing original and revised proposals with color-coded differences.
Comprehensive Justification: For each revision, the system generates a detailed justification citing specific community feedback that supported the change, including:
Direct quotes from community members
Priority rankings with similarity scores
Relevant pairwise relationships
3. Human Final Control
Human Final Control: The community reviews all suggested revisions and maintains final editorial control before advancing to voting.
Community Demonstration Screens.
Side by Side Proposal Editing - History Screen in Scroll forum
With 30 years of leadership experience, the author served as CEO of an industrial water treatment company that achieved remarkable 100x value growth over 12 years. Under his leadership, the company bootstrapped and self-financed through organic growth before a successful private equity exit. His blank boarding methodology was instrumental in this success, identifying strategic priorities and driving organizational alignment. He is now the founder of SimScore, creating technology to make his proven methodologies accessible to organizations of all sizes.
Thank you so much for submitting this proposal, and for the demo today in the RARI community call, it was impressive. One of the strengths that your proposal solves is the use of AI to consolidate and analyze a current proposal to refine it before on-chain voting. I do have a couple of questions that need some understanding:
The SimScores that are calculated for each proposal. Are they determined/re-adjusted every time a comment is added to the proposal, or a one-time or “once a week” update to provide a revised analysis based upon new feedback? And, are there any considerations either in this proposal or future of what other uses that earned SimScores can be used for, from either a Discourse contributor or for the DAO?
I looked at your provided Notion page (SimScore API - What is it?), and wondered how SimScore can reduce any potential gaming by commenters to “game the scoring”, or is that is even possible with the use of AI (and I’m presuming that as a LLM, it will adapt and gains knowledge the longer it’s used on Discourse)? Can you provide some more details of how SimScore prevents scenarios like this?
Can you give a more detailed breakdown of the integration tasks (broken down by month), since I believe that you might want to consider a milestone approach for payment terms, based upon deliverables. Per your proposal, it looks like the integration would be completed in month 3, and launched for use in month 4.
Also regarding budget, can you provide a breakdown of the budget for this proposal?
And more of a clarification question, this AI analysis used by SimScore will still require the proposal author to manually capture the analysis’s and incorporate into a revised proposal version (and this leads to my question above if the analysis updates from prior and/or consolidates from prior analysis’s from additional discourse comments) to allow for an easier update of their proposal, before they submit it for on-chain voting on Tally?
To ensure that the forum does not get cluttered, we think the number of SimScore revisions posted in the forum should be 1-3 times depending on the complexity of the proposal being considered. We plan to integrate SimScore into the topic edit button within Discourse.
At this time, we have focussed on Auto proposal editing and transparent justifications. So no other considerations such as earned SimScore points have been developed.
I’ll answer the rest of your reply a little later. Waiting for feedback from my team.
Thanks so much, Paul for your responses so far, and for collecting further information from my questions.
I really like how this has been thought through regarding SimScore revisions and auto edit on proposals, as I can see that truly helping proposal authors with revisions (and capturing everything from comments).
From my POV the way we’ve been working over the past year (or moer) actually is very similar to the flow you’re suggesting.
Rari DAO has already been creating mechanisms that allow new working groups to form, take ownership of certain functions, and be accountable for them. Most of these initiatives starting in the forum.
And while I understand how AI tooling like SimScore could help speed up some parts of the process, I’m not sure if it’s something the DAO would benefit from at this point.
Also, on the $8k integration fee: is this same plugin integration going to be offered to other DAOs too? If so, would there be an option for Rari to just go with the usage-based fee instead?
following @coffee-crusher 's suggestion above: a breakdown of the $8k for “Discourse Plug-in Integration with SimScore” can be helpful to understand better the ROI for this proposal.
Yes if a user, wrote the same or similar opinion over and over again it would influence the SimScore analysis. Those responses would more highly prioritized.
We have not worked directly on this problem yet. However there are several ways to counteract this.
Checking whether a particular users responses are duplicates or quasi duplicate. Then alerting system to this for manual review.
Token confirmation, to ensure that user is a delegate. Presuming the delegate incentives could be effected if gaming system was discovered.
We could specify these into the Discourse plug-in.
Month 1: Setup, Core Integration & Agentic Workflow Development
Milestone 1: Kickoff, Planning, and Initial Integration
Tasks:
Week 1:
Project kickoff, finalizing requirements.
Setup repositories, project structure, and initial setup.
Week 2-3:
Integrate the SimScore API with Discourse (data collection, parsing replies into paragraphs, priority ranking, conceptual space mapping, and pairwise relationships).
Develop the Discourse plugin interface and start with the “Analyze with SimScore” button (visible to OP and trusted roles).
Set up initial Agentic Flow architecture, focusing on Evaluation Agent and Diff Agent.
Deliverables:
Project plan and technical specifications finalized.
The current specification, leans towards having a minimal number of SimScore Revisions per proposal, so that the output doesn t clutter the forum.
Whan a proposer, chooses the edit button on the topic, they will have an option to run the SimScore Revision before they make their own revisions. Should they wish to, the proposer could then create their own revision with manual edits as they see fit.
The SimScore justificaion would be a reply in the forum.
The long term vision for Simscore is to create modular products that solve the issues related to proposal quality, decision throughput and effort hours.
The blend of modules we envision include, forever questions, unbiased synthesis of community opinions, change management approaches and Ephemeral Small Group work.
The Discourse / SimScore API integration project we are proposing here involves the unbiased synthesis of community opinions. The need for this type of integration depends mostly on scale. When there is a small group of 3-6 people the group dynamics can often lead to good results that avoid groupthink, anchoring and social / power influence.
This proposal is designed for those instances where there is scale due the number of replies and a large diversity of users. The SimScore approach will ensure that the community’s feedback is incorporated in an unbiased, auditable and deterministic way. As an example when there are only 20 ideas to consider, there are only 190 pairs of ideas….when there are 200 ideas to consider there are 19,900 pairs of ideas. This geometric growth is the reason that delegate effort (time) can become overwhelming. SimScore’s purpose is to reduce that overwhelm while also improving the decisions and throughput.
I hope this response meets the needs of your concerns.
While automating parts of the governance process is an interesting idea, we think this tool is better suited for bigger DAOs, where the workload on delegates and the number of proposals and feedback is higher. Currently we don’t think having this service would really impact the DAO’s day-to-day activities to justify the costs.
I do not believe our DAO is in need for a tool like this. Overall costs are quite high for just a discord bot, and I personally would not benefit in using it. While most delegates are not using discord that often, I do not see why they couldn’t ask Grok or Chatgpt to generate simliar results, which are basicaly free to use.
Looking to the amount of proposals we have put up in a years time, which would be roughly 20, that would mean we would be paying $500 per proposal just to have a discord bot improve and edit our proposals.
I have to agree with @forexus comment regarding the cost per proposal - $600 is too high a cost for nice to have some additional benefits. Currently the pricing model for RARI is greater than the ROI for governance improvements, and it’s not something that I can support.
Thanks for your objections. I’ll try to make some points here to eleborate and possibly change opinions.
RARI DAO is expanding the delegate pool and will experience increases in replies / rationales. This is the goal of the Delegate Incentive program. To date the number of delegates interacting per proposal has ranged from 10-23.
However the current number of delegates with over 2,000K RARI is 38. This means RARI can expect more inputs from delegates scattered through the Discourse forum and other places in excess of 30 or 40 replies soon.
Research consistently shows human decision quality degrades when processing more than 20-30 complex information inputs simultaneously (Hwang & Lin, 1999; Streufert et al. 1967).
By this view, RARI DAO is right on the cusp of a new paradigm. Based on research., personal experiences and the current state of large DAOs that move inextricably towards centralization: The question is would it make sense for RARI DAO to be pro-active and avoid the proven pitfalls that can be expected as RARI moves to the next level of scale.
Gen AI could give similar results for free. @forexus
This statement is not born out by evidence.
Gen AI may overrepresent some perspectives, under represent others or simply hallucinate. Also it does not meet a DAO fundamental value of transparency. Gen AI is a black box. There is no audit trail or repeatability.
SimScore provides a consistent statistical baseline that doesn’t shift. It is transparent, auditable and repeatable. This constrains the variability of an AI agent, as an interpreter of consensus patterns rather than a primary synthesizer.
The Discourse integration we have proposed in not a Discord bot. A complete spec has been provided above clearly shows that.
In response to the quote above, The integration costs make up $400.00 / proposal. And are 1 time cost. The ongoing cost of $ 120.00 / proposal. Relative to the cost of the Delegate incentive program $6,000 / proposal it would increase cost by 2%.
That being said, the proposal still needs to add value.
Based on the delegate incentive proposal and an in-depth chat with @jarisjames, there appears to be an opportunity. For each proposal, delegates must post a written rationale. This makes up a third of the incentive payments. However, no unbiased analysis is done with these rationales to attempt to improve the proposal quality. Being that the delegate effort is already made and is paid for, it seems to us that a 2% premium for such an analysis would be warranted.
Regarding the Discourse integration, following an in depth convo with @jarisjames, it is clear that the work he is doing for the Delegate Incentive DashBoard is aligned and duplicative. For this reason, I would like to work with him and any others to prepare a new specification to eliminate this duplication. This could lower the integration costs significantly and possibly to $0.00.
We do not pay $6000 per proposal, this is a misrepresentation of the delegate incentive program.
I am in principal against high spend on further governance items. The delegate program is new and we must see how it performs as it progresses. It will be adjusted as we observe delegate behaviour under this program. I see no significant value from hurriedly adding on further complexity and cost at this stage.
We discussed this feature at proposal stage and I’m curious if you already have an integration with the daospace tooling or other similar dao tooling which Jaris or Sixty have built?
Just to clarify, I’m not affiliated with SimScore or this proposal.
The Delegate Incentive Dashboard I’m building is a separate initiative under daospace, focused on summarizing forum rationales and tracking delegate performance.
While I had a conversation with @maets23, I haven’t committed to any collaboration, and the tools I’m building serve different goals than what’s proposed here.
No, I have not had any interation with @jarisjames or @sixty regarding integrations. My only interaction with RARI Foundation or the DAO so far is a 30 min call with @JanaBe, which led to a 5 minute presentation during the Tuesday Governance call and an hour long conversation with @jarisjames.
