Hello Optimism community!
The bleu team is excited to start building a sybil detection algorithm for Farcaster using social graph data. Our goal is to enhance the integrity of Farcaster and the ecosystem around it.
We’ll post bi-weekly updates here as comments. Your feedback is invaluable for us!
For transparency, you can track our progress on our GitHub Repository.
We look forward to your insights as we go!
Hello everyone!
We’ve officially kicked off the Farcaster Social Graph project and have begun working on our first milestone: “Requirements Definition and Data Extraction”. This milestone should be complete in the next few weeks!
| Milestone | Type | Status | ETA | |
|---|---|---|---|---|
 |
Requirements Definition and Data Extraction | Critical | In progress | 2 weeks |
 |
Algorithm Design and Architecture | Critical | Planned | 3 weeks |
 |
Algorithm Development and Evaluation | Critical | Planned | 5 weeks |
 |
Algorithm Integration and Monitoring | Critical | Planned | 2 weeks |
We plan to run an instance of Hubble to get Farcaster links data and work on replicating it into a Postgres database for further analysis. This approach seems to be our best path forward to get the graph and apply the Sybil Detection algorithms.
Before our next update, we aim to complete the following tasks:
Hey folks! We’ve made significant progress on the project since our last update. We’ve completed our first milestone and are well into the next phases of the project.
| Milestone | Type | Status | ETA | |
|---|---|---|---|---|
|
Requirements Definition and Data Extraction | Critical | Wrapping up | 2 weeks |
|
Algorithm Design and Architecture | Critical | In progress | 3 weeks |
|
Algorithm Development and Evaluation | Critical | In progress | 5 weeks |
|
Algorithm Integration and Monitoring | Critical | Planned | 2 weeks |
We’ve pivoted from our initial plan to run a private Hubble instance. Instead, we’re now using the Neynar API, which provides data in Parquet format. This shift allows for more efficient processing and reduces infrastructure costs. Key achievements in data extraction:
After exploring several graph-based Sybil detection algorithms, including SybilRank and SybilBelief, we’ve selected SybilSCAR as our primary method. SybilSCAR offers the best balance of speed, memory allocation, and accuracy for our needs.
We’ve begun applying it to our transformed Farcaster social network data and are developing a Sybil detection API. This API will manage and update datasets while providing an endpoint to check whether a Farcaster user is classified as a Sybil or a benign user.
Hey folks! We’ve been refining the graph-based algorithm, testing, and improving our training dataset. While we’ve been able to accomplish less than we wanted in the last two weeks, we’ll pick up some speed as another teammate is jumping on the project!
| Milestone | Type | Status | ETA | |
|---|---|---|---|---|
|
Requirements Definition and Data Extraction | Critical | Wrapping up | 2 weeks |
|
Algorithm Design and Architecture | Critical | In progress | 3 weeks |
|
Algorithm Development and Evaluation | Critical | In progress | 5 weeks |
|
Algorithm Integration and Monitoring | Critical | Planned | 2 weeks |
We extracted Gitcoin badges from noisy users detected by the graph-based algorithm, enriching our training and test datasets. These stamps provide an additional layer of verification for users who use Warpcast and have registered at least one stamp on Gitcoin.
Since the API must support a large amount of data, it is important to have an algorithm that can be executed asynchronously and to provide fast benign/sybil checks. We’ve built on top of the graph-based SybilScar algorithm to have a smaller memory footprint for the full dataset so that we can get a sybil response for each farcaster user in that dataset with subsecond response times.