Hi, I would love to get some ideas and feedback around the topic of reputation systems. Specifically, reputation for decentralized or sybil-prone contexts (crypto).
I’ve been working on building reputation systems that serve a variety of use cases in crypto. I believe we need a public good (protocol) for reputation.
recently published some thoughts on how a reputation layer is needed for permissionless innovation and novel social and consumer experiences
Are there specific reputation related protocols from web2 to model/learn from? Most of the research we did looked at private models and systems (Pagerank) controlled and managed by monopolies.
In crypto, Token Curated Registries ([TCRs])(Token-Curated Registry with Citation Graph - NASA/ADS) and Prediction Markets are good reference primitives, but haven’t been successfully utilized to solve meaningful problems at scale.
This makes me think about how videogame reputation scores work. In multiplayer games, leavers get penalized. If you fail to start a game on Lichess, leave a game halfway through, etc. you’ll be temporarily banned.
“We’re moving towards making social feeds gateways to meaningful, interactive, multiplayer experiences.”
I think there could be enough similarity between literal games and web3 social to find a solution in the former. In your research did you look at in-game reputation protocols?
I really like GitHub’s general approach. Commit events are the proof of work, number of forks give you an idea of the real popularity of the project, and the stars are derived from these real metrics. In general I share Jeff Bezos’ bias against “proxy metrics” made up out of thin air especially when it comes to reputation.
Two Boydian ideas are helpful for thinking about reputation I think. First is “do something > be somebody.” Reputation metrics should measure what route doin not who you are. So perhaps discounted forgetting/fading factors. Not like nobility.
Second is his advice “when your boss asks you for loyalty, gives him integrity, when he asks for integrity, give him loyalty.” In reputation systems we often try to game things by “loyalty to the algorithm.” This leads to degenerate behavior and Goodhart’s Law effects. A good reputation system should reward integrity in behavior, not loyalty to the algorithm. This is a paradoxical requirement. The algorithm cannot reward people trying to do well with it. Not sure how yto pull this off, but no web2 reputation system pulled it off. GitHub comes close though.
On Doing something > Being somebody: yes I agree, and that’s why there needs to be a constant feedback loop that establishes ‘proof of work’. And this work should be time decayed so that one doesn’t sit on accrued reputation which might be practically outdated.
And on loyalty to good behavior vs. algorithm → how to codify good behavior is difficult, unless a community comes to consensus on the definition and is open to iterating/evolving it over time.
The key protocol question we’ve been thinking about is how to decentralized the service layer which can enable reputation compute layer (algo layer) across many utilities (use cases or contexts), without acting as the controlling algo/oracle of reputation. In web2, this service layer became the centralized company itself - Google, Ebay, PayPal, Airbnb, etc.
Specifically, reputation for decentralized or sybil-prone contexts (crypto).