[PIG] Community Search Engines: Event & Chat Artifacts as Research Building Block

Community Search Engines - Event & Chat Artifacts as Research Building Block

Team member names

The protocol of in-person conferences, symposium proceeedings, and other classic gatherings are at best stuck in a published PDF or slide deck world. If you’re lucky, event websites have permalinks that remain year over year, and maybe a youtube playlist or two.

And, we have yet to develop a protocol that better supports digital communities, hybrid events, unconferences, and other emerging forms.

We propose a Community Search Engine Protocol for events & digital communities.

Short summary of your improvement idea

Events, Cozy Web digital groups, and other gatherings are all flow, no stock. Every group should have access to easy tools to gather community context.

Part community search engine with high relevance resources from people you have trust with, part serendipitious discovery from webring surfing.

In the age of AI, leaning into a trusted community context can further help to elevate signal from noise.

What is the existing target protocol you are hoping to improve or enhance?

Today’s events, from meetups to traditional full conferences, at best use a static snapshot of an event websitem, slides, and speaker bios. If the event has enough funding ($1000s of dollars), there are high quality video recordings, and maybe some attempt is made at transcripts.

For Open Space Technology / unconference techniques, images of sticky notes and personal write ups, and perhaps professional facilitation summaries or live graphic facilitation could be part of the artifacts.

What is the core idea or insight about potential improvement you want to pursue?

Digital gardens with valuable backlinks are stuck in single-player mode or commercial platforms. Chat communities, especially those on commercial platforms, are constantly losing context, or their entire archives. An easy toolkit for communities to gather context, resources, and new insights that can grow over time. Community Search in a box.

What is your discovery methodology for investigating the current state of the target protocol?

We’ve run events and communities of this form for years. We will look to find other pains and problems by interviewing other community leaders and recruiting early testers.

In what form will you prototype your improvement idea?

We will produce a toolchain of code and configuration as well as test installs of a community search engine.

Some of the building blocks we will look to use as starting points:

  • Lieu an existing open source community search engine
  • The Noosphere tool-for-thought protocol
  • Social feeds of community participants from Bluesky, ActivityPub, and Farcaster as a source of new resources
  • RSS feeds that are aggregated and synced p2p as local resources for each person using IPFS – both as distributed archive and way to include ongoing content over time

How will you field-test your improvement idea?

Causal Islands Toronto 2023: Causal Islands as a traditional centrally organized multi-day conference. The current archive, including high quality. It has spawned a Discord and Podcast and desire for more “Causal Islands” style events, including one “community edition” in LA. Participants have said they’d like to see more of these events, as well as better find a home for computational research / future of computing topics embodied by the original conference.

Can we turn a conference into a community, and facilitate a template for future events?

We will start by using the 2023 event outputs, including inviting attendees of that event to add their personal resources, in creating a community search engine.

The Discord formerly known as Fission: Fission the company is shutting down, but the community Discord continues. Chat is notoriously bad at capturing long term context, and Discord as a commercial platform is not a good long term home.

We will explore extracting links and people into a community search engine.

This approach should be expandable to other Discord-based communities.

This approach may also be interesting for the SoP community itself.

We will look to recruit other communities to try out the process, including beta testing if technical people can effectively configure and deploy the tooling on their own, as well as differing approaches to activating the communities themselves.

Who will be able to judge the quality of your output? Ideally name a few suitable judges.

  • Gordon Brander
  • Ronen Tamari

All of these might also be collaborators. Ideally communities and people who try and install / deploy this will be those who can help judge if it’s helpful

How will you publish and evangelize your improvement idea?

We will publish a repo containing the community search engine, configuration, and deployment guidelines to get up and running.

We will create an onboarding and community activation checklist to help communities be successful.

We will find seed communities to deploy into, and host introductions and kick off events both virtually and in person.

What is the success vision for your idea?

Communities of interest that form around events, digital places like chat groups, and other gatherings have an easy way to deploy a community search engine that provides both archival as well as ongoing high context trusted resources over time.


Love this!

Boris has been a really important community organizer in Vancouver. (And is the reason I heard about SoP)


love this, we desperately need better ways to make knowledge sharing more multi-player. I’m seeing the meta-protocol for this as one tackling search and discovery for the purpose of community building. Would be really cool to try this out at the edge city events in addition to the general boom in distributed, IRL gatherings.

1 Like

Please DO let me know if you run an event or community and would be up for an interview.

Here’s DevCon:


1 Like

lol I co-organized an un-conference and a technical specification corpus with Boris and I feel a tiny bit like I’m being sub-tweeted here. I’ve also run various different virtual-only uncons over the years and I can’t endorse this project enough. We need this so hard and no one knows the pain better than Boris.


Haha. It’s me too! It’s all of us! This is hard! It’s lots of work! We need toolkits and need to interconnect islands!

Spinning up a place that grabs participant context and something that isn’t a “resources” page filled with stale links.

And absolutely acknowledging that “good stuff” ends up on socials! Subtweets are good, actually!


@kneelingbus’s post is very relevant to this work:

I realize we’re proposing something that potentially further degrades what we’ve thought of as the open Internet aka “enclave-gated internet” from the article. (Read the whole thing — I’m still digesting the many excellent phrases)

I realize that these community search engines are creating more cozy web spaces. Also important background reading The Dark Forest and the Cozy Web — Boris Mann's Homepage

I’ve made a page in my notes to capture more research and reading


Some recent research on Discord for developers


I don’t really think of what I’m proposing as being focused on Q&A. It’s the community discussion itself that is interesting.

So perhaps making clear that this is communities of interest. People will ask questions (and there’s meta around community norms / resources itself), but I’m thinking of spaces where discussions and info is shared.

Hmmm. There’s probably been work done on classifying community types?

Website crawl that turns sites into “LLM ready Markdown” GitHub - mendableai/firecrawl: 🔥 Turn entire websites into LLM-ready markdown

There is lots of formats for search indexes, this is probably useful for a number of things including safe markup for transclusion.

(post deleted by author)

1 Like

Amazing write up of your background! I’d love to interview you regardless.

I’m thinking a lot about additional community feedback loops, online and in person.

I think we’re at a time when there’s a stronger understanding that “just put it online and you can use Google to find your own stuff” is not viable any more. And so there are opportunities for more.

Thank you again for the detail here!

Discord Archive tooling shared with me https://chromewebstore.google.com/detail/discrub/plhdclenpaecffbcefjmpkkbdpkmhhbj

I realize that the fission discord server is an amazing resource…
I learn something everytime I browse a topic
Considering to use this extension to fully rip it and try to publish some link lists and summaries via ipfs some how

Thanks Joshua! This is exactly why I think this is a useful protocol to work on.

I found a 2021 talk that I hosted about a personal setup for local data — including classic QS body metric:

Video here:

I wonder how much QS personal tendencies can transfer to pooled data for a community?

(post deleted by author)

1 Like

haha. Well, this was a small series of tech talks I hosted at Fission that was more along the lines of “own your data” and “local first” than QS primary. There are lots of islands that don’t talk!

Also, I was the host, Dima / Karlicoss was the presenter. So “you” as you’ve listed in here is probably Dima in London, UK.

Yes, I’m thinking much more about shared knowledge substrates. Own your data, but pool it too.

I’m not the best person for this, but the stuff that you’re talking about fits into future of computing / local first / content addressable data for sure :slight_smile:

I’m happy to talk more “offline” and can see what Karlicoss is up to these days / connect you to them.

(post deleted by author)

1 Like

I’m delighted to see so many people and projects focusing on “communities.” Perhaps you’d like to learn more about PIG: Improved social evaluation protocol for a more diverse and inclusive future - we too are interested in the value of content within community settings, exploring ways to record that content and its context. We’ve already built a demo prototype for small-scale experimentation. But we aim higher - to develop a universal protocol for recording content across various scenarios. Perhaps we could collaborate and explore further together?

We believe there are rich opportunities in this space, and would welcome joining forces with fellow travelers. If this piques your interest, I’m keen to discuss potential collaboration. Please let me know your thoughts.

1 Like

I think the angle I’m taking is going to be focusing on search / recommendations / discovery on top of indexed content. And especially links to who sources what.

Absolutely would love to connect and experiment with what others are doing.

eg a “content index” or “search index” should be something that can be specified and be interoperable.

1 Like