PIG: RSS All The Things

SoP 2024 RFC
1

Title:
RSS All The Things

Team Member Names:

  • E. Ramona Waldman
  • Steve Waldman

Short Summary of Idea:
RSS is an enormously flexible protocol whose potential was occluded when proprietary social media drew internet creativity onto platforms.

Now that the limitations and predations of proprietary platforms have become clear, we think the time is ripe to explore how RSS might be extended to organize public, network-scale applications that might occlude the proprietary platforms.

Response to Questions:
The protocol we wish to improve is building public, network-scale applications around RSS.

RSS is an existing standard, and, at version 2.0.x, is in its way complete. We do not propose to modify that specification.

RSS is designed to be extended, however. The RSS specification served as the core of a set of conventions that made possible one of the great decentralized internet applications, the commonwealth of letters known as the blogosphere. More recently it has been extended — in just the way it was designed to be extended — to support a new set of conventions and a new decentralized, network-scale application: podcasting.

Our insight, or conjecture, is that RSS represents a solid foundation for a wide variety of such applications, one that invites the emergence of new protocols because it is simple to work with, and permits diverse users to pick and choose what and how they wish to support extensions to the simple foundation at its core. We think, had proprietary platforms not drawn users and developers away, better versions of the applications those platforms now provide would have emerged on top of RSS. We seek simply to jumpstart a resumption of that process.

Our methodology will be simply construction in public, of extensions to RSS under the terms the RSS specification foresees, and prototype applications demonstrating and supporting those extensions.

We hope to produce:

  1. A set of draft extensions to RSS, some as very general enrichments, some for applications including blogging and microblogging (reorganized and enhanced), online reviews, and some (not all) aspects of social media.
  2. A prototype server for dynamic, RSS-first applications. (RSS-first meaning RSS is the canonical expression and logical storage format, rather than metadata accompanying and describing some other application.)

To field test our work, we mean to first “dogfood” it (one of us runs a menagerie of blogs he would replace), and hopefully leverage our collaboration with Summer Of Protocols to inspire interest in standing up fledging, interoperating RSS services as forums for hosting their own creative work — and the work of commenters and collaborators. We would publish text (descriptions of the projects, extensions to RSS), and an open-source reference implementation.

We’d hope nearly anyone familiar with conversation and creativity online could judge the quality of the output. Much of what we are up to is pushing forward the vision of RSS pioneer Dave Winer, who for example is thinking about applications like textcasting and artcasting. If he is willing (we have not been in touch), he would be a great judge and collaborator.

The success vision is emergence of a variety of new, nonproprietary, applications controlled by creators rather than infrastructure providers, replacing existing platform-bound applications, and doing things we’ve not yet imagined.

Other notes:
A perhaps fuller description than this is available here, where we have begun to organize and brainstorm.

Disclosure:
We are husband and wife proposing to collaborate.

13 Likes
2

The attestation/identity aspect of this sounds particularly interesting to me - I can see how extending RSS might be better than trying to embed such signatures in HTML. This opens up some interesting zero-knowlege cryptography use-cases, as well as being an interesting primitive for citation graphs. Proving that aggregated, syndicated, or archived content hasn’t been tampered with would also be possible using the same mechanism. This feels like a more pressing concern than it did twenty years ago!

3

Another potential collaborator/reviewer:

1 Like
4

Yes! It’s becoming pretty obvious, I think, that in the not too distant future text (or creative work of any kind) without reliable attribution will become some mix of meaningless and dangerous. HTML is perhaps the wrong level of thing to try to sign — should UI elements that will likely change and evolve around core content also be signed? Should less-UI dependent renderings of the same core work lose the attestation?

HTML, XML, and JSON all suffer from the problem that certain aspects of the formats are supposed to be whitespace invariant, while in other aspects whitespace can be significant, so some care in canonicalization is necessary. That grows more complicated under formats extensible with new elements like RSS.

XML does offer a solution, but its flexibility and complexity arguably are in tension with Dave Winer’s (imho wonderful) ethos that emphasizes simplicity and parsimony and extensibility over technical completeness and perfection, that fundamentally seeks to come up with as few primitives as do the job (but no fewer) and entice developers on both sides of the spec to just hack the thing together because it’s fun.

RSS builds on XML, which as a project is a bit ironically a whole retrofuture whose philosophy of specification was the other one. XML put experts in committees to design careful, complete specifications (XPath, XSD, signatures, etc) with learning curves. It sought enterprise adoption before developer adoption rather than the other way around.

XML tools exist, and remain widely implemented and supported, thanks especially to certain enterprise applications. But developers broadly still don’t love them. Alternatively, every XML document has a well-defined encoding (in practice almost always UTF-8, though of course it’s complicated). Any XML element can have an id attribute. So a quick and dirty kind of attestation would be to restrict signatures to one-element at a time, and restrict those to elements containing only whitespace plus a single CDATA node, which when paired with the encoding, defines an unambiguous byte string. So we could define elements like <myprefix:signature target="$id"></myprefix:signature> (with optional attributes to specify hash and signature algorithms, but a decent default if unspecified) very easily. It becomes a bit more work, though, to sign the binding of title, author, and content. With XML signature, an author could sign a full item, or the full document.

Tradeoffs!

1 Like
5

I love this piece, and we would love to work with Chris Krycho.

The idea of units of content as sprouts, rather than finished works spat out fully formed in sequence, is simply more accurate. It does suck that (<atom:updated /> notwithstanding), few tools treat items (or Atom entries) as sprouts. Common tools permit no way of specifying more meaningful updates on the creation side, and feed readers (in my limited experience!) don’t resurface whatever rare items might be so annotated.

I’m a bit less pessimistic than Chris, I think, about the possibility of RSS to accommodate gardens, even though it certainly was not built for them. RSS explicitly permits namespaced elements, like Atom elements, to be embedded as extensions. Following the Dave Winer philosophy of keeping constructs minimal to entice everybody in on the same one, I’d suggest we first encourage creation tools to enable and readers to support <atom:updated />. Many RSS feeds embed Atom elements already. We ought to evangelize the idea that in a typical, most-recent-X-posts feed, posts should be reverse-sorted by update time, if it exists, and then by <pubDate />.

I love his suggestion of including CHANGELOG style histories, and if possible automatically computed diffs (although in the RSS style, this should be an optional layer above mere updated, so developers don’t experience these ideas as an all-or-nothing a cliff of new work). In my own main blog (but not its more recent successors, I rather painstakingly embedded update histories by hand.

The current proposal does seek to address some of the pain points Chris identifies in an aside, about the ambiguity of what one is getting from an RSS feed. Is one getting a full history, or just the most recent posts? Is one getting full content, or summary description? We propose to describe in explicit metadata both a curation dimension (“last n” or “by category” or intersections thereof or “everything” are all curations) and a completeness dimension, ranging from mere announcement to summary to full-content to full-content-embedding-local-links (useful both for archiving projects and migrating them across hosts). Dynamic services might generate alternatives on demand. We could layer some conventions that static sites could also use, so whatever a default rss feed looks like (publisher’s choice!), it might optionally be accompanied by suffixed alternatives, which if present would allow applications to more easily foresee what they get.

A fun synchronicity is that we intend to develop our XML namespace proposals precisely as sprouts. I publish a blog now at https://tech.interfluidity.com/. It’s statically generated, with default permalinks like $year/$month/$titleish/index.html. But the generator lets me override that.

XML namespaces are identified by a URL, which is arbitrary, but conventionally points to a description of the namespace. We intend simply to publish blog entries with permalinks overridden to $namespace/index.html, and let those sprout via updates as we feel our way through our proposals. The (provisional!) namespace URLs will then be https://tech.interfluidity.com/$namespace (or perhaps http://tech.interfluidity.com/$namespace is better, the site redirects anyway).

Hopefully we’ll bootstrap our way into giving many of Chris’ suggestions a shot!

6

I love this proposal, particularly for the idea of RSS-first apps.

Regarding use cases, funnily enough I actually thought of diffs and changelogs as a possible use case yesterday before seeing your comment above. And I discarded it as a bit of a stretch for RSS :rofl: Looking forward to being proved wrong. I’ve actually been noodling on version management and diff/merge workflows recently (in the context of SVG images).

I’d love to hear about more use cases as y’all think of them. Like, ideally we’d have a nail before we take out this hammer of RSS… textcasting and artcasting are interesting but not quite utterly inspiring, I think…

Heh, a wild idea: subscribe to a movie as an RSS feed. Segments come in over time and the reader plays them as it receives them.

1 Like
7

I love this proposal idea, and RSS seems so ripe as a 3rd-party flexible standard to help make a human-paced internet.

I would also be very curious, as you work on the technical components of your project, to hear more of a history of why RSS has fallen into the (lack of?) use it currently has, and ideas for the social protocol change necessary to promote its use more widely. Private platforms ala twitter attract users via their centrality and easy UI… how can we attract people to a protocol?

8

Just thinking what is really missing is a discovery protocol for discovering a sites or users feed, or feeds. It would be nice to have it in a webfinger response or just a .well-known subtype. It could either give a “meta” feed of feeds or have a mechanism for finding a feed or feeds itself or ideally both options. It would be nice for an ActivityPub user to be able to add feed discovery for their feed or feeds too.

2 Likes
9

Another thing that needs facilitating is sites that share libraries of feeds, or feeds of new feeds. This may require protocol additions like the idea of a meta feed or feed of feeds.

10

A possible third idea is a parallel JSON based protocol to the XML one ? Just suggesting as I think this should be looked into.

11

I personally am a binger, so I’m not sure I’d be excited about a film delivered in segments! But I could enjoy series delivered as short, self-contained, episodes. And some people love podcasts like Serial that build suspense too. (I get impatient!) I can see lots of uses for video delivered as RSS feeds, “vlogging”, announcements, movie trailers. And feed “readers” for short video, which let viewers define folders or groups of feeds could be a sane alternative to TikTok and Reels.

1 Like
12

I was thinking wilder than that :joy: Like, literally playing a movie by subscribing to its feed. Then you’d unsubscribe when you were done.

It’s a silly idea. RSS is for async notifications.

13

whatevercasting is a pretty different way to organize the web, in its way a throwback to the early web. Once upon a time they used to tell us — or we used to tell them — that you should only use semantic tags in your HTML. UI stuff should be layered as CSS so it can be overridden, and so core content could easily be reused in other contexts.

No one really gave a, um, hoot. HTML/CSS provides too temptingly rich an environment for visual tweaking. We used to define our visual layouts as <table>, God help us, and that was before JavaScript became a well-developed super-powerful UI temptation.

But if we steal a page from podcasting, RSS encourages disentangling what’s basic from more ornamental aspects. Client apps distinguish themselves by the aesthetic experience, by how fully and beautifully they support the ornamental.

For textcasting, Dave Winer suggests simple styling (bold, italic) and links as the basic core. But as Apple did with podcasting, extensions might allow posters to include the CSS that might style posts (for the feed or overriding at the item level), which clients could choose to use, or to ignore (just like a podcast client might display or ignore cover images).

Still, as you say, it doesn’t at first blush sound all that compelling. Just as neither users nor authors much cared about maintaining the semantic vs visual distinction back in the day, they still don’t care now.

But artcasting is (shh! don’t tell!) pretty clearly inspired by Instagram and its successors. textcasting is an RSS answer to Twitter / Facebook / Mastodon etc.

For these applications authors (now mere “users”) gave up all creative control over visual elements in exchange for

  1. Ease of authorship, or “posting”
  2. Low friction discoverability offered by the platforms

If we can make progress on those things in an RSS infrastructure and let users add visual metadata — that basic feed readers ignore but elegant clients look for to render, we end up with

  • something much better than the hideous visual monoculture of a facebook or twitter or even mastodon
  • that respects the semantic / visual distinction respected, enabling all kind of remixing of core content and adds accessibility for free
  • hosted on the public internet, with no platform walls
  • offered in a standard, easy-to-read format that a wide variety of clients (and aggregators, and indexers) can work with

I think when you put all that together, I think you do get something pretty compelling, a new kind of web that retains the freedom and author control of the early home-page web, adds the easy participation of walled-garden web 2.0, and offers the lego-block openness that motivated scolds to try to enforce a separation of the visual from the semantic.

Much easier said than done, of course! Building and evangelizing RSS-first dynamic services that enable low-friction creation by non expert users is a non trivial task. On discoverability, the continued existence and widespread deployment of feed readers, however vanilla, at least provides a starting point!

1 Like
14

Kind of a chunky stream!

15

The diff idea really comes from Chris Krycho! (Thanks @erlend_sh!)

Great minds really do think alike, but unfortunately it wasn’t me!

I think it’s making diffs embeddable is an excellent idea, but also potentially challenging.

Chris’ core plea that we treat items as “sprouts” rather than atoms I think is brilliant and important and not so hard to get a start on. Lots of times I want to post a start to something that will evolve, rather than a finished product. The default blog treatment highlights the initial stub, then hides all the important evolution, the opposite of what I want.

Lots of the sprout vision can be achieved with an <atom:updated /> tag and discretionary changelog, which should be pretty simple.

Augmenting those with diffs and revision tracking would be much better still!

16

Great question!

RSS has in a sense already failed, so why shouldn’t it fail again? But then Easter is a good holiday for RSS, because it is always dying or dead, but nearly all regularly updating sites offer feeds.

I guess my main conjecture on this is that the platforms were zero-setup dyamic services that made posting and chatting and participation very easy, while RSS typically accompanies bespoke websites that take at least some technical chops and set-up work before participating.

If we describe a standard protocol for posting into feeds as well as reading from them, and implement the protocol in a community-hostable, open-source reference application, then posting RSS would be more competitive with posting into Twitter or Facebook. If we can extend RSS with some features that render the posts much richer, and that give authors the kind of ownership that comes with ones own website rather than a platform, maybe we can draw people in. Lots of feed readers exist and lots of us still use them, so that can help bootstrap the process.

Obviously the proprietary platforms do a lot more than just let users post easily. They offer rich discovery through their internal network effects and their algorithms. They are optimized to elicit engagement and provide “dopamine hits”. We wouldn’t want to reproduce or compete across all of those dimensions. A lot of people are unhappy with how those platforms operate, dopamine hit or no. We are delighted that people are experimenting with Mastodon and other freer, less evil alternatives. But we think that RSS could serve as a great infrastructure for such alternatives that hasn’t yet been fully explored.

17

Oh I hadn’t clicked on Feeds are Not Fit for Gardening — Sympolymathesy, by Chris Krycho yet. Or I’d clicked and bounced. Thanks for bumping it again. Now I see what you mean. Very interesting… :thinking:

One way to think about RSS is as just a channel for notifications. A lot of what Chris Krycho is talking about is just the ability to decide what notifications to send out independently of what edits are made. The way I tend to think about that is as little warrens where conversations happen. I have ~20 places I can post notifications about my stuff. And I like the control of being able to decide on a per-post basis which of those 20 places is worth pinging (with the risk of people tuning me out if I get too noisy). Hmm, so I guess I’m saying I don’t buy RSS as a very good way to program notifications, because there’s no audience segmentation. So Chris Krycho’s approach is interesting, but I’m inclined (after 15 minutes of thought) to say it’s not really about RSS at all.

18

There are lots of ways Mastodon/ActivityPub and RSS can interoperate. I built a notification service that, among other mischief, posts links to my blog posts to my mastodon account. Using bridges like that, any RSS feed can be attached to a dedicate ActivityPub account, so ActivityPub users can follow the blog. They could provide full text rather than just links, if the ActivityPub server you notify permits that.

Of course Mastodon already implements bridging the other way, ActivityPub → RSS, since every user publishes an RSS feed automatically!

Dynamic RSS services will have to define some version of identity and account portability, like Mastodon does. In the spirit of RSS and my own laziness, I’d like this to be much simpler than what Mastodon does. But if we can interoperate via webfinger, that would be really great. I don’t have a good sense yet of how doable or not that would be.

I don’t have a great answer to the more general discoverability question. The existing infrastructure surrounding RSS means that anything that exports it at least has a potential readership among existing feed-reader users. I think that’s an asset for bootstrapping interest.

But the extensions we are proposing to propose won’t be supported by existing readers. We’ll face the joint problem of getting people, including developers, to notice we’ve defined metadata that could enable realy rich, fully public, applications they’d love to uild and use AND helping early tentative adopters to get their work discovered before whatever our extensions enable is fully supported by clients.

19

RSS is just a representation, and the same content can — often should — be conveyed over multiple feeds. You can have all the audience segmentation you want, as long as you have ways to help inform the segments which feeds they should be following!

I agree that Chris Kryko’s post is largely about decoupling announcements from the default of once-only-at-first-pubication, and choose for “sprouts” a more suitable policy. I think that too is something RSS should easily be extendable to support.

(Arguably Atom already has extended RSS to support “sprouts”, though I don’t know how commonly readers honor embedded <atom:updated> tags.)

20

Absolutely. Feeds need to reference feeds. To ease discovery, and also feed items might reference comment or reply feeds.

Dave Winer has already extended RSS to embed blogrolls and follows as OPML subscription lists, which are precisely a kind of meta feed.

For my own podcasts, I built a service that subscribes to OPML, this meta feed, fetches all the podcast feeds, fetches all the individual podcast items, and then combines them into one feed to rule them all!

Supporting and surfacing <source:blogroll /> and <source:subscriptionList /> is something more clients should do. We’d hope to use and build on these definitions.