MCP: REST reborn?

CP is trending in my social media feeds. One popular take is “what’s the big deal? Isn’t MCP just APIs?”

Jacob Colling, author of AI tool Caret (check it out!) put the refutation to this quite succinctly

MCP enables self discovery of tools and auto updated call structures. That’s something you can’t do with a REST API client

A Tale of Two RESTs

There’s irony here. “REST” actually has two meanings. 99.9% of the time, when practitioners say “REST” they are referring to industry REST – which is basically RPC over HTTP with JSON organized around “resources”: it is absolutely true that the APIs of this tradition do not support any kind of self-discovery. On the other hand, there’s hypermedia nerd REST. This is REST as Roy Fielding (who coined the term “REST” in his 2000 dissertation) described it. Self-discovery is not only supported by OG REST, it is the foundational idea of OG REST. My favorite place to read about this is from Carson Gross on htmx.org. You can read “How Did REST Come To Mean The Opposite of REST?”. “HATEOAS” is also useful jargon to search for, if you’re interested in learning more about the tradition of self-describing REST.

You should take the time to understand this history. It’s not just about REST. Few know this, but the entire web is a battleground between these two competing visions: Hypermedia vs RPC; browser-as-universal-client vs. browser-as-bespoke-application-delivery-platform, user agent vs ui toolkit. Not just the web: all of software is a clash between two fundamentally different philosophies: one empowers the user, one empowers the developer. My favorite discussion of this is Eric Gade’s talk “In search of the Personal in Personal Computing” (which I was lucky enough to attend in person!) The thrust is, “personal” computing has gotten a whole lot less personal over the years. “View Source” on the web used to be a useful thing to do – you could remix things you saw and liked. Going back further, “Hypercard”, a precursor to the Web, was a visual environment for authoring applications that came with every Mac. It had a very accessible, natural-language-like programming language “Hypertalk” that made it easy for the end user to inspect, modify, and redistribute. Even further back, Smalltalk “for children of all ages” didn’t really distinguish between “programmers” and “users” at all. There was just computing, and participating in it.

It’s not exactly good vs. evil. Obviously, I romanticize the “lost vision” of the early web, and the idea of “empowering users” seems very noble on the face of it, but the fact of the matter is that most users don’t actually want to be empowered. They want software that works and that they don’t have to think about, and the natural way to achieve this is to cede all the power to the developers. There are good reasons why software is the way it is today.

MCP: The battle continues

Generative AI is a paradigm shift. Modern LLMs stand to democratize computing. You can now use ordinary language to accomplish sophisticated computing tasks that, just a few years ago, were only within the capabilities of the “scribal class” (to use Gade’s analogy – seriously watch the talk) of software professionals. So it may be time to revisit old battles: like self-discovery, for instance.

To be honest, I never thought self-describing APIs (HATEOAS, to use the jargon) actually made that much sense. It’s a cool idea, but it puts so much burden on the user. The user has to be the one that does the discovering. You can’t really consume the self-description of an API programatically, at least, not in anything but a very generic way, which means you can’t automate anything for the user – which really limits how useful a self-describing application can be.

But now, everything’s changed. LLMs can usefully consume the self-description of an API and provide automation for the user on top of the basic operations exposed by the API. And, like Jake Colling noted so incisively, self-discovery is the distinguishing feature of MCP. Compare MCP to GPT actions for instance. With GPT actions, you upload an OpenAPI spec that defines the actions the API supports ahead of time, i.e. there is “out of band” communication re: the specific protocol how the client (ChatGPT) will talk to the server (the API). OpenAPI is very much a technology after the tradition of RPC APIs. The client and server must agree on what the protocol is before any communication takes place. There is strict versioning. Whereas, with MCP, the client connects to the server, and only then discovers what actions it can take. And this may change during the course of the session.

At the end of the day, both technologies allow for much the same thing – developers write tools, and put them in the hands of users chatting with an LLM. What matters more than the specific technologies used is the struggle of ideologies. Is the user being empowered? Or is the developer being empowered? Right now, the state of affairs in Generative AI chat applications reminds me very much of the early web. Users are very much empowered. There are lots of options to choose from, when it comes to LLM assistants – much like there were many different browsers in the early days of the web, and users have a lot of power and responsibility over the experience they have when they use an application like Claude Desktop or ChatGPT. You can set your own prompt, for example, which gets applied regardless of which tools you are using – much like in early versions of Internet Explorer, you could set your own CSS stylesheet that would be applied across every website you visited.

But I think, over time, something to watch for is control being ceded back from the users to the developers. One scenario – if you’re the developer of an MCP Server, or GPT action – you might have users complain that your tool isn’t working the way it’s supposed to work, and the reason might turn out that they’re using Gemini as their model, when you did all your testing with OpenAPI and Claude. Or the reason might be, they had a really weird custom prompt that they set in their assistant that ended up interfering with tool calls. This is analogous to how, back in the 2000s, as a web developer you’d get a bug report from users, and it would turn out they were using some version of Internet Explorer that you didn’t test on, or maybe they were using Opera – and all the different web browsers implemented web technologies differently. As web developers moved away from building basic websites with forms, and moved towards “Web 2.0” and building highly specific, interactive experiences, heavily dependent on the granular details of how the predominant browsers operated – this created pressures that ultimately destroyed the variety of browsers that users had to choose from, and what killed the personalizability of the web. This same dynamic could happen to AI chatbots, too. Developers trying to build highly specific experiences for users could push the industry towards a few winners, away from variety and customizability, and towards one-size-fits-all conformity. I don’t think this is inevitable, by any means – maybe it’s not even likely: generative AI and the web are two very different technologies – but it’s something to think about.

Obligatory calls to action

If you are building in the AI space, my plea to you is simple: take advantage of the amazing ability of generative AI to adapt to your users preferred way of doing things. Lean into personalization and customizability. In traditional, pre-generative-AI software, in order to build an amazing experience you often had to curate the heck out of it. You had to build one, clear, obvious, extremely opinionated way of doing things. That is still true to some degree, but much less so. Get with the times: the age of “personal” in computing is due to return. Embrace it!

My other call to action: if you haven’t, take some time and make sure you understand the history here, and the distinction between the RPC and Hypermedia approaches to web technology. Read the Hypermedia propaganda on htmx.org; better yet: go through the exercise of implementing a toy project in HTMX (actually implement it, don’t just vibe code it). Even if you don’t drink the Hypermedia kool-aid, understanding this aspect of the web platform will actually help you write RPC-style web APIs better, and save you from a lot of foolishness. Namely, a lot of parts of the web platform, are intended for a Hypermedia context, and offer basically no value if you’re writing RPC-style APIs. For example, POST, PUT, and PATCH. Before I found enlightenment, there were hours of my life I wasted worrying about verb distinctions in private APIs. The law of the hammer strikes again: the HTTP spec provides these things, there must be some benefit to distinguishing them, right? Well, the benefit that a generic Hypermedia client with no specific knowledge about your API can display different UI for a partial update vs. a full update vs. original creation. If your API client has out-of-band, specific knowledge about your API, there’s really no non-documentary benefit to drawing a distinction (well, PUT is technically idempotent, so I guess there are cache implementations). More generally, a lot of the reason why the HTTP spec is so specific about a lot of things is for the benefit of generic clients, reasons that are completely inapplicable to 90% of “REST” APIs that exist. Learning about Hypermedia will help you understand the “why” of so many parts of the web better.

Peace!