Rebuilding My Website for the Agentic Web

Lumosbusiness.com has been alive as a blog since 2010, with several migrations since its inception:

• The first iteration was on a custom stack built by a web developer when the site was first launched on lumosforbusiness.com; that site lived there for several years before the stack itself was deprecated
• The second iteration was launched as a Wordpress on DigitalOcean as a Droplet, where it ran for several years; one day I woke up and the whole site was down because the Droplet had not upgraded the infrastructure stack properly
• From there, I moved the Wordpress setup to Siteground and essentially had to launch the blog (as it looks in its current iteration) from scratch

Each iteration was costly in some sense. The site used to rank as the #1 for many of the business model posts, and relaunching each version forced almost a ground-up rebuild for many of those same top-ranked pages. Moreover, there was the messy manual function of having to reimport the posts onto each new host. While it was a good learning lesson in managing websites, I would rather focus my time elsewhere.

In the last year or so, I have ceased from blogging not due to lack of ideas, but a sense of staticness with the website builder and experience itself. Siteground is a reliable option but the costs for basic Wordpress hosting add up when I am not doing anything with the site itself.

The Dawn of the Agentic Web

A few years ago when I was researching ChatGPT (see Generative AI Business Model) and other related early-stage AI companies, I had a couple of realizations:

• The web was about to change substantially and structurally
• The way agents were crawling and many cases training off of open blogs made it essentially pointless to blog to gain any visibility

I thought about and was encouraged to do something like Substack, but I liked the idea of maintaining control of the content on my own website.

Despite a few sparse posts on social commerce and video commerce in the interim years (a current area of focus and activity), I decided to leave it alone and dive deeper into some of the earliest agentic coding applications; my first foray was in tools like Lovable and Replit, and then earlier this year I started to get heavy into Claude Code.

After being mind blown at the capabilities of Claude Code and related tools (Codex, Cursor, etc.) I started to lean into the coding agents, spending nights and weekends becoming proficient in Claude Code specifically.

That's when I started reading posts - on social media and in blogs - about how people were completely upending their current webstack and deploying a new version of headless using Claude Code.

Flipping the Wordpress Stack Upside Down

Wordpress blew up as a tool in the 2000s, becoming the early and dominant blogging platform; first, personal blogs, and then as a tool to launch business websites that also had blogs. Blogging in the 2000s and 2010s was a popular tool to gain SEO clout in the search engines, and become wedged into specific niches as an 'expert.'

As social media and video have become dominant, and Google has changed its search algorithm in recent years, it has become extremely difficult to rank an independent blog high in the search rankings. You are deboosted relative to major publishers in most cases.

As mentioned above, many LLMs (Large Language Models) also train their models on specific niche content on the internet to build their own expertise. Those same LLMs are becoming the predominant search engines of our time; yet in most cases, they won't cite your website unless they can in fact crawl it properly to begin with.

Thus two key levers of blogging/writing have been structurally inverted in the last few years: distribution has been dismantled, and LLMs have taken over.

This has rendered the traditional Wordpress web-engine model effectively useless, as an individual can pay to keep it hosted/up-to-date and see virtually zero benefit from it.

And so 'headless' has become a new theme. On the back of Claude Code, the core premise of headless web development is that the major components of the site - content, blog posts, etc - are a set of pre-built static files, compiled from the GitHub repo on push, served from an edge cache. Nothing renders in real-time which is why it's fast (and free for now!).

The Death Of The CMS?

CMS or Content Management Systems are the interfaces that many people - technical and non-technical alike - use to manage their websites.

Platforms like Wordpress, Webflow, and a host of others create the version of the website that is displayed to a site visitor in any given minute. But in the moment where you want to change a product name, email address, or write a blog post, you use the CMS.

The CMS is an interface that makes it easy to manage content on a website without any technical know-how. Especially for individuals or brands who have hundreds or thousands of webpages.

Claude Code drives the agentic coding function while a framework like Astro creates a structure to manage and deploy content. Unlike a CMS, however, there is no official 'interface' to edit content unless you build it yourself. In its place, you use coding agents (or edit the code yourself) to make even small text changes.

Let's say you are operating in a headless setup and using Claude Code:

• You prompt in the specific set of changes you want to make
• Claude Code updates the code in the repo
• The site displays the updates as soon as the repo is pushed live

It works completely fine, but it creates an entirely new array of workflows and friction compared to the traditional CMS; the average person doesn't want to be messing around in 'code' especially where Terminal commands are involved.

The TLDR - the CMS is definitely not dead. But it will be reworked because while 'content management' itself becomes more difficult in several ways, content distribution gets much easier.

YAML, LLMs.txt, and Agentic Discovery

Conversely, the structural dimensions and additional levers of control in the headless website setup enable new levels of customization and distribution.

That's because agents are now crawling the web at an exponential pace and parsing through every byte of content at their disposal. Those same agents are responsible for surfacing results in mainstream chat LLMs like ChatGPT, Gemini and others.

So now the question about future websites can be framed properly; how do agents crawl, index, and think about the websites they read?

How do Agents Read Websites?

Agents don't browse websites the way that humans do, they extract. They are on the hunt for facts they can analyze and interpret. In the context of a website, they are looking for: who is behind it, what the entity or individual does, what their services are, etc. To an agent, the traditional spit and polish of a well-crafted website is invisible; this includes spiffy layouts, glossy hero sections, and splashy images - none of it gets parsed.

YAML (YAML Ain't Markup Language) is the language for transcribing 'the record' or the content ledger that lives in the repo. Everything major on the old LumosBusiness.com website is now structured and recorded in the same way it would be on a conventional website, creating sections around the offering, about, etc. Technically, the breakdown is YAML for the facts, markdown for the writing.

The current surface you are reading this on, the website, is showing the data as a rendering from a single source, the code repo. In this case, Astro is used to deploy the files into the rendered version of the website, which is about as plain vanilla as it gets in terms of design.

So what's rendering on this version of the website that is helping agents read through and parse through the website:

• The human pages - what you are seeing as you navigate through the site's menu
• The JSON-LD graph - the machine-readable entity data embedded in every page
• llms.txt - the agent's table of contents
• /agents.md - the operator's manual: how an agent should read and act on the site
• Posts .md - a plain markdown mirror of every single post

As you - a human - are reading this post, an agent could be as well. And that is the bet we are making here, that agents are about to take over the web - at least from a crawling perspective. To index and appear on the agentic web, which eventually comes back full circle to drive human traffic, one must start making these types of bets now.

The Bets - Agentic Websites in the Future

• The website becomes a renderer - the above-mentioned YAML files and corresponding content ledger in the repo serves as 'the record.' From here, a website becomes one of multiple possible versions of the record that can be generated. As the web morphs and changes, this type of structural agility becomes a major asset. As a major bonus, no more website migrations!
• The audience split - there are now two major consumer types on the internet, agents and humans. The days of the heavy, design-driven Javascript websites with multiple third-party Cookies loading on every request are gone, the future needs to be designed around legibility for agents. From a content perspective, this means stripping out a lot of the fluff and shipping the facts in as clear and readable format (for both audiences) as possible.
• Beyond readable, it becomes actionable - the frontier part of the site: a MCP server that will let agents connect and take action. If you think of this across service categories and professional services, this will mean things like quoting, booking, and other major actions happening agent-to-agent via MCP. The current version of the site already accepts structured endpoint inquiries, and a full MCP is already built and slated for future activation. MCPs become the major gateway to go from agentic web browsing to agentic booking.

Closing Summary

This site is early and experimental for a reason, it serves as a published record of what has been done to date and where we might go in the future. Nothing here is finalized or polished, the post serves as a rationale as to why these set of actions were taken and what we might expect to see in the future.

You don't even have to take the post's word for any of it.

View source. Fetch /llms.txt. The record this post describes is the same one serving it.