Agentic Websites - Whitespace and Opportunities

Back in early July, I started thinking about what 'the future of websites' could look like. Having spent the previous months deep in Claude Code and agentic coding, it was clear that the whole structure of websites was about to change. But to what? That was the question.

After some deep research, it became apparent that the answer to the question was multi-faceted and complex:

• There were layers that were being built for product companies by platforms like Shopify to make it work out of the box for the agentic web
• There were components - APIs, MCPs, new integrations - that could be bolted together for a service company or independent operator that technically worked, but were still high friction
• There were new standards being developed for the agentic web (UCP, etc.) that will one day become standardized rails, but it's still very early days

The sheer depth and fragmentation of the information made it feel overwhelming, but I wanted to start somewhere, so I decided to redo my own website first. This site - lumosbusiness.com - has technically been around since 2010 through various iterations, all of which have the traditional hosted CMS (Content Management System) route.

Well that site is now built!

The whole process and structure of the new site is documented here - Rebuilding My Website for The Agentic Web. Instead of the typical CMS/WordPress stack, we are now dealing in the world of YAML, /agents.md, JSON, and a publishing system that works by essentially editing code in an IDE.

It is an experiment, a live experiment. Functionally, that means that the research will continue and it will continue on this blog. Upon getting this first iteration out the door, I thought it would be good to reflect and review the original research and see where we are at now across multiple threads. So here it is, a review of the core pieces of the original thesis, 6 weeks later.

By no means is this meant to be an exhaustive list or comprehensive review of everything that's out there; that would be a full-time job to keep up with. Instead, it is meant to review the core thrusts and evaluate them against first-principles, an artifact that will live on for those who are curious about the thought process behind this site and where things might be going in the future.

Let's dive in!

Thesis 1 - Shopify and the Inflection Point

Projected Impact: Shopify will help drive standards for the broader web

Shopify is a $200 Billion dollar company. Their CEO - Tobi Lutke - is not only one of the top entrepreneurial voices in tech across commerce, but also a top-tier developer himself. Shopify has been very early in their positioning for 'agentic commerce.' When they started releasing a set of agent standards for their storefronts earlier in 2026, it represented a 'watershed' moment for the future of the agentic web.

In Q2 '26 they added six agent endpoints (Craftshift), all automatically generated for a shop's store profile and catalog:

Path	Format	Purpose
/llms.txt	Markdown	Short curated notes for AI Agent and LLMs
/llms-full.txt	Markdown	Extended version with more endpoints and data hooks
/agents.md	Markdown	Full agent instruction manual; UCP flow, MCP (Model Context Protocol) usage, rules
/.well-known/ucp	JSON	UCP discovery and capability detection
/api/ucp/mcp	JSON-RPC over HTTP (MCP)	Programmatic commerce endpoint for AI agents
Sitemap_agentic_discovery.xml	XML (linked from /sitemap.xml)	AI-focused sitemap surfacing three discovery files

For Shopify's millions of merchants and more than 75,000 Enterprise customers (StoreLeads), this works out of the box. There is no app to download or theme settings to customize, agents can crawl Shopify sites and purchase without the store owners themselves needing to be AI Engineers to set it up.

First Principles - Who Are These 'Agents' and How Do They Read?

'Agents' is a somewhat ambiguous term in this context; these are not self-sufficient bots buying shoes for their hypothetical selves. Shopify has a preselected group of official 'agentic storefronts' that can act on behalf of users to purchase goods from the agentic interface:

• ChatGPT by OpenAI
• Copilot by Microsoft
• AI Mode in Google Search and via Gemini App
• Shopify's own Shop App

(Shopify)

'Agentic Storefronts' is simply a sales channel for consumers who begin transacting via their favorite AI apps; nevertheless, the agents need to be able to read and parse the store's content to understand all the salient details of a potential transaction, this means:

• Using /agents.md as essentially the instruction manual for navigating the store, including the Shopify Catalog that maps out products available and the key data points
• Leveraging UCP (Universal Commerce Protocol) for transacting across different merchant and payment terminals
• Reading through key details in Markdown (.md) and in some cases Text (.txt) to extract information related to questions or concerns a user may have

This makes Shopify stores visible in search on ChatGPT, Gemini, and more, and enables the transaction to originate and complete without the user themself visiting the actual Shopify site. Shopify takes a 4% AI Transaction Fee on these platforms like ChatGPT and puts in place a policy that requires human review for "buy-for-me agents.'

Thesis 2 - New Agent Readability Standards will Emerge

Projected Impact: /agents.md will move from the de facto coding agent standard file into the broader web for agent navigation

Back in early July, there was a lot of signal across various vendors, blogs, and social media that llms.txt would become the standard for agent crawlability on any website.

In my own experiment, I added llms.txt to the site (see Rebuilding My Website for the Agentic Web) as a test. Website builders like Wix started adding support for llms.txt (Wix), while many popular WordPress SEO plugins added that same capability.

The challenge with llms.txt back then, from a countersignal perspective, was that adoption was still relatively low, yet those who had implemented it were not seeing measurable results. Ahrefs found that 97% of /llms.txt files received zero requests, while Limy found ~408 llms.txt requests in 500M+ bot events (Limy). Furthermore, Google and major LLM search players (including OpenAI, Meta, etc) had not committed to using llms.txt for indexing.

Then this last week, on August 25, 2026, the Shopify CEO Tobi Lutke came out swinging on X on the /agents.md standard that is generated in the repos of agentic coding projects for agent memory and context management:

I'm thinking about banning Claude Code at Shopify until they change their mind and read AGENTS.md and .agents/skills etc. Insisting on only reading CLAUDE.md sometimes leads to split brain problems where different team members use different tools. Just unnecessary …

So many people telling me tricks for dealing with this- I know them all. But agents and claude files are recursively applied through the tree. With thousands of developed [SIC] in a mono repo, it just does happen that one directory is missing one of the two files and this means that a subset of devs work with lobotomy. We fix this with automation but it's a stupid complexity tax that shouldn't have to be paid.

(@tobi X.com)

Since Claude Code uses /claude.md as the de facto agent file, and the other coding agent uses /agents.md, this creates a 'complexity tax' for a company like Shopify that is trying to standardize agent crawlability and navigability on the web.

Based on a cursory sentiment analysis, many top developers and engineers support Tobi's stance as a means to cut complexity as agentic commerce is projected to start scaling up.

In fact, this research led me to change this site itself while reviewing it. /agents.md went live for this site on the weekend after originally focusing more on /llms.txt. The research never sleeps!

First Principles - The /agents.md standard

In Claude Code, OpenAI Codex, Cursor or any other leading coding agent, the /agents.md or /claude.md file is of paramount importance to the success of the coding project.

Coding agents load that into context as their memory, so it must be succinct and actionable.

You would be forgiven for thinking that the /agents.md files for coding agents and storefronts are the same, but in fact they are not. The original /agents.md is an OpenAI-led standard from 2025, now under the Linux Foundation. It lives in the code repository and instructs agents working on the project. That's where Tobi's complaint above is referenced from.

Shopify's /agents.md borrows from that convention, but is structurally different. This is a file at the web root of the storefront that tells agents about the key details of the store and where the transaction endpoints live (UCP, MCP).

Hard to say (yet) if /llms.txt 'lost' to /agents.md yet for the web as a whole, but its early performance has been underwhelming and firms such as Shopify continue to push in the other direction.

Shopify made /agents.md the canonical reference - ie. the standard or established rule - for their store back in May 2026. Now, with the shot across the bow to Claude and Claude Code, it represents an opportunity for broader standardization across the web as a whole.

UCP (Universal Commerce Protocol) - a partnership between Shopify and Google - is tied into this as well, since the /agents.md file defines certain contractual parameters. UCP is part of a broader 'agentic commerce' standards agenda for how agents transact (UCP) as a REST and JSON-RPC protocol for transactional data. This contrasts against the ACP standard by OpenAI and Stripe, projecting a future 'protocol war' between the different tech heavyweights.

Thesis 3 - The Lack of the Demand Side (Consumers) and Why It's Early Days

Projected Impact: At some point, the consumer side will crack open and the demand for top-ranked brands will be exponential

Shopify represents the supply side for commerce and product-centric companies. The same set of standards are non-existent on the service economy side; it is a fragmented mix of solutions depending on the web stack.

The demand side was, and still is, essentially non-existent for 'agentic commerce.'

Back in March 2026, OpenAI pulled out of the Instant Checkout pilot with Shopify with only about a dozen merchants live, shifting the model to discovery-in-agent, checkout-at-merchant.

According to Checkout.com's June 2026 'Agentic Commerce 2026' Report (Checkout), agents accounted for only about 3% of transactions; this contrasts to about 9 out of every 10 businesses readying themselves for the demand side pipes to open up.

In recent months, however, demand-side activity around browsers continues to look promising. Perplexity's Comet browser received a favorable ruling in their court case against Amazon, while Claude in Chrome and Gemini in Chrome continue to gain traction.

The TLDR is that the average consumer is not currently transacting on ChatGPT, Gemini, Copilot or other AI chat apps. The supply-side platforms themselves continue to ship in preparation for the day the spark is lit, with Shopify being the torchbearer.

First Principles - It's Tepid Now, but The Growth Will be Parabolic

We have already seen many different instances of inflection in the early AI days: first it was ChatGPT, Perplexity, and the dawn of the AI-consumer chat boom; then it was the coding agents, and the absolute explosion in token volume for Claude Code, Codex and more; next it certainly seems like someone will crack 'AI Shopping.'

There are a variety of reasons why it hasn't taken off yet:

• Lack of interoperability between platforms makes creating a smooth, frictionless consumer experience across platforms difficult
• Consumer trust issues with AI persist, some people are not interested in using those channels and will stick to the traditional eCommerce path
• Innovation in the consumer experience itself; there are still multiple form factors (such as video) that may make the whole AI chat experience for commerce gel

It's hard to predict what and when will bring the demand side to life, but as we have seen with other instances in this market, when it does, it will be electric and lead to parabolic growth.

Thesis 4 - Headless CMS will Replace Traditional CMS, but it Won't Be Seamless

Projected Impact: Simple products to manage the gap between a traditional CMS and a headless CMS will emerge, but they aren't here yet

Back in early July, there were many people experimenting with something like the following:

• Migrate off WordPress
• Use Claude Code to manage the migration
• Test a platform like Sanity as a headless CMS option
• Relaunch the new site

This was essentially the research backbone for my own site migration from WordPress. I landed on a bespoke set of primitives without any third-party interface as documented in Rebuilding My Website for the Agentic Web.

The gist is YAML frontmatter, MD (Markdown), JSON-LD, and /agents.md for navigation after having upgraded the original llms.txt thesis. Content is 'managed' through Claude Code and the Cursor IDE, so it is very coder-centric. The CMS is a missing piece compared to the WordPress days, but as a whole it works much better than the old way now that I am fluid on Claude Code and can navigate the codebase for updating content and making posts myself.

But this isn't going to work for the vast majority of non-technical people. The CMS plays an essential role for the average business owner who wants to change their header photo or add a new product page.

From a quick scrape, it appears that the major platforms who could become Headless CMS standards have pivoted more towards agentic content management or operations:

• Sanity rebranded as a content operating system
• Contentful was acquired by Salesforce
• Payload was acquired by Figma

That makes sense commercially, as headless-CMS standards become the agent content infrastructure; all the same, the core problem remains a big one to solve going forward.

First Principles - Website Management Needs to Become Simple on Headless

'Headless' simply means that the site can be rendered from the code repo across multiple different surfaces, with the record being maintained in the code as the ultimate reference.

We have seen 'headless' in the eCommerce space for several years, but it hasn't completely taken off due to the added complexity the process adds to the management of a store.

Websites will be the same. The average person doesn't want to deal with the complexity of managing a website through a coding agent:

• There is a huge learning curve to manage, especially when it comes to troubleshooting any problems that might emerge
• There is a cost of needing a subscription to a service like Claude Code, OpenAI Codex or Cursor that must be paid
• Having third-party contractors manage that process might work for some businesses, but for smaller operators the cost doesn't make sense

Shopify has this covered for merchants and anyone with a product focus. But if you are selling services in any capacity, publishing a blog, or otherwise just looking to create a website, there is no obvious option:

• Relying on platforms or solutions like WordPress, Squarespace, Wix, Webflow or others comes with a set of tradeoffs that become more obvious as the agentic web grows
• Managing a customized stack requires ongoing maintenance, learning, and managing various friction points compared to the above

One day, we will move towards the magic middle, but for now it's still a grind one way or the other.

Thesis 5 - The Services Economy will go Agentic Next

Projected Impact: The impact of the agentic web on the local services economy will probably be bigger than 'agentic commerce' for eCommerce.

Think of the setup of the typical local services company:

• A website made on Squarespace, Wix, WordPress, or some other platform
• Google Business Profile (GBP) and a focus on getting as many good Reviews as possible to boost GBP and Map Pack rankings
• Some form of digital advertising on Google, Meta, or other platforms
• FSM (Field Service Management) platforms that help with booking, scheduling, managing jobs, etc.

Every layer of this stack is ripe for disruption in the agentic web, including potentially the most prevalent layer of the stack, GBP. Many service businesses can create a bare bones, basic website over a weekend and then go all-in on getting five-star reviews on their GBP. This will have them become #1 on local search for their trade (ie. 'local plumbers') and map pack.

This whole process can be gamified, however, and leaves room for so much innovation as the agentic web takes shape.

Google itself announced agentic booking for local services at Google I/O in May 2026 (Google). This means essentially an AI itself parsing the service providers website and phoning the business. This is a marker of the future demand side in the services economy.

In fact, the services side of the agentic economy may play out in the opposite way that the products side is playing out. We see the plumbing being built out in a standardized way for small and medium-sized merchants on Shopify, and demand staggering behind.

Looking at FSM (Field Services Management) platforms as a proxy, there is no such agentic standardization on the supply side; yet Google themselves are implementing the pieces to make agentic search plug into the local services economy. ServiceTitan, Jobber, and Housecall Pro have all shipped AI, but it's inbound voice and internal copilots, not external-facing quote or booking endpoints.

This Thesis could be carved out for an entirely new post in the future, but to keep it simple for now:

• Consumers will originate their queries on an agentic search interface like Gemini or ChatGPT in a much different way than they currently use the web; the query is more conversational rather than a set of fairly specific search terms
• Only those services companies whose websites are easily readable or crawlable for the key details will be surfaced by the agent back to the consumer
• Unlike online shopping, a services booking payment doesn't need to happen in the agentic interface, but the quoting and booking likely will
• That means there is an entire restructuring process for the website that is needed; relying on the website builder to spur this is not the wisest move, seeing as how they don't have deep expertise in every Services vertical
• There is a huge opportunity for the major FSM platforms, but even in the event a platform like Jobber or Angi cracks this, there is still the need for a local services company to understand and implement it on their own site and ensure they rank properly on search

First Principles - Thesis 4 (Headless CMS) and Thesis 5 (The Services Economy) tie together, the stack needs to become more seamless

The key pieces of the stack are starting to emerge. We see crossover standards in eCommerce (ie. /agents.md) and the opportunity to experiment on a custom stack through something like Claude Code.

But this won't work for everyone. People running services companies need simple solutions to manage this part of their web presence.

Whether it's custom agents, novel CMS implementations, new search strategies, or a combination of all of the above, it's hard to say. But the services economy presents a huge opportunity for agentic web development and lead generation.

Overall - Seize the Moment

Week by week, everything changes in this space and it is impossible to keep up with. Parts of this post will probably be obsolete within a few weeks, while others may have enough prescience to last through the end of the year.

Rapid change is the only constant and the tools are there for any operator to experiment with: Claude Code, Codex, Cursor, APIs for agents, MCPs for running these experiments from chat.

What could change this post in the future?

• A major AI platform committing to fetch /llms.txt or /agents.md in production would settle the debate (Thesis 2)
• Agent transactions crossing 10% (from today's 3%) would be enough to declare the demand side has cracked open (Thesis 3)
• A turnkey product that gives non-coders a real CMS on a headless stack would close the gap (Thesis 4)
• A major FSM (Field Services Management) platform shipping an external agent-facing quote or booking endpoint would mean the services economy is reaching an inflection point (Thesis 5)

The complexity in the technical setup has been replaced by the complexity of a million moving pieces making many things seem fragmented and difficult to comprehend. But in chaos, lies a set of amazing, once-in-a-lifetime opportunities for reinvention and market momentum. Let's see what happens.

Fetch /agents.md. It wasn't there last week, and this post is why it is now.