Nobody evaluating an agency wants to hear that the thing their board is asking about is unmeasurable. So vendors do not say so. They sell dashboards instead, with rankings and share-of-voice tiles that update every day, and they book revenue off a quiet promise that the number on the dashboard means something.
I think the honest answer is that measuring AI visibility is the industry’s largest open problem, and the people best placed to say so are starting to say it in public. That is where this piece sits. A commerce director asked me last month how we measure whether a brand is being recommended by AI, and I did not have a satisfying answer. I went and read what the people with the sharpest views on this are saying, and the shape of a defensible position fell out of their disagreements.
The disagreements matter. One camp has decided this is a tracking problem and is building the tools. Another camp is quietly admitting the tools cannot see what the question needs them to see. A third is reporting that the money attached to this channel, measured honestly, is still a fraction of the money attached to search. All three can be right at once, and reconciling them is the actual work.
What follows is not a recap. It is five positions, each stated in the voice of someone on the record, each tested against the figures underneath it, and a build section at the end that answers the question your board will actually ask: what should we instrument on Monday, and how much will it cost.
The people I went and read
Five named voices, all with a published statement under their byline on the question of measuring AI visibility. Nothing below is an interview; every quote is drawn from a public recording, blog post or press release, with a link.
- Rand Fishkin, co-founder and CEO, SparkToro. Two 2026 posts and a research study with Gumshoe.ai, running 2,961 prompts across ChatGPT, Claude and Google AI.
- Jodi Cerretani, CMO, WordPress VIP. Co-host of the enterprise AI discoverability webinar series with 97th Floor, three sessions aired between August and October 2026.
- Paxton Gray, CEO, 97th Floor. Co-host of the same series, writing on measurement explicitly.
- Jim Yu, founder and CEO, BrightEdge. On record naming AI measurement as the product category his firm is building.
- Marie Haynes, consultant, cited for her October 2026 analysis of twelve months of Google Analytics 4 data across 100 websites.
The disagreements group into five positions. They are not a tidy progression; two of them straight-up contradict each other.
Position one: AI rank is not a thing you can measure, and anyone selling it as one is selling you a map of a landscape that will not sit still
Rand Fishkin and Patrick O’Donnell of Gumshoe.ai ran a repeated-prompt experiment through ChatGPT, Claude and Google AI, 2,961 runs in total, and Fishkin’s reported conclusion in Search Engine Journal is as blunt as these things get: any tool that gives a ‘ranking position in AI’ is full of baloney
. The underlying numbers are the reason. The same prompt returned the same brand list in under 1% of responses, and the same list in the same order in under 0.1%. If the thing you are measuring is noise 99 times out of 100, there is no useful rank to average.
Fishkin’s methodological point is that you can measure brand presence across a large sample of runs even when individual rank positions are unstable. That is a real finding and it is the honest base for an AI visibility report. It is also structurally different from what most of the tools on the market claim to deliver, and a buyer should be able to tell the difference by Monday afternoon.
Jodi Cerretani makes a complementary point from a different direction. In the first session of her series with 97th Floor she said, on record and about her own company’s product, that current measurement tools, mine included, here at WordPress VIP, they really measure humans because they run on JavaScript in the browser. And AI crawlers don’t execute JavaScript.
A vendor undercutting her own product in public is not a rhetorical move, it is a tell. The tools cannot see the thing you are asking them to see, and the people building them know it.
Position two: it is correlation, not causation, and the industry’s credibility depends on saying so out loud
Paxton Gray is the most useful voice on this, because he runs an agency selling AI visibility work and still says the harder thing. In session one, at the twenty-four minute mark, he put it this way: it’s one thing to see are we showing up, and then it’s another thing to see is showing up having an impact on the bottom line. And that is a big gap that I don’t think there’s really great answers for right now. There’s a lot of correlation and not causation.
He returns to the point in session two, naming measurement as the biggest bottleneck that I’m seeing in the industry to advancing the strategy in AI search
. The honest consequence is that the metric your dashboard is trending is not the metric your CFO is going to accept as proof of impact, and the correct thing to do about that is to tell the CFO so before they find out from the deck.
Gray’s own suggestion is to measure this the way a billboard gets measured: are people searching my brand? Are people coming direct to my site? Do they convert better?
That is a leading-indicator model, and it is the model that fits the thing being measured. It is also the model that the vendors with real conviction appear to be converging on.
Current measurement tools, mine included, really measure humans because they run on JavaScript in the browser. And AI crawlers don’t execute JavaScript.
Jodi Cerretani, CMO, WordPress VIP
Position three: measure it as a brand channel, invest in it as a brand channel, and stop pretending the funnel starts in a chatbot
Fishkin’s 22 October 2026 SparkToro post is the clearest expression of this view I have seen in public. His formulation is that AI is, probably, quite influential on some buyers in some sectors
, and the operational consequence is that it should be invested-in and measured as such
. He credits Mike King of iPullRank with the framing of AI as a brand channel rather than a performance channel, which is the frame doing the real work.
Jim Yu puts a vendor’s version of the same case on the record. In BrightEdge’s September 2025 press release launching its AI measurement product, he says the companies that win will be the ones that understand how they appear in AI search and why
, and that marketers can finally measure AI’s impact, act on insights in real time
. The product name on the release is AI Hyper Cube and the pitch is a dashboard. The honest reading of Yu’s own release, however, is in the next paragraph down: BrightEdge’s own data had AI search accounting for less than 1% of referral traffic, with organic search still carrying the conversions.
So the brand-channel case is not that AI visibility is small and ignorable, nor that it is the next performance line item. It is that the right-sized investment is one that treats this the way a serious marketing team treats upper-funnel work: measured in brand searches, in direct arrivals, in share-of-voice sampling across the LLMs, and in the quality of what the models say when they do mention you. Treated like a billboard. Not like a Google Ads campaign.
Position four: the volumes that justify this being somebody’s biggest line item are not yet in evidence
This is the position I would not have been able to publish in April. The evidence has arrived in the last six weeks. Marie Haynes’ October 2026 analysis, covering twelve months of Google Analytics 4 data across 100 websites and over 145 million sessions, reports ChatGPT referrals at 0.62% of Google Organic volume, rising to 0.73% once all LLMs are counted. Google still accounts for roughly 97% of the conversions in that dataset, which Fishkin, citing Haynes in his Boondoggle post, calls roughly 27 times all LLMs combined
.
Fishkin’s post also pulls in a 95-site study from SEOWorks UK, which found organic search sending about 39 times more traffic than all LLMs, and Kevin Indig’s 12-site analysis, which could not detect a reliable link between AI visibility and branded or direct traffic. BrightEdge’s own September 2025 figure, from the Yu release, lands in the same place: less than 1%. The volumes are small enough that a serious buyer should not be asked to restructure a search budget around them.
Fishkin’s own summary line, from the same post, is the one to carry into a board meeting: Should companies restructure their budgets around appearing in AI tools? Probably not.
That is not an argument for doing nothing. It is an argument for right-sizing. If AI visibility is 1% of your referral traffic and three times out of four it converts worse than organic, you still want to be in the citation set, because being absent is cheap to notice and expensive to live with. You do not want it to be your biggest budget line.
Position five: the honest tools name their blind spots on the page, and that is the thing to buy on
Google’s Search Console added a Generative AI performance report in June 2026, covering impressions in AI Overviews, AI Mode and Discover’s generative features. The report ships impressions and nothing else: no clicks, no click-through rate, no queries, no average position. The platform owner knows exactly what you want it to tell you and has shipped the one metric it is sure of. The silence on the rest is the data point.
Cerretani and Gray close the same loop from the agency side. The measurement product the WordPress VIP series is nominally supposed to sell at the end is quietly described, in Cerretani’s own words, as something that measures humans not crawlers. Gray calls out correlation and not causation
while he is on screen with a slide about his firm’s work. These are the people worth hiring. The ones who do not say any of this on camera are the ones to be wary of.
What this looks like inside a real WooCommerce estate on Monday morning
So, concretely. If you run a mid-market WooCommerce or enterprise WordPress store and you want to measure AI visibility well enough to defend the budget to a CFO, here is the stack that we would build, with rough effort against each line.
- Server-log reading for AI crawler traffic. This is the one measurement that does not run on JavaScript, so it does not have Cerretani’s problem. GPTBot, PerplexityBot, ClaudeBot, Google-Extended, Applebot-Extended and the rest leave real lines in a web server’s access log. A WooCommerce estate on managed hosting usually keeps raw logs for 30 to 90 days. A one-off pass turns up two things worth knowing: whether the crawlers are reaching your product pages at all, and whether any of them is being accidentally blocked at the firewall or the WAF. Cerretani reported one enterprise client whose compliance plugin was denying roughly 15% of ChatGPT’s requests. The pass itself is a day of work and the fix, where there is one, is usually a configuration change.
- GA4 referrer segmentation. Add a channel group for AI referrers (chat.openai.com, perplexity.ai, claude.ai, gemini.google.com and the rest), separate from Direct. Haynes’ study is the published base rate for what you should expect to see: a fraction of a percent of organic volume, converting at a mixed rate. A WooCommerce install can carry this in a few hours of GA4 configuration, and the first useful chart falls out at the ninety-day mark.
- Search Console’s AI Overviews report, read as impressions. Half a day to turn it on where you have access, and read it the way Google shipped it: as upper-funnel reach, not performance. Changes in that number tell you whether your presence in AI answers is moving. They do not tell you whether anyone clicked, and the report does not pretend otherwise.
- Branded search as the trailing indicator. This is Gray’s billboard argument instrumented. Pull branded-query volume from Search Console and direct arrivals from GA4, run them against the AI presence trend, and expect to see a lagged correlation at best. The honest reading, which is Gray’s own, is that the correlation is suggestive and not conclusive. We ship it with a caveat on the dashboard rather than without one.
- Appearance-rate sampling across LLMs. The Fishkin methodology, scaled down. Fifteen to twenty prompts that a real buyer would plausibly use to look for you, run twenty to thirty times each per LLM, every two to four weeks. A junior analyst and a spreadsheet get you a usable baseline inside two weeks. The number you track is the share of runs in which your brand appears in the answer, by LLM and by prompt. We would not pay a vendor for this until the methodology inside the vendor is as explicit as what Fishkin and Gumshoe published for free.
- Governance: one named owner across the whole stack. The owner is probably marketing. In session two, Cerretani reported that only 5% of audience companies had a clear owner for this work. If that is still the case inside your organisation, the single most leveraged thing on this list is naming one.
Rough cost on a WooCommerce estate we would configure: a two to three week engagement for the first baseline, including the log pass, the GA4 and Search Console configuration, and the first appearance-rate sweep. Then roughly a day a month to re-run the sweep and update the dashboard. That is a brand-channel budget line, not a performance-channel budget line, and it is the right order of magnitude for a channel delivering under 1% of referral traffic today and quite possibly more tomorrow.
The piece nobody has solved, and the reason this list will change again next quarter
None of the people above claims to have closed the gap between appearance and outcome. Gray says so on camera and sells the work anyway. Fishkin says so in print and recommends measuring as a brand channel. Yu says the problem is being solved and then shows figures that suggest the problem is smaller than the solving suggests. Cerretani says the dominant tooling, including hers, is looking in the wrong place. The honest state of the field in late October 2026 is that we have better questions than we did in April, and better numbers for sizing them, and the measurement that proves business outcome from AI visibility has not been published by anyone yet.
So the recommendation that follows from this symposium is not a stack of software. It is a budget position and a cadence. Treat AI visibility as a brand channel, size the investment to the volume it is actually producing, instrument the upper-funnel surrogate metrics that do not depend on client-side JavaScript, and sample appearance rate yourself rather than buying it. Revisit the sizing when the volume moves, which it will. And hire the agency that will tell you about the correlation-not-causation problem up front rather than the one that will not.
If you want help building the stack above on a WooCommerce or enterprise WordPress install, that is work we do, and we will tell you the honest state of each metric as we ship it. That is the version of this piece that is useful to a buyer. The WP VIP cluster on the site carries the companion arguments on governance and ownership for teams at that scale.
Last modified: October 24, 2026
