You probably do not need an AI visibility tool yet: many tools measure positions, run prompts too few times and use made-up questions. Measure yourself first and only buy a tool once the supplier answers your questions about method and calculation properly.
You probably do not need an AI visibility tool yet. Many tools measure positions, run prompts too few times or use made-up questions. On top of that, AI is still a small source of traffic, so most businesses gain more from other work.
That does not mean measuring is pointless. It means you first need to know exactly what a tool measures. Below we list the pitfalls, the right moment and the questions to put to the supplier.
Four pitfalls of AI tracking tools
The first pitfall is the position. AI answers vary so much that a rank position says little, see why a fixed position in ChatGPT does not exist.
In the Office Hours of March 2026 it was put briefly: "Rankings? Probably nonsense." So rank positions are most likely nonsense.
The second pitfall is too little repetition. For a measurement with a margin of roughly plus or minus one percentage point, you need to run a set of prompts about twenty times, according to SparkToro (March 2026). A tool that asks each prompt once a week mainly measures chance.
The third pitfall is made-up prompts. When hundreds of people wrote a prompt for the same intent, the average semantic similarity was only 0.081 (SparkToro, February 2026). A tool with a short list of invented questions therefore misses how your customers really ask.
The fourth pitfall is the difference between API and app. In the GEO benchmark by Gold Lemon (September 2026) the ChatGPT app and ChatGPT via the API shared only 2 of their top 3. The app named 4.0 agencies per answer on average, the API 2.3.
Also pay attention to the tone of a mention. In the same Gold Lemon benchmark of September 2026, 52.6% of mentions were a recommendation, 47.0% neutral and 0.4% critical. A tool that only counts names misses that difference.
The API is the technical access for software, the app is what your customer actually uses. A tool that queries via the API therefore does not necessarily see what your customer sees in the app. So always ask which of the two a tool measures.
I would really urge you to stop throwing your money at AI tracking products unless they have great answers to these things and research back findings around this.
Put simply: stop spending money on AI tracking products unless they have good answers on these points. They should also back up their findings with research.
How big is AI as a channel, really?
AI is growing fast, but still refers few visitors. AI tools send less than 1% of all traffic, Fishkin reported in June 2026 based on Similarweb data from the US. At the same time, more than 20% of Americans use an AI tool 10 times or more a month.
Google's AI Mode is also still small. From January to April 2026, 0.34% of searches went there (SparkToro, June 2026). It is growing fast, though: at I/O 2026 Google reported more than 1 billion monthly users.
The share of searches is small as well. In Fishkin's research into 41 large desktop sites in the US, Google had 73.7% of searches and AI tools 3.2%. Those figures cover the fourth quarter of 2025 (SparkToro, ZCM Field Notes, data Q4 2025).
That does not mean AI is unimportant. People are influenced in AI answers without you seeing it directly in your analytics. It does mean that an expensive tracking subscription is rarely the first priority for a small business.
The risk of overinvesting
In January 2026 Fishkin predicted that companies will massively overinvest in tracking AI mentions and sentiment. At the same time they will underinvest in almost every other form of marketing measurement. In his view, the market for AI tracking, excluding Google, will grow beyond 200 million dollars (SparkToro, January 2026).
That is money that does not go into the work itself. Appearing in AI answers is mainly linked to mentions outside your own site, such as on Reddit and YouTube (SparkToro, October 2025). More on that in how evidence outside your site gets you mentioned.
In the Office Hours of March 2026 Fishkin summed up his concern like this:
My worry was that marketers are getting fleeced because there is a lot of hype around AI.
He means that marketers are being fleeced because of the hype around AI. If you keep a cool head, you invest in the work first and only then in measurement.
When a tool does make sense
A tool can make sense once AI has become a noticeable channel for you. Think of a category in which customers often ask an AI for advice and where you are already working seriously on visibility. Then you want to follow trends that you can no longer keep up with by hand.
A tool is worth considering if:
- you already measure yourself and the manual tallying takes too much time;
- you want to follow several engines and many customer questions at once;
- you link the results to direct traffic and branded searches;
- the supplier reports percentages and shows the calculation.
Until then you can measure perfectly well yourself, with our method for measuring AI visibility. How to link it to your own figures is explained in branded searches as evidence of AI mentions.
Questions for the supplier
- How often do you run each prompt, and what margin of error goes with that?
- Where do the prompts come from: real search and customer data or made up?
- Do you measure via the app or via the API, and for which engine?
- Do you report a visibility percentage or a position?
- Can we inspect the raw answers and the calculation?
This advice comes straight from Fishkin, from the research of January 2026:
Before you spend a dime tracking AI visibility, make sure your provider answers the questions we've surfaced here and shows their math.
So do not spend a penny until the supplier answers these questions and shows the sums. A good supplier does not find these questions awkward, but obvious.
What you can do now
- First measure yourself for a month with a simple tally sheet per AI engine.
- Check in your own analytics whether direct traffic and searches for your brand name are rising.
- Send the supplier questions above to every provider you are considering.
- Spend the budget you save on mentions, reviews and content outside your own site.
Frequently asked questions
Do I need an AI visibility tool?
Probably not yet. For most businesses AI is still a small source of traffic and you can measure by hand. A tool becomes useful once manual measuring can no longer keep up. See also: Zero click: why website traffic is no longer a good KPI.
What does an AI tracking tool actually measure?
It asks AI engines questions and records whether you are mentioned. The quality depends on how often, with which questions and by which route it does that. See also: Getting mentioned in ChatGPT: the evidence lies outside your website.
Why are positions in an AI tracking tool unreliable?
AI answers change in names and order from one time to the next. A position therefore says little. A percentage across many repeats says more. See also: Measuring AI visibility.
How often should a tool run a prompt?
Often enough to average out chance, so certainly not once a week. Ask the supplier what margin of error goes with their number of repeats. See also: Best GEO agency: how to recognise a fair ranking.
What is the difference between measuring via the API and via the app?
The API is the technical access for software, the app is what your customer uses. The two do not always give the same names. So ask which of the two a tool measures. See also: Follow-up client calls.
Why are made-up prompts a problem?
People describe the same problem in very different ways. Prompts that do not come from real customer questions measure a situation your customer never experiences.
Can I measure AI visibility without a tool?
Yes. Ask your most important customer questions repeatedly in each engine, tally whether you are mentioned and work out a percentage. It takes time, but no subscription.
When is a tool worth the money?
When you want to follow many questions across several engines and manual measuring takes too much time. The tool must then report percentages and show its calculation.
Is spending money on AI tracking money down the drain?
Not necessarily. The risk is that you put too much into measuring and too little into the work that delivers visibility, such as mentions and reviews.
How do I know whether more AI visibility pays off?
Look at direct traffic and searches for your brand name in your own analytics. If they rise too, that is a sign that AI mentions are working.
Sources
- Office Hours: Can You Actually Track AI Visibility? (SparkToro) (March 2026)
- Why Everyone Prompts AI Differently (Amanda Natividad, SparkToro) (February 2026)
- GEO-benchmark oktober 2026 (Gold Lemon) (September 2026)
- In 2026, Less than One Third of Google Searches Still Send a Click (Rand Fishkin, SparkToro) (June 2026)
- If Search Captures Demand, Public Evidence Creates It (SparkToro, ZCM Field Notes) (data Q4 2025)
- The 5 Big Trends that Will Dominate Marketing in 2026 (Rand Fishkin, SparkToro) (January 2026)
- We've Learned More About How to Appear in AI Answers (Rand Fishkin, SparkToro) (October 2025)
- NEW Research: AIs are highly inconsistent when recommending brands or products (Rand Fishkin, SparkToro) (January 2026)