One answer is a snapshot, not a measurement
A website may appear in a ChatGPT response, then disappear when the same question is asked again the next day. That does not necessarily mean its positions have fallen. In an experiment by SparkToro and Gumshoe, 600 volunteers ran 12 prompts through ChatGPT, Claude and Google AI, and the list of brands varied significantly between runs.
The models did not return a fixed set of links for each question. They produced a probabilistic set that changed with every repetition.
Measure visibility as a probability
Visibility in ChatGPT is the probability that a website will appear in an answer to a specific query, so a single check is not a valid measurement. One response is one observation of a random process, not a stable ranking result as in classical SEO.
Reports should state how many times each question was repeated and show the result range. Without both, a visibility figure can give a misleading impression of consistency. A series of one or two runs provides too few observations and a high variance.
The metric should be the share of answers containing the website across repeated runs of the same prompt. Tracking should also cover ChatGPT, Claude and Google AI separately. The experiment found variation between models was no smaller than variation between repeated runs within one model, making a combined 'presence in AI search' metric unreliable.
For teams monitoring a brand or domain in AI responses, this means recording repeated runs of each prompt, calculating the share of mentions and reporting the margin of error. A before-and-after comparison following a content or page change cannot be separated from ordinary model variation without that data.
This approach differs from conventional SEO tracking, where a site's position for a query is generally more stable between checks. A one-off snapshot from Serpstat or Ahrefs can still provide a useful picture. ChatGPT requires series-based monitoring because the same wording can produce different sources on every run.
