Methodology

How we measure AI visibility

Every AI Visibility Score is produced by a transparent, reproducible process. This page explains exactly how.

The query panel

Each month, All In Wales runs a fixed set of real-world customer queries for every area and category combination in Wales. Queries are written to reflect how actual customers phrase requests to AI assistants — not SEO keyword lists. The same queries run every month, so scores are comparable over time.

Query intent types (8 per area × category)

Best ofWhat are the best restaurants in Tenby?
Visit intentI'm visiting Hay-on-Wye next weekend. Where should I eat?
AttributeDog-friendly cafés near Aberystwyth?
FamilyFamily-friendly pubs near Llandudno?
BudgetWhere can I eat in Cardiff on a budget?
Coastal viewGood restaurant with a sea view in Mumbles?
Day planPlan a day in Conwy: where to eat, what to see.
Opening hoursCafé in Portmeirion open on a Sunday?

Coastal-view queries only run for coastal areas. Category-agnostic queries (e.g. "Plan a day in…") run once per area regardless of category.

AI engines and weighting

Queries are sent to four AI engines simultaneously. Each engine has a weight that reflects its estimated share of real consumer AI search usage. Engine weights are reviewed quarterly.

Engine	Model	Weight
ChatGPT (OpenAI)	GPT-4o mini with web search	55%
Gemini (Google)	Gemini 2.0 Flash	25%
Claude (Anthropic)	Claude Haiku	10%
Perplexity Sonar	Llama 3.1 Sonar Small	10%

Mention extraction

After each engine returns its response, a second AI pass extracts the business names mentioned. Extracted names are fuzzy-matched against the All In Wales businesses database using normalised Welsh text (accent-insensitive, case-insensitive). A match is recorded as a mention with the position (first, second, third…) in which the business appeared.

Position matters: being named first in an AI answer carries more weight than being named fifth. The scoring model discounts later positions using a logarithmic decay function.

Scoring model

Each mention generates a score contribution:

contribution = engine_weight × (1 / log₂(position + 1))

Contributions from all mentions across all engines and queries are summed, then normalised to a 0–100 scale relative to the best-performing business in the same area × category set. A score of 100 means the top-ranked business for that period.

Score interpretation

80–100
High visibility
Named in most AI answers for your category and location.
50–79
Moderate visibility
Named in some answers; gaps in specific engines or query types.
20–49
Low visibility
Rarely named — missing structured data or underrepresented location.
0–19
Not visible
Not found in AI answers for the current panel period.

Limitations

–AI engine responses vary between runs. A single panel period is a snapshot, not a permanent record.
–Businesses not yet in the All In Wales database cannot be scored even if AI names them.
–Engine weights are estimates. Actual consumer AI usage varies by query type and region.
–Welsh-language queries are not yet included in the panel. This will change in a future release.