How to assess evidence in connected TV advertising reviews

What to take away

A connected TV advertising review methodology sets out the scope of a review, separates vendor claims from independent evidence and editorial judgement, and explains how claims are checked before publication.

Reviews should explain the basis for any scores, with each score tracing to a cited source or a dated observation.
Vendor claims are labelled as claims, never presented as verified performance.
Independent evidence means regulator research, official statistics, published rate cards or platform documentation, not a sales deck.
Editorial judgement is allowed and flagged as judgement.
No review claims hands-on testing unless the reviewer ran the campaign, and that is rare.

The sections below set out the principles used to scope a review, assess evidence and check it before publication.

Define what a review covers and what it excludes

Every review opens with a scope statement. It names the vendors assessed, the period covered, and the buyer type the scores suit. A platform that suits a £5,000 a month advertiser may score badly for a broadcaster with its own ad server.

Review scope statement

Name vendors assessed
State period covered
State buyer type scores suit
List exclusions plainly
Disclose pricing source

State the exclusions plainly. If a review did not access a vendor's demand-side platform, say so. If pricing came from a published rate card rather than a negotiated quote, say that too. Buyers forgive gaps that are disclosed.

The wider structure is set out in how this site reviews and compares connected TV advertising, which explains how criteria are chosen before any vendor is named.

Separate claims, evidence and judgement

Three layers sit in every review, and mixing them is the commonest failure.

Vendor claims are what a supplier says about itself: reach, completion rates, brand lift. Label them and date them.

Independent evidence is checkable. Regulator research on audience responses to video advertising is one example. The research published by the ASA and CAP is a fair starting point when a review touches audience attitudes to online and video ads.

Editorial judgement is the reviewer's own view, formed from the evidence. Keep it in its own field so readers can discount it.

Match the evidence to the claim being tested

Different claims need different proof. A claim about streaming's share of television viewing should be checked against a measurement panel rather than a press release. Nielsen's The Gauge publishes that split between streaming and linear viewing.

A claim about format or buying route is best checked against the platform's own documentation or an industry body. Thinkbox's overview of the different ways to use TV advertising covers BVOD and sponsorship routes, which is useful when a review compares buying models.

Where no independent source exists, the claim is scored as unverified.

What should influence score weighting?

Weight by consequence, not by ease of measurement. Pricing transparency, measurement quality and contract terms carry more weight than interface polish, because they change the outcome of a campaign.

What should disqualify a review?

Undisclosed commercial relationships, scores with no stated source, and any claim of hands-on testing that did not happen. Any of the three should stop publication.

Use a checklist before publishing

Scope statement names vendors, period and buyer type.
Every score cites a dated source or a labelled judgement.
Vendor claims are marked as claims throughout.
Pricing figures are quoted from a named source or labelled as illustrative, for example a team paying £400 a month.
No hands-on testing is claimed unless it occurred.
Conflicts of interest are disclosed in the review itself.
Scores are reproducible by a second reviewer using the same notes.

Buyers weighing agencies rather than platforms can see the same rules applied in connected TV advertising agency reviews in England. There, service quality and reporting practice are scored instead of ad-server features.

Common questions

These answers cover evidence handling and the limits of review claims.

Does a review need to test the platform?

No. Most buying decisions rest on documented capability, pricing structure and measurement, which can be assessed from published sources. Claim testing only when it happened.

How should unverifiable claims be handled?

Record them, attribute them to the vendor, and exclude them from the score. A claim that cannot be checked should never lift a rating.

Can one reviewer's judgement be trusted?

Only if it is labelled and the evidence sits beside it. A reader who disagrees with the judgement can still use the evidence, which is the point of separating them.