Measuring advertising effect properly

Every advertising platform reports conversions and every one of them reports too many, because each counts contacts it touched rather than sales it caused.

The difference that matters

Attribution answers: which contacts preceded the purchase? Causation answers: would the purchase have happened without the advertising?

Those questions have different answers, and the gap is often large. Remarketing, brand search advertising and affiliate voucher partners are the three channels where the gap is routinely biggest, because all three touch people who had already decided.

The three tests worth running

1. The switch-off test. Turn one channel off completely for four to six weeks and watch total demand, not the channel's own reporting.

Cost: whatever the channel would have produced. Value: the only unambiguous answer available.

2. The regional holdout. Run the campaign in five districts and not in three comparable ones. Compare enquiry rates. Cleaner than a switch-off because seasonality affects both groups equally.

3. The staggered start. Launch in one region, then a second three weeks later, then a third. Each launch produces a visible step if the channel works.

Which channels can be tested

Almost everything is testable. The reason tests are rare is not difficulty but that nobody wants the answer.

The measurements to keep running

Alongside tests, three continuous figures give an honest picture.

1. Total enquiries per week, regardless of source, plotted over two years.
2. Cost per enquiry across all spend, total advertising cost divided by total enquiries. Crude and unfoolable.
3. The direct question. Ask every enquirer how they came to you and record the answer verbatim.

The third contradicts platform data often, and where they disagree the customer is usually closer to the truth about the first contact and the platform closer about the last.

The honest report

A report that claims precision it does not have is worse than one that admits uncertainty. State what was measured, what was estimated, and what was assumed.

The three-column format works: what we know, what we believe, what we are testing next.