How to evaluate agency performance beyond vanity metrics

To evaluate a paid-media agency beyond vanity metrics, judge it on contribution to profit rather than platform ROAS, clicks or impressions. Agree a true-profit definition up front, give the agency the cost data it needs, and review campaigns on profit per unit of spend, incrementality evidence, creative pipeline health and the quality of its reasoning.

Why do agency reports default to vanity metrics?

Not because agencies are cynical. They report what they can see and what they control. Platform ROAS, CTR, CPM and conversion counts arrive automatically from Meta and Google. Returns data, payment fees, cost of goods and discount leakage sit in your systems, often behind a finance team that has never been asked to share them. So the agency optimises the number it has, and both sides then argue about whether that number means anything.

That is the trap described by one of the clearest tensions in SOMIN's listening on paid media: brands optimise ROAS and CPA while hidden costs go unexamined, and agencies are rewarded for the metrics that flatter rather than the ones that pay. Fixing it is a shared job. If you want an agency judged on profit, you have to hand over the inputs.

What should an agency scorecard measure?

A good scorecard has five lines. Each one should be agreed in writing at the start of the quarter.

1. True-profit ROAS by campaign: gross profit on kept orders divided by spend, using the method in our true-profit ROAS guide.
2. Incrementality evidence: holdouts, geo tests or at least a blended MER trend that shows ads are adding sales, not just claiming them.
3. Budget discipline: how quickly spend moved away from campaigns past their profitable point.
4. Creative pipeline health: number of new concepts tested, how they were chosen, and how quickly tired creatives were replaced.
5. Quality of reasoning: whether recommendations come with a clear why, and whether the agency flagged problems before you found them.

Notice what is absent: impressions, reach and CTR as goals. They are useful diagnostics. They are not outcomes.

Questions to ask in your next agency review

• Which campaign would you cut first if budget fell by a fifth, and why?
• Which of our products have the highest return rates, and how does that change your bidding?
• How much of the revenue you reported is also claimed by our other channel?
• How did you choose which creatives to test this month?
• What did you stop doing this quarter, and what did it save?

A strong agency enjoys these questions. They are the questions it would ask itself if it had your data.

Is it fair to judge an agency on profit?

Only if the agency can influence it and see it. That means sharing returns, margins and fees, and agreeing how attribution overlap will be handled. It also means accepting that true-profit numbers are smaller than platform numbers, and that a campaign moving from 1.1 to 1.3 true-profit ROAS may be a bigger achievement than one moving from 4.0 to 5.0 platform ROAS.

Agencies that already work this way tend to lean on audience evidence rather than gut feel. The Havas Media case study on somin.ai and the wider SOMIN for agencies overview show how agency teams use SOMIN, an AI audience-research platform, to back recommendations with real audience signals. If you are weighing an agentic model, where AI agents run research, creative and optimisation and humans own judgment, AgentC in our network is built around exactly that split.

A simple monthly ritual

Once a month, put three views side by side: platform ROAS, true-profit ROAS and blended MER. Ask the agency to explain any campaign where those three disagree. Disagreement is where the learning is. If the agency's explanation is "attribution is complicated", push for a test. If the explanation is specific, say high returns in one category, or overlap with branded search, you have a partner worth keeping.

These conversations are getting more common among Singapore marketers. They came up repeatedly in community discussions around Marketing Mondays, where the weekly format of one real signal decoded into a clear takeaway suits exactly this kind of agency question.

Definitions

• Vanity metric: a number that rises easily and feels like progress but does not map to profit, such as impressions or platform-reported revenue on its own.
• Incrementality: sales that would not have happened without the ads.
• Attribution overlap: the same order claimed by more than one platform.

What does a good agency report look like?

A strong monthly report fits on two pages. Page one shows the agreed profit view: true-profit ROAS by campaign, blended MER, and the budget moves made during the month with the reason for each. Page two covers creative: what was tested, why those concepts were chosen, what was rested, and what is queued next. Impressions and CTR appear only as supporting detail where they explain a change. If your current report runs to thirty slides and none of them shows profit, ask for this format instead. Most good agencies will welcome the clarity.

Do not ask your agency for better numbers. Ask for more honest ones, and give them the data to produce them.

How FMedia helps

FMedia gives brand and agency the same profit view. It nets returns, fees, shipping and discounts from attributed revenue, shows true-profit ROAS next to platform ROAS for every campaign, and keeps a simple agency scorecard of budget moves and creative refresh cadence. Because recommendations show their reasoning, reviews become a discussion of decisions rather than a negotiation about whose dashboard is right.

That changes the tone of renewals. Good agencies get credit for the profit they actually create, and weak arguments built on vanity numbers stop working, on both sides of the table.

Frequently asked questions

What are vanity metrics in paid media?

Numbers that look like progress but do not map to profit: impressions, reach, CTR and platform-reported revenue in isolation. They are useful diagnostics, not targets.

What data should I share with my agency?

Return rates by product, cost of goods or margin bands, payment and shipping costs, and discount usage. Without these, an agency can only optimise platform numbers.

How often should I review agency performance?

A light weekly check of spend and alerts, a monthly profit review comparing platform ROAS, true-profit ROAS and MER, and a quarterly scorecard tied to renewal decisions.