Fix the variables you control before you start: one dial cadence, one script, tagged sources, a defined success threshold written down in advance, and a sample big enough that a single week does not swing it. Most vendor tests fail as experiments because the leads were worked differently from the incumbent’s, which measures the floor rather than the feed.
A test is only worth running if its result would change what you do. That means deciding the threshold before the leads land, because a number decided afterwards is a number chosen to justify whatever you already believed.
| Metric | What it tells you | Confounded by |
|---|---|---|
| Contact rate | Whether the numbers are real | Time to first dial |
| Qualification rate | Whether the screening held | Rep consistency |
| Statements received | Whether interest was genuine | How the ask was made |
| Submissions | Whether files were fundable | Packaging quality |
| Funded | The only number that pays | Everything above it |
| Cost per funded deal | The comparison figure | Sample size |
Read them top down. A poor funded rate with a strong contact rate is a floor problem; a poor contact rate is a feed problem. Most arguments between brokers and lead vendors are two people looking at different rows of that table.
If the budget allows, split it across two vendors and tag both. A single vendor test tells you how one feed performed on one fortnight of your floor; two tell you which feed performed better on the same fortnight of the same floor, which is a far stronger signal for the same money.
About fifty. Below that a single good or bad week swings the entire result, and you learn what happened that fortnight rather than whether the feed works.
Working the new leads differently — faster, or with a better rep — than the incumbent’s. The test then measures attention rather than lead quality, and the advantage vanishes the month you switch over.
Contact rate, qualification rate, statements received, submissions, funded deals and cost per funded deal — read in that order, because a weak funded rate with a strong contact rate points at your process rather than the feed.
Two, if the budget allows, tagged separately. Two vendors on the same fortnight of the same floor is a far stronger comparison than one vendor against your memory of the last one.
Before the leads arrive. A threshold set afterwards is a threshold chosen to confirm what you already suspected, and it makes the whole exercise decorative.
We will tell you straight up if we cannot help you. No commission deals, no free trials, no chasing you for three weeks.