On this page
A test is only worth running if its result would change what you do. That means deciding the threshold before the leads land, because a number decided afterwards is a number chosen to justify whatever you already believed.
Set these before the first lead arrives
- Sample size. Fifty is where averages start to settle; below that one week swings the whole read.
- The cadence, written down, identical to what your incumbent leads get.
- Source tagging in the CRM, so the comparison exists at all.
- The success threshold: "this works if we fund two" beats arguing about it later.
- Who is working them. One rep on new leads and a different one on old leads is not a test.
What to measure, in order
| Metric | What it tells you | Confounded by |
|---|---|---|
| Contact rate | Whether the numbers are real | Time to first dial |
| Qualification rate | Whether the screening held | Rep consistency |
| Statements received | Whether interest was genuine | How the ask was made |
| Submissions | Whether files were fundable | Packaging quality |
| Funded | The only number that pays | Everything above it |
| Cost per funded deal | The comparison figure | Sample size |
Read them top down. A poor funded rate with a strong contact rate is a floor problem; a poor contact rate is a lead problem. Most arguments between brokers and lead vendors are two people looking at different rows of that table.
Two vendors beats one
If the budget allows, split it across two vendors and tag both. A single vendor test tells you how one vendor performed on one fortnight of your floor; two tell you which vendor performed better on the same fortnight of the same floor, which is a far stronger signal for the same money.
Questions brokers ask
How many leads do you need to test a vendor properly?
About fifty. Below that a single good or bad week swings the entire result, and you learn what happened that fortnight rather than whether the leads work.
What is the most common mistake in a lead vendor test?
Working the new leads differently — faster, or with a better rep — than the incumbent’s. The test then measures attention rather than lead quality, and the advantage vanishes the month you switch over.
What should you measure in a lead test?
Contact rate, qualification rate, statements received, submissions, funded deals and cost per funded deal — read in that order, because a weak funded rate with a strong contact rate points at your process rather than the leads.
Should you test one vendor or two?
Two, if the budget allows, tagged separately. Two vendors on the same fortnight of the same floor is a far stronger comparison than one vendor against your memory of the last one.
When should you decide what counts as success?
Before the leads arrive. A threshold set afterwards is a threshold chosen to confirm what you already suspected, and it makes the whole exercise decorative.
Alex Makowski
Founder, Infinite Bookings
Runs the lead generation operation behind Infinite Bookings — paid traffic, the funding application, and the delivery pipeline that puts records into brokers’ CRMs.
Reachable directly at alex@infinitebookings.com or 732-609-7182.