Flow-Based Pricing: How Per-Test Billing Broke QA (And How to Fix It)

You pay your QA vendor for 100 tests per month. Next month, you have 120 tests. The month after, 145.

Your coverage dashboard is green. Your VP of Engineering sees the numbers going up and feels good about the investment. Everyone ships with confidence.

Here's what nobody is asking: do the tests actually prove your application works?

Probably not. And the reason isn't incompetence — it's that the pricing model itself rewards the wrong behavior.

The Per-Test Trap

Suppose your checkout workflow has four steps: browse products, add to cart, enter payment details, confirm the order.

A thoughtful engineer writes one test: navigate to the product page, click through the full flow, assert the confirmation page renders with the correct order number. One script. Complete coverage of the most important thing your application does.

But the vendor bills per test. And somehow, "checkout coverage" from that vendor arrives looking like this:

• test_001: verify the product page loads
• test_002: verify the "Add to Cart" button exists
• test_003: verify the cart count increments
• test_004: verify the cart renders the correct product name
• test_005: verify the cart renders the correct price
• test_006: verify the checkout page loads from the cart
• test_007: verify the address form renders
• test_008: verify the address form validates required fields
• test_009: verify shipping options appear
• test_010: verify the credit card form renders
• test_011: verify the "Place Order" button is clickable
• test_012: verify the success page renders

Twelve tests. Billed twelve times. Twelve green checkmarks every CI run.

But do you know whether checkout actually works?

You don't. Because not a single one of those tests proves that a customer can complete a purchase. They all test rendering states. The only test that proves checkout works is one that actually performs checkout from start to finish — and that's not what per-test pricing incentivizes anyone to write.

This isn't malice. The vendor isn't trying to scam you. They're responding rationally to the incentive structure you gave them. If you pay per tomato seed, you'll get a lot of seeds and no tomatoes.

The Invoice Is the Smallest Problem

The billing problem is irritating but manageable. You can negotiate. You can audit. You can cap test counts.

The real damage is what you're left maintaining.

Twelve separate test files. Twelve points of failure for every UI change. When your designer tweaks the checkout layout, how many of those tests break? Probably most of them. You've multiplied your maintenance surface by twelve to achieve worse coverage than one well-written flow.

And those twelve tests aren't just a maintenance burden — they're twelve browser context initializations, twelve teardowns, and twelve times the CI minutes. You're paying in dollars, compute, and developer patience. The invoice is the smallest line item on the receipt.

CI run times inflate. Developers start ignoring failures because "it's probably just the button test again." The test suite that was supposed to build confidence has trained your team that test failures are noise.

Per-test pricing doesn't just produce bloated suites — it devalues the entire concept of automated testing in your organization.

The Structural Fix: Pay for Journeys, Not Clicks

There is an alternative, and it's simpler than you'd think.

One billable unit = one user journey from entry to measurable outcome. A checkout flow that covers product selection through order confirmation is one unit. The vendor can't atomize it into twelve tests because twelve fragments don't add up to the journey. The pricing model makes fragmentation impossible.

Incentives realign completely.

When you charge per journey, the vendor's goal is to cover each journey as efficiently as possible — one script, complete coverage, minimal maintenance footprint. Writing twelve rendering checks for checkout doesn't move the needle on revenue. The only rational move is to write the one test that actually matters.

This isn't about trusting your vendor to "do the right thing." It's about designing the contract so that doing the right thing is the easiest way to make money.

Why Nobody Talks About This

Per-test pricing has been the industry default for decades. It's how tools were priced. It's how agencies billed. Nobody stopped to ask if it made sense — it was just how QA billing worked.

But per-test pricing exists because it's easy to measure, not because it's the right unit of value. Nobody wakes up in the morning hoping their test count went up. They wake up hoping their checkout flow didn't break. The unit of measurement should match the unit of value.

The Three-Question Audit

You can evaluate whether your QA vendor's pricing is aligned with your interests in about thirty seconds.

Ask these three questions:

1. Do you charge per test, or per user journey covered? If the answer involves test count, you're paying for volume.
2. If I request coverage for a new user flow, how many billable units is that? If the answer is "it depends on how many tests we write," your cost is unpredictable by design.
3. Does your pricing model reward you for writing fewer tests? If the answer is no, the incentives run opposite to what you want.

A vendor who charges per journey can answer all three in a single sentence. A vendor who charges per test will need a spreadsheet.

The Takeaway

This isn't just a QA problem. Any service billed per unit has this tension — the vendor's revenue grows with output regardless of whether the output delivers value. Agency retainers with hourly billing. Content marketing priced per article. DevOps consulting billed per ticket closed.

The difference is that QA hasn't had the conversation yet. Marketing debated "per article vs per outcome" a decade ago. DevOps is actively debating it. QA is still defaulting to per-test because nobody has pointed out how broken it is.

Fix the incentive. Charge for what matters. Everything else follows.