playwright-score
A deterministic, AI-free quality score for Playwright specs. Lint anti-patterns, get a 0–100 grade, and gate CI—especially for AI-generated tests.
Built and maintained by QA Guardian. Open methodology. Community standard profile; optional house-style guardian profile.
Install & run
# Install npm install -D @qaguardian/playwright-score # Score a suite (scoped package binary) npx -p @qaguardian/playwright-score playwright-score ./tests --profile standard --threshold 80 # Guardian / codegen profile npx playwright-score ./flow.spec.ts --profile guardian --threshold 75 --format json
Exit codes: 0 pass · 1 below threshold · 2 tool error. Formats: text, json, markdown, sarif.
Why teams use it
One number, frozen formula
sqs-v1 is versioned and documented. Same inputs always produce the same score—safe for PR gates and AI regen loops.
Built on community rules
Uses eslint-plugin-playwright for real best practices, then adds suite metrics: locator mix, empty expects, structure smells.
Made for AI-generated tests
We dogfood it on QA Guardian codegen. Score → findings → repair → rescore. Rules grade; models write.
What gets scored
Dimensions combine into a single 0–100 score with letter grades A–F. Details live in the open METHODOLOGY.md.
Playwright hygiene
No hard waits, force clicks, networkidle, missing awaits, element handles.
Assertions
Tests must assert; prefer web-first expect patterns.
Locators
Ratio of getByRole/Label/TestId/Text vs raw page.locator() (statistical).
Structure
Focused/skipped tests, oversized files, describe shape.
Guardian conventions (optional)
House rules: no Date.now() IDs, no waitForLoadState, test.step guidance, and more.
Free scorer. Full coverage when you want it.
playwright-score grades the code you already have. QA Guardian maps critical journeys, generates maintainable Playwright, and keeps it green with human-in-the-loop review—using the same quality bar we open-sourced.
- You own standard Playwright TypeScript—export anytime
- AI drafts + senior engineers verify
- Parallel cloud runner, not a brittle Selenium grid
# example output
Playwright Spec Score sqs-v1 · profile=standard Score: 86/100 (B) · PASS (threshold 80) Dimensions: playwrightHygiene: 90 assertions: 100 locators: 70 structure: 95 Findings: [warning] playwright/no-raw-locators …
FAQ
Is the score AI-based?
No. The score is fully deterministic and AI-free. Same files always produce the same score under sqs-v1. AI can generate or repair code using the findings; rules grade the code.
How is this different from eslint-plugin-playwright?
We run eslint-plugin-playwright under the hood for community best practices, then add a versioned 0–100 score, suite metrics (locator mix, empty expects), CI exit codes, and an optional guardian profile for house conventions.
What is the guardian profile?
An optional profile for teams (including QA Guardian codegen) that enforce extra conventions like banning Date.now() for IDs and waitForLoadState. It is clearly labeled house style—not official Playwright doctrine. Default for public use is the standard profile.
Can I use this in CI?
Yes. playwright-score exits 0 when the score meets your threshold and 1 when it fails. JSON, markdown, and SARIF outputs are supported.
Who maintains it?
QA Guardian (qaguardian.com). We dogfood it on AI-generated Playwright flows in our managed QA platform. The core scorer is MIT open source.
Score your suite. Own your Playwright.
Star the repo, drop it in CI, or let QA Guardian build and maintain journey coverage for you.