playwright-score
A deterministic, AI-free quality score for Playwright specs. Lint anti-patterns, get a 0–100 grade, and gate CI—especially for AI-generated tests.
Built and maintained by QA Guardian. Open methodology, one standard profile — community best practices, not house rules.
Install & run
# Install npm install -D @qaguardian/playwright-score # Score a suite (scoped package binary) npx -p @qaguardian/playwright-score playwright-score ./tests --profile standard --threshold 80 # JSON output for CI / tooling npx playwright-score ./flow.spec.ts --threshold 80 --format json
Exit codes: 0 pass · 1 below threshold · 2 tool error. Formats: text, json, markdown, sarif.
Why teams use it
One number, frozen formula
The scoring model is versioned (v3) and documented: the weighted share of your tests that are clean. Same inputs always produce the same score—safe for PR gates and AI regen loops.
Built on community rules
Uses eslint-plugin-playwright for real best practices, then adds suite metrics: locator mix, empty expects, structure smells.
Made for AI-generated tests
We dogfood it on QA Guardian codegen. Score → findings → repair → rescore. Rules grade; models write.
What gets scored
Dimensions combine into a single 0–100 score with letter grades A–F. Details live in the open METHODOLOGY.md.
Playwright hygiene
No hard waits, force clicks, networkidle, missing awaits, element handles.
Assertions
Tests must assert; prefer web-first expect patterns.
Locators
Ratio of getByRole/Label/TestId/Text vs raw page.locator() (statistical).
Structure
Focused/skipped tests, oversized files, describe shape.
Tested against real code, not just our own fixtures
17 public Playwright suites, scored with the published package against each project's actual source — chosen to include both well-known, heavily-engineered platforms and smaller, less mature projects. Not curated to look good; 5 of 17 fail the default threshold, for real, verifiable reasons.
17
suites scored
1,230
spec files
5,654
tests covered
12 / 17
passed threshold
| Repo (source scanned) | Score | Grade | Result | Files | Tests |
|---|---|---|---|---|---|
| Playwright (own TodoMVC example) | 98/100 | A | PASS | 24 | 24 |
| Storybook | 95/100 | A | PASS | 7 | 25 |
| n8n | 95/100 | A | PASS | 262 | 1004 |
| freeCodeCamp | 94/100 | A | PASS | 85 | 299 |
| dub | 94/100 | A | PASS | 22 | 224 |
| Documenso | 93/100 | A | PASS | 126 | 1109 |
| Supabase | 91/100 | A | PASS | 31 | 254 |
| PostHog | 88/100 | B | PASS | 43 | 119 |
| Grafana | 87/100 | B | PASS | 212 | 667 |
| sencho | 85/100 | B | PASS | 26 | 148 |
| Mattermost | 83/100 | B | PASS | 291 | 1034 |
| novu | 82/100 | B | PASS | 2 | 2 |
| Immich | 78/100 | C | FAIL | 13 | 43 |
| cal.com | 73/100 | C | FAIL | 53 | 253 |
| openplayerjs | 73/100 | C | FAIL | 8 | 77 |
| livecodes | 70/100 | C | FAIL | 14 | 234 |
| TheCyberHub | 56/100 | F | FAIL | 11 | 138 |
Repo names link to the exact source scanned. Full methodology, findings breakdown, and a re-runnable script to reproduce every number here live in VALIDATION.md on GitHub.
Free scorer. Full coverage when you want it.
playwright-score grades the code you already have. QA Guardian maps critical journeys, generates maintainable Playwright, and keeps it green with human-in-the-loop review—using the same quality bar we open-sourced.
- You own standard Playwright TypeScript—export anytime
- AI drafts + senior engineers verify
- Parallel cloud runner, not a brittle Selenium grid
# example output
Playwright Spec Score v3 · profile=standard Score: 86/100 (B) · PASS (threshold 80) Dimensions: playwrightHygiene: 90 assertions: 100 locators: 70 structure: 95 Findings: [warning] playwright/no-raw-locators …
FAQ
Is the score AI-based?
No. The score is fully deterministic and AI-free. Same files always produce the same score under the versioned scoring model (v3). AI can generate or repair code using the findings; rules grade the code.
How is this different from eslint-plugin-playwright?
We run eslint-plugin-playwright under the hood for community best practices, then add a versioned 0–100 score, suite-level metrics (locator ratio, assertion-delegation tracing, Page Object Model import resolution), and CI exit codes.
Does it grade real-world code accurately, or just clean examples?
We validate it against real, public Playwright suites on an ongoing basis — not just our own fixtures. See the real-world results table on this page, or re-run the validation yourself from the GitHub repo.
Can I use this in CI?
Yes. playwright-score exits 0 when the score meets your threshold and 1 when it fails. JSON, markdown, and SARIF outputs are supported.
Who maintains it?
QA Guardian (qaguardian.com). We dogfood it on AI-generated Playwright flows in our managed QA platform. The core scorer is MIT open source.
Score your suite. Own your Playwright.
Star the repo, drop it in CI, or let QA Guardian build and maintain journey coverage for you.