Open source · MIT · scoring model v4

playwright-score

Name: playwright-score
Author: QA Guardian

A deterministic, AI-free quality score for Playwright specs. Lint anti-patterns, get a 0–100 grade, and gate CI—especially for AI-generated tests.

Built and maintained by QA Guardian. Open methodology, one standard profile — community best practices, not house rules.

View on GitHub Methodology (model v4)Managed Playwright coverage

Install & run

# Install
npm install -D @qaguardian/playwright-score

# Score a suite (scoped package binary)
npx -p @qaguardian/playwright-score playwright-score ./tests --profile standard --threshold 80

# JSON output for CI / tooling
npx playwright-score ./flow.spec.ts --threshold 80 --format json

Exit codes: 0 pass · 1 below threshold · 2 tool error. Formats: text, json, markdown, sarif.

Why teams use it

One number, frozen formula

The scoring model is versioned (v4) and documented: the weighted share of your tests that are clean. Same inputs always produce the same score—safe for PR gates and AI regen loops.

Built on community rules

Uses eslint-plugin-playwright for real best practices, then adds suite metrics: locator mix, empty expects, structure smells.

Made for AI-generated tests

We dogfood it on QA Guardian codegen. Score → findings → repair → rescore. Rules grade; models write.

What gets scored

Dimensions combine into a single 0–100 score with letter grades A–F. Details live in the open METHODOLOGY.md.

Playwright hygiene

No hard waits, force clicks, networkidle, missing awaits, element handles.

Assertions

Tests must assert; prefer web-first expect patterns.

Locators

Ratio of getByRole/Label/TestId/Text vs raw page.locator() (statistical).

Structure

Focused/skipped tests, oversized files, describe shape.

Tested against real code, not just our own fixtures

100 public Playwright suites, scored with the published package against each project's actual source — chosen to include both well-known, heavily-engineered platforms and smaller, less mature projects. Not curated to look good; 35 of 100 fail the default threshold (80), for real, verifiable reasons.

Tested on 2026-09-24 · v2.1.0 · model v4

100

suites scored

7,281

spec files

39,059

tests covered

65 / 100

passed threshold

		Grade	Result	Files	Tests
pinterest/gestalt playwright/accessibility	100/100	A	PASS	173	173
TryGhost/Ghost e2e/tests	99/100	A	PASS	99	343
hashicorp/vault ui/e2e	98/100	A	PASS	32	51
microsoft/playwright examples/todomvc	98/100	A	PASS	24	24
plausible/analytics e2e	98/100	A	PASS	12	71
segmentio/analytics-next packages/browser-integration-tests/src	98/100	A	PASS	4	7
cloudflare/templates playwright-tests	97/100	A	PASS	41	63
glpi-project/glpi tests/e2e/specs	97/100	A	PASS	106	391
shopify/hydrogen e2e/specs	97/100	A	PASS	34	201
dubinc/dub apps/web/playwright	96/100	A	PASS	34	365
keycloak/keycloak js/apps/admin-ui/test	96/100	A	PASS	72	452
mozilla/fxa packages/functional-tests/tests	96/100	A	PASS	111	429
penpot/penpot frontend/playwright	96/100	A	PASS	64	506
temporalio/ui tests	96/100	A	PASS	52	236
xmlui-org/xmlui xmlui	96/100	A	PASS	406	6880
SigNoz/signoz tests/e2e	95/100	A	PASS	51	312
argos-ci/argos tests	95/100	A	PASS	21	89
baloise/design-system packages/core	95/100	A	PASS	15	192
nextcloud/server tests/playwright/e2e	95/100	A	PASS	101	395
storybookjs/storybook code/e2e-internal	95/100	A	PASS	7	26
Automattic/jetpack projects/plugins/jetpack/tests/e2e/specs	94/100	A	PASS	10	16
LambdaTest/playwright-sample playwright-test-ts/tests	94/100	A	PASS	4	33
carbon-design-system/carbon e2e	94/100	A	PASS	95	423
freeCodeCamp/freeCodeCamp e2e	94/100	A	PASS	83	282
strapi/strapi tests/e2e	94/100	A	PASS	100	280
WordPress/gutenberg test/e2e	93/100	A	PASS	341	1922
apache/apisix-dashboard e2e/tests	93/100	A	PASS	125	208
documenso/documenso packages/app-tests	93/100	A	PASS	135	1183
godotlauncher/launcher e2e	93/100	A	PASS	17	113
google/site-kit-wp tests/playwright/specs	93/100	A	PASS	8	36
n8n-io/n8n packages/testing/playwright	93/100	A	PASS	279	1044
radix-ui/primitives e2e	93/100	A	PASS	11	105
sanity-io/sanity e2e	93/100	A	PASS	65	171
supabase/supabase e2e	92/100	A	PASS	33	264
umami-software/umami tests/e2e	92/100	A	PASS	8	38
coder/coder site/e2e	91/100	A	PASS	41	96
primer/react e2e	91/100	A	PASS	79	195
vendure-ecommerce/vendure packages/dashboard/e2e	91/100	A	PASS	44	212
cupcakearmy/cryptgeon test	89/100	B	PASS	11	24
element-hq/element-web apps/web/playwright	89/100	B	PASS	154	753
woocommerce/woocommerce plugins/woocommerce/tests/e2e	89/100	B	PASS	213	846
PostHog/posthog playwright	88/100	B	PASS	43	121
go-gitea/gitea tests/e2e	88/100	B	PASS	29	56
wireapp/wire-webapp apps/webapp/test/e2e_tests	88/100	B	PASS	38	132
clerk/javascript integration/tests	87/100	B	PASS	130	577
grafana/grafana e2e-playwright	87/100	B	PASS	213	680
nuxt/nuxt test/e2e	87/100	B	PASS	23	171
shopify/cli packages/e2e/tests	87/100	B	PASS	11	26
checkly/checkly-cli examples/advanced-project/src	86/100	B	PASS	4	6
mtlynch/whatgotdone e2e	86/100	B	PASS	12	24
zotero/web-library test/playwright	86/100	B	PASS	17	104
Studio-Saelix/sencho e2e	85/100	B	PASS	34	200
formbricks/formbricks apps/web/playwright	85/100	B	PASS	48	132
Automattic/wp-calypso test/e2e	84/100	B	PASS	122	187
saleor/saleor-dashboard e2e-legacy	84/100	B	PASS	39	158
twentyhq/twenty packages/twenty-e2e-testing/tests	84/100	B	PASS	6	8
usebruno/bruno tests	84/100	B	PASS	488	1583
RocketChat/Rocket.Chat apps/meteor/tests/e2e	83/100	B	PASS	176	1050
handsontable/handsontable tests/e2e	83/100	B	PASS	159	1526
adobe/spectrum-web-components gen2/packages/swc/components	82/100	B	PASS	26	185
hashicorp/consul ui/packages/consul-ui/e2e-tests/tests	82/100	B	PASS	15	82
novuhq/novu apps/dashboard/tests	82/100	B	PASS	2	2
appsmithorg/appsmith app/client/playwright/tests	80/100	B	PASS	6	15
elastic/elastic-charts e2e/tests	80/100	B	PASS	29	385
microsoft/fluentui packages/web-components/src	80/100	B	PASS	43	737
KittyCAD/modeling-app e2e/playwright	79/100	C	FAIL	50	309
mattermost/mattermost e2e-tests/playwright	79/100	C	FAIL	116	467
payloadcms/payload test/admin/e2e	79/100	C	FAIL	8	272
eclipse-theia/theia examples/playwright/src/tests	78/100	C	FAIL	16	97
immich-app/immich e2e/src/specs	78/100	C	FAIL	13	43
invoiceninja/invoiceninja tests/e2e	78/100	C	FAIL	37	335
tanstack/table examples	78/100	C	FAIL	398	1278
apache/superset superset-frontend/playwright/tests	77/100	C	FAIL	32	100
apache/zeppelin zeppelin-web-angular/e2e	77/100	C	FAIL	54	286
guardian/dotcom-rendering dotcom-rendering/playwright/tests	77/100	C	FAIL	16	76
pradosoft/prado tests/playwright	77/100	C	FAIL	214	319
cloudflare/vinext tests/e2e/pages-router	76/100	C	FAIL	31	144
mermaid-js/mermaid e2e	76/100	C	FAIL	75	456
microsoft/vscode test/componentFixtures/playwright	76/100	C	FAIL	7	18
withastro/astro packages/astro/e2e	75/100	C	FAIL	50	311
bbc/simorgh ws-nextjs-app/playwright	74/100	C	FAIL	10	116
BuilderIO/builder packages/sdks-tests/src/e2e-tests	72/100	C	FAIL	52	199
ionic-team/ionic-framework core	72/100	C	FAIL	505	2962
openplayerjs/openplayerjs e2e	72/100	C	FAIL	8	77
tagspaces/tagspaces tests/e2e	72/100	C	FAIL	37	307
tldraw/tldraw apps/examples/e2e	72/100	C	FAIL	35	213
toeverything/AFFiNE tests/blocksuite/e2e	72/100	C	FAIL	99	1013
calcom/cal.com apps/web/playwright	71/100	C	FAIL	53	253
live-codes/livecodes e2e	70/100	C	FAIL	14	246
wekan/wekan tests/playwright/specs	70/100	C	FAIL	106	441
QwikDev/qwik e2e	69/100	D	FAIL	69	449
currents-dev/currents-examples playwright/pnpm/tests	69/100	D	FAIL	8	12
gitkraken/vscode-gitlens tests/e2e/specs	69/100	D	FAIL	26	238
EnixCoda/Gitako e2e	66/100	D	FAIL	36	53
openreplay/openreplay frontend/tests/playwright	66/100	D	FAIL	5	5
Flagsmith/flagsmith frontend/e2e/tests	62/100	D	FAIL	20	27
activepieces/activepieces packages/tests-e2e	58/100	F	FAIL	5	5
th3cyb3rhub/TheCyberHub e2e	56/100	F	FAIL	11	138
DataDog/documentation hugo/e2e	50/100	F	FAIL	31	150
getsentry/spotlight packages/spotlight/tests	33/100	F	FAIL	6	47

Repo names link to the exact source scanned. Full methodology, findings breakdown, and a re-runnable script to reproduce every number here live in VALIDATION.md on GitHub.

Open tool · managed platform

Free scorer. Full coverage when you want it.

playwright-score grades the code you already have. QA Guardian maps critical journeys, generates maintainable Playwright, and keeps it green with human-in-the-loop review—using the same quality bar we open-sourced.

You own standard Playwright TypeScript—export anytime
AI drafts + senior engineers verify
Parallel cloud runner, not a brittle Selenium grid

Book a demo Playwright infrastructure

# example output

Playwright Spec Score v4 · profile=standard
Score: 86/100 (B) · PASS (threshold 80)

Dimensions:
  playwrightHygiene: 90
  assertions: 100
  locators: 70
  structure: 95

Findings:
  [warning] playwright/no-raw-locators …

FAQ

Is the score AI-based?

No. The score is fully deterministic and AI-free. Same files always produce the same score under the versioned scoring model (v4). AI can generate or repair code using the findings; rules grade the code.

How is this different from eslint-plugin-playwright?

We run eslint-plugin-playwright under the hood for community best practices, then add a versioned 0–100 score, suite-level metrics (locator ratio, assertion-delegation tracing, Page Object Model import resolution), and CI exit codes.

Does it grade real-world code accurately, or just clean examples?

We validate it against real, public Playwright suites on an ongoing basis — not just our own fixtures. See the real-world results table on this page, or re-run the validation yourself from the GitHub repo.

Can I use this in CI?

Yes. playwright-score exits 0 when the score meets your threshold and 1 when it fails. JSON, markdown, and SARIF outputs are supported.

Who maintains it?

QA Guardian (qaguardian.com). We dogfood it on AI-generated Playwright flows in our managed QA platform. The core scorer is MIT open source.

Score your suite. Own your Playwright.

Star the repo, drop it in CI, or let QA Guardian build and maintain journey coverage for you.

GitHub Talk to us