Skip to content
LLMAssert
Playwright Plugin

Assert the unpredictable

LLM-powered assertions for Playwright. Test your AI outputs with matchers you already know.

Get Started FreeRead the Docs
Example Playwright test using LLMAssert assertions
import { test, expect } from '@llmassert/playwright';

test('chatbot stays grounded in docs', async ({ page }) => {
  await page.goto('/chat');
  await page.fill('[name="message"]', 'What is the refund policy?');
  await page.click('button[type="submit"]');

  const reply = await page.textContent('.chat-response');
  const docs = await page.textContent('.source-context');

  // LLM-aware assertions — just like Playwright
  await expect(reply).toBeGroundedIn(docs);
  await expect(reply).toBeFreeOfPII();
  await expect(reply).toMatchTone('helpful and professional');
});

How it works

Three steps to start testing your LLM outputs.

1

Install

Add @llmassert/playwright to your project. One package, zero config.

pnpm add -D @llmassert/playwright
2

Write

Swap your import and add LLM assertions alongside your existing Playwright tests.

await expect(reply).toBeGroundedIn(docs);
3

Run & See

Run your tests with npx playwright test. Scores and trends appear on the dashboard.

npx playwright test

5 assertion types for LLM outputs

Each assertion returns a score, reasoning, and pass/fail — powered by LLM judges. All support .not for negation.

Groundedness

Catches hallucinations. Verifies the response is grounded in the provided source context.

await expect(reply).toBeGroundedIn(docs);

PII Detection

Detects personal data leaks. Ensures no emails, phone numbers, or sensitive info in output.

await expect(reply).toBeFreeOfPII();

Tone Matching

Validates tone and sentiment. Check that responses match your brand voice.

await expect(reply).toMatchTone('professional');

Format Compliance

Enforces structured output. Verify JSON responses match your expected schema.

await expect(reply).toBeFormatCompliant('valid JSON with name and email fields');

Semantic Match

Measures semantic similarity. Compare outputs against expected answers with fuzzy matching.

await expect(reply).toSemanticMatch('Order #123 has been shipped');

Simple, transparent pricing

Start free — no credit card required. Scale as your test suite grows.

Free

100 evaluations / month · 1 project · No credit card required

Get Started Free

Starter

$29/month
  • 5,000 evaluations / month
  • 3 projects
  • Included: Dashboard analytics
  • Included: All 5 assertion types
  • Included: CI integration
  • Included: Custom thresholds
  • Included: Cost tracking
Get Started

Pro

Popular
$79/month
  • 25,000 evaluations / month
  • 10 projects
  • Included: Dashboard analytics
  • Included: All 5 assertion types
  • Included: CI integration
  • Included: Custom thresholds
  • Included: Cost tracking
Get Started

Team

$199/month
  • 100,000 evaluations / month
  • Unlimited projects
  • Included: Dashboard analytics
  • Included: All 5 assertion types
  • Included: CI integration
  • Included: Custom thresholds
  • Included: Cost tracking
Get Started

Start testing your LLM outputs today

Free tier, no credit card required. Add LLM assertions to your Playwright tests in minutes.

Get Started Free