Validating API Responses Against OpenAPI in CI

This guide is part of Mock Servers & Contract Testing from OpenAPI within OpenAPI & AsyncAPI Schema Authoring. It covers the cheapest contract check available: validating the responses your existing test suite already produces, rather than generating new requests. If you have integration tests, you are already exercising the API — this just starts checking the answers against the document.

Validation added to an existing test suite The same integration tests run unchanged, but their traffic passes through a proxy that checks every response against the specification before returning it. existing tests unchanged prism proxy checks every response against the contract the real API under test conforms pass violates 500 → fail

Problem & Context

Most teams already have integration tests that call their API and assert on the results. Those assertions are hand-written and partial: they check the two or three fields the test cares about and ignore everything else in the response. A field that changed type, a status code that changed meaning, a header that disappeared — none of it is noticed, because no assertion mentions it.

Meanwhile the specification sits alongside, describing what all of those responses should look like, and nothing compares the two.

Response validation closes that gap for free. The requests already exist; the oracle already exists; the only missing piece is something that puts them together. Compared with generated contract testing, this covers less of the documented surface — only what the suite happens to exercise — but it covers exactly the paths your product actually depends on, and it costs one configuration change rather than a new test strategy.

Step-by-Step Solution

1. Route the existing test suite through a validating proxy

Prism’s proxy mode forwards traffic to the real API and validates every response on the way back:

npx @redocly/cli@2 bundle openapi/api.yaml -o dist/api.bundled.yaml
npx prism proxy dist/api.bundled.yaml http://127.0.0.1:3000 --port 4020

Point the suite at the proxy instead of the API:

API_BASE_URL=http://127.0.0.1:4020 npm test

No test changes. Every response the suite already produces is now checked.

2. Fail on violations rather than logging them

By default the proxy warns and passes the response through. --errors turns a violation into a 500 with a violation report, which makes the test fail:

npx prism proxy dist/api.bundled.yaml http://127.0.0.1:3000 --port 4020 --errors

A violation then surfaces in the test output like this:

GET /invoices → 500
{
  "type": "https://stoplight.io/prism/errors#VIOLATIONS",
  "title": "Request/Response not valid",
  "validation": [
    { "location": ["response", "body", "next_cursor"],
      "severity": "Error",
      "message": "should be string" }
  ]
}

3. Add an in-process assertion for unit tests

Where tests call the handler directly rather than over HTTP, validate inside the process:

// test/helpers/contract.js
const { OpenAPIValidator } = require('express-openapi-validator');
const validator = new OpenAPIValidator({ apiSpec: 'dist/api.bundled.yaml' });

/** Assert a response conforms to the contract for its operation. */
async function expectConformsToContract(res, { method, path }) {
  const errors = await validator.validateResponse(res, { method, path });
  if (errors?.length) {
    throw new Error(
      `Response violates the contract for ${method} ${path}:\n` +
      errors.map((e) => `  ${e.path}: ${e.message}`).join('\n'),
    );
  }
}

module.exports = { expectConformsToContract };

// test/invoices.test.js
const { expectConformsToContract } = require('./helpers/contract');

test('GET /invoices returns a conforming page', async () => {
  const res = await request(app).get('/invoices?limit=1');
  expect(res.status).toBe(200);                 // the assertion you already had
  await expectConformsToContract(res, { method: 'get', path: '/invoices' });
});

4. Measure documented-operation coverage

Validation only checks what the suite exercises, so measure what it does not. An operation the suite never calls is an operation nothing validates:

// test/helpers/coverage.js — record every operation the suite touched
const touched = new Set();
afterEach(() => { /* record from the request log */ });

after(() => {
  const spec = require('../dist/api.bundled.yaml');
  const documented = Object.entries(spec.paths).flatMap(([p, item]) =>
    Object.keys(item)
      .filter((m) => ['get','put','post','delete','patch'].includes(m))
      .map((m) => `${m.toUpperCase()} ${p}`));

  const untested = documented.filter((op) => !touched.has(op));
  const pct = Math.round(((documented.length - untested.length) / documented.length) * 100);
  console.log(`contract coverage: ${pct}% (${untested.length} untested)`);
  untested.forEach((op) => console.log(`  UNTESTED ${op}`));

  if (pct < 80) {
    console.error(`Contract coverage ${pct}% is below the 80% floor`);
    process.exit(1);
  }
});

contract coverage: 83% (4 untested)
  UNTESTED DELETE /invoices/{id}
  UNTESTED GET /invoices/{id}/pdf
  UNTESTED POST /invoices/{id}/void
  UNTESTED GET /health

5. Sample production traffic on a schedule

The shapes that only occur with real data never appear in a test fixture. Validate a sample of production responses so that drift shows up before a consumer reports it — sampling a small percentage is enough, and it costs nothing when nothing is wrong.

What each approach covers Response validation covers the paths the suite exercises, generated testing covers the whole documented surface shallowly, and production sampling covers the shapes real data produces. three approaches, three different slices of the surface response validation covers: what the suite already exercises cheapest to add deep, narrow generated testing covers: every documented operation broadest shallow, wide production sampling covers: shapes real data actually produces finds what tests cannot real, unplanned

Complete Working Example

A CI job that starts the API, puts a validating proxy in front of it, runs the existing suite through the proxy, and enforces a coverage floor:

# .github/workflows/response-validation.yml
name: Response validation
on:
  pull_request:
    paths: ['src/**', 'openapi/**', 'test/**']

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci

      - name: Bundle the contract
        run: npx @redocly/cli@2 bundle openapi/api.yaml -o dist/api.bundled.yaml

      - name: Start the API under test
        run: |
          npm run start:test &
          npx wait-on http://127.0.0.1:3000/health --timeout 60000

      # --errors turns every contract violation into a failing request,
      # which turns it into a failing test with no test changes at all.
      - name: Start the validating proxy
        run: |
          npx prism proxy dist/api.bundled.yaml http://127.0.0.1:3000 \
            --port 4020 --errors &
          npx wait-on http-get://127.0.0.1:4020/health --timeout 30000

      - name: Run the existing suite through the proxy
        run: npm test
        env:
          API_BASE_URL: http://127.0.0.1:4020
          CONTRACT_COVERAGE_FLOOR: '80'

      - name: Publish the coverage summary
        if: always()
        run: cat contract-coverage.txt >> "$GITHUB_STEP_SUMMARY"

Expected summary on a healthy run:

contract coverage: 91% (2 untested)
  UNTESTED DELETE /invoices/{id}
  UNTESTED POST /invoices/{id}/void
0 contract violations across 148 validated responses

The two numbers answer different questions and you need both. Zero violations says everything the suite exercised conforms. Ninety-one percent coverage says how much of the documented surface that actually is — and the two untested operations are the ones where a contract violation could ship without anything noticing.

Gotchas & Edge Cases

Health and metrics endpoints are usually undocumented. They exist, the suite calls them, and the proxy reports every response as violating a contract that never mentions them. Either document them or exclude those paths from validation deliberately; do not lower the severity globally to make the noise go away.

The proxy changes the base URL the suite sees. Tests that assert on absolute URLs in response bodies — a Location header, a pagination next link — will see the origin they were generated with, not the proxy’s. Assert on paths rather than absolute URLs, which is more robust regardless.

A response can conform and still be wrong. Schema conformance says nothing about whether the numbers are right. Keep behavioural assertions; this replaces none of them, it only adds a check nobody was doing.

format keywords are validated inconsistently. format: date-time is enforced by most validators, format: email by some, and custom formats by none. A response that passes locally may fail under a different validator version, so pin the validator alongside the specification.

Coverage can be gamed into meaninglessness. A test that calls an operation and asserts nothing still counts as coverage. Treat the percentage as a prompt to look at the untested list, not as a target to raise.

Start with the proxy, add the rest only if it earns its place. Of everything in this guide, the proxy is the one piece worth adopting on day one: it requires no test changes, no new dependency in the application, and no decision about where assertions belong. Run the existing suite through it, look at what it reports, and you will learn more about the state of your contract in an afternoon than a week of reading the document would tell you. The in-process helper and the coverage gate are refinements — useful once the proxy is green and staying green, unnecessary noise before that.

Expect the first result to be uncomfortable and act on it in the right order. A suite that has never been validated will typically report violations in a handful of places and undocumented responses in many more. Fix the undocumented responses first: they are usually one line in the specification each, they clear most of the output, and they make the genuine schema violations visible instead of buried. Only then work through the remaining violations, deciding case by case whether the document or the implementation is the thing that is wrong. Doing it in the other order means fighting through noise to find the two findings that mattered.

Order of triage on the first run Clear undocumented responses first because they are one-line specification fixes, then work through genuine schema violations case by case. 1. undocumented responses one line of YAML each clears most of the output 2. schema violations decide: document or code? now visible, not buried

FAQ

How is this different from contract testing with generated inputs?

Generated testing invents requests to probe the whole documented surface. Response validation reuses the requests your suite already makes, so it costs almost nothing to add and validates exactly the paths your product depends on. They complement each other: one is wide and shallow, the other narrow and deep.

Do I need a proxy, or can I validate in process?

Either works. A proxy needs no code change and catches everything crossing the wire, including responses from middleware your tests never call directly. An in-process assertion is easier where tests invoke the handler rather than HTTP. Most teams end up with both.

What should happen when a response is valid but undocumented?

Fail. An undocumented status code is drift regardless of whether the body is well-formed, and it is the single most common finding when teams first turn this on. Decide whether the specification or the implementation is wrong, and fix that one.