Detecting Breaking Changes with oasdiff

This guide is part of Breaking Change Detection & Spec Diffing within SDK Generation & Changelog Automation. It covers the tool itself: getting a trustworthy breaking-change report out of two OpenAPI documents, and tuning it until the output is short enough that people read it. The motivating failure is one nobody catches in review — a schema tightened, a field removed, an optional parameter made required — because none of those look alarming in a 900-line YAML diff.

How oasdiff compares two documents Both specifications are bundled into resolved single documents, compared, and the result is split into breaking changes and a full changelog. base spec multi-file head spec multi-file bundle both removes $ref noise oasdiff breaking report exit 1 → gate the merge full changelog → release notes

Problem & Context

Reviewing a specification diff for compatibility is a task humans are measurably bad at. The edits that break consumers are small and look innocuous — a type narrowed from string to an enum, a property moved into required, a maxLength reduced, a response field deleted because a search suggested nothing used it. Each is one or two lines in a diff that may run to hundreds, and each invalidates requests that consumers are sending right now.

The mechanical nature of the judgement is exactly why a tool should make it. “Does this change invalidate a request a working consumer already sends, or a response they already parse?” is answerable from the two documents alone, without knowing anything about the product. oasdiff answers it, classifies every change, and returns an exit code you can gate on.

What determines whether the team actually uses it is signal quality. A report that fires on every reworded description is one people stop reading within a fortnight, and the settings below exist entirely to prevent that.

Step-by-Step Solution

1. Install oasdiff and bundle both sides

curl -fsSL https://raw.githubusercontent.com/oasdiff/oasdiff/main/install.sh | sh -s -- v1.11.7
oasdiff --version

oasdiff version 1.11.7

Bundling both sides is not optional. Comparing unresolved multi-file documents makes every relocated $ref look like a structural change:

git show origin/main:openapi/api.yaml > /tmp/base.yaml
npx @redocly/cli@2 bundle /tmp/base.yaml   -o /tmp/base.bundled.yaml
npx @redocly/cli@2 bundle openapi/api.yaml -o /tmp/head.bundled.yaml

2. Run the breaking-change report

oasdiff breaking /tmp/base.bundled.yaml /tmp/head.bundled.yaml --format text
echo "exit: $?"

1 breaking changes: 1 error, 0 warning
error	[response-property-removed] at /tmp/head.bundled.yaml
	in API GET /invoices/{id}
		removed the response property 'legacy_total' for the response status '200'
exit: 1

Each finding carries a rule id in brackets. That id is what you reference in the ignore file, so read the report as a list of rule identifiers rather than as prose.

3. Exclude prose from the comparison

Documentation edits are the most frequent specification change and the least interesting for compatibility:

oasdiff breaking /tmp/base.bundled.yaml /tmp/head.bundled.yaml \
  --exclude-elements description,examples,title \
  --format text

This single flag removes most of the volume from a typical report.

4. Encode your compatibility policy in an ignore file

Suppress only what your policy genuinely permits, and give every entry a reason and a scope. An unexplained, unscoped suppression is indistinguishable from an accident six months later:

# .oasdiff-ignore.yaml
# Our published compatibility policy requires tolerant readers: consumers must
# ignore unknown response fields. Documented at /compatibility-policy.
- id: response-property-added
  reason: 'Tolerant-reader policy — see /compatibility-policy.'

# The sandbox host is explicitly not part of the contract.
- id: api-server-removed
  path: /invoices
  reason: 'Sandbox server entry only; production servers covered separately.'

5. Produce a changelog from the same diff

The same comparison yields the complete list of changes, which is a far better basis for release notes than a commit log:

oasdiff changelog /tmp/base.bundled.yaml /tmp/head.bundled.yaml --format text

2 changes: 1 error, 1 info
error	[response-property-removed] at /tmp/head.bundled.yaml
	in API GET /invoices/{id}
		removed the response property 'legacy_total' for the response status '200'
info	[request-parameter-added] at /tmp/head.bundled.yaml
	in API GET /invoices
		added the new optional 'query' request parameter 'cursor'
Tuning the report down to signal Bundling removes reference noise, excluding prose removes documentation churn, and a scoped ignore file removes policy-permitted changes, leaving only real findings. raw 140 findings + bundled 38 findings + no prose 9 findings + policy ignores 2 findings — both real credibility is the asset — a noisy report is an ignored report

Complete Working Example

A script that produces both reports and exits with a classification the caller can act on:

#!/usr/bin/env bash
# spec-diff.sh — classify a contract change as major, minor, or patch.
set -uo pipefail

BASE_REF="${BASE_REF:-origin/main}"
SPEC="${SPEC:-openapi/api.yaml}"
IGNORE="${IGNORE:-.oasdiff-ignore.yaml}"

mkdir -p /tmp/specdiff

# Resolve both sides. Without this, moved $refs read as structural changes.
git show "${BASE_REF}:${SPEC}" > /tmp/specdiff/base.yaml
npx @redocly/cli@2 bundle /tmp/specdiff/base.yaml -o /tmp/specdiff/base.bundled.yaml >/dev/null
npx @redocly/cli@2 bundle "${SPEC}"              -o /tmp/specdiff/head.bundled.yaml >/dev/null

COMMON=(--exclude-elements description,examples,title)
[ -f "${IGNORE}" ] && COMMON+=(--warn-ignore "${IGNORE}")

# Breaking report drives the gate; the exit code is the signal.
oasdiff breaking /tmp/specdiff/base.bundled.yaml /tmp/specdiff/head.bundled.yaml \
  "${COMMON[@]}" --format text > /tmp/specdiff/breaking.txt
BREAKING=$?

# Changelog drives the release notes and the minor/patch distinction.
oasdiff changelog /tmp/specdiff/base.bundled.yaml /tmp/specdiff/head.bundled.yaml \
  "${COMMON[@]}" --format text > /tmp/specdiff/changelog.txt

if [ "${BREAKING}" -ne 0 ]; then
  LEVEL=major
elif [ -s /tmp/specdiff/changelog.txt ]; then
  LEVEL=minor
else
  LEVEL=patch
fi

echo "level=${LEVEL}"
echo '--- changes ---'
cat /tmp/specdiff/changelog.txt
if [ "${LEVEL}" = major ]; then
  echo '--- BREAKING ---'
  cat /tmp/specdiff/breaking.txt
fi

# Exit 0 always: the caller decides what to do with the level.
exit 0

Expected output for an additive change:

level=minor
--- changes ---
1 changes: 0 error, 1 info
info	[request-parameter-added] at /tmp/specdiff/head.bundled.yaml
	in API GET /invoices
		added the new optional 'query' request parameter 'cursor'

Exiting 0 regardless is deliberate. This script classifies; it does not decide. Keeping the decision in the caller means the same script serves the pull request comment, the release job, and a local check without any of them needing different behaviour.

Gotchas & Edge Cases

A shallow clone has no base commit. git show origin/main:... fails in CI when the checkout used the default depth of one. Set fetch-depth: 0, or the job fails with an error that looks like a missing file rather than a missing history.

Ignore entries without a path apply everywhere. - id: response-property-added with no path suppresses that rule across the entire API. That may be exactly what a tolerant-reader policy intends, but it is worth being explicit about, because the next person to read the file cannot tell a deliberate global policy from a hurried suppression.

Diffing against a moving branch hides earlier changes. If main already contains two unreleased breaking changes, diffing a new pull request against it reports only the third. For the release decision, diff against the last released tag instead.

Exit codes differ between subcommands. oasdiff breaking exits non-zero when it finds breaking changes; oasdiff changelog exits zero regardless. Wrapping the breaking call in set -e without care aborts the script before the changelog runs, which is why the example uses set -uo pipefail rather than set -euo pipefail.

The tool cannot know your semantics. A field renamed from total to total_cents is reported as one removal and one addition, not as a rename with a unit change. The classification is correct — it is breaking — but the release note needs a human to explain what actually happened.

Calibrate against history before trusting the classification. The fastest way to know whether your configuration is right is to replay it over the last year of releases. Check out each release tag in turn, diff it against its predecessor, and compare what the tool says against what actually happened — the releases that broke consumers should classify as major, and the quiet ones should not. An afternoon of this usually surfaces one or two policy questions nobody had ever answered explicitly, which is a far better time to answer them than during an urgent release. It also gives you a concrete answer when somebody asks whether the gate would have caught the last incident.

Treat every recurring false positive as a configuration bug. It is tempting to tell the team “ignore the ones about descriptions” and move on, but a report that requires verbal caveats to interpret has already stopped being a gate. Each time somebody has to explain away a finding, either add a scoped ignore entry with a reason or adjust the excluded elements. The goal is a report where every line is something a reviewer should look at, because that is the only state in which people keep looking.

Calibrating against release history Replaying the diff across consecutive release tags and comparing the classification to what actually happened validates the configuration before it gates anything. replay the last year of tags, then compare v2.4 → v2.5 tool says: minor v2.5 → v3.0 tool says: major compare against what actually happened they agree configuration is calibrated they disagree a policy question to settle now

FAQ

Why does oasdiff report breaking changes when I only moved a file?

It is comparing unresolved documents, so a relocated $ref reads as a structural change. Bundle both sides into single resolved documents before diffing and the noise disappears — this one step removes the large majority of false positives.

Is adding a response field breaking?

It depends on your compatibility policy. It is safe for tolerant readers and breaking for strictly validating generated clients, which describes most SDKs with validation enabled. Decide, write it into the ignore file with a reason, and state the expectation in your public compatibility documentation so consumers know what tolerance you assume.

What should I diff against — main or the last release?

Both, for different questions. Diff against the merge base for the review comment, so a reviewer sees what this change does; diff against the last released tag for the release decision, so unreleased changes earlier in the cycle are not missed.