API Contract Testing: Schema, OpenAPI and Consumer-Driven Pact

A backend developer renames price to amount. Every backend test still passes. The release goes out, and the Android app shows “Rs null” on every product. Contract testing exists to stop exactly this: it checks that the API keeps the promises its consumers rely on, automatically, on every change. In this lesson you’ll learn the three levels of API contract testing, and run a real consumer-driven contract with Pact against the live practice API.

Why Functional Tests Don’t Catch Contract Breaks

Level 1: Schema Contract Tests

The simplest contract: validate every response against a schema. The practice API publishes its own user schema at GET /schema/user:

curl -i \
  "https://sqatools.in/wp-json/sqa-api/v1/schema/user"

Because it says "additionalProperties": false and lists every required field and type, any renamed, removed, retyped or extra field fails validation. You already wrote this test in the JSON Schema lesson; module 6 turns it into a reusable framework helper.

Strength: easy, no coordination needed. Weakness: the provider defines the schema, so it validates what the provider thinks consumers need, not what they actually use.

Level 2: OpenAPI Diff (Breaking-Change Detection)

If the team keeps an OpenAPI file in Git, compare the old and new versions on every pull request:

# Fails the build on breaking changes such as removed fields or new required parameters
oasdiff breaking main-openapi.yaml pr-openapi.yaml --fail-on ERR

Strength: catches changes before code is even deployed. Weakness: only as good as the spec; if the code drifts from the spec, nothing notices (combine with Level 1).

Level 3: Consumer-Driven Contract Testing with Pact

In consumer-driven contracts, each consumer writes down exactly what it sends and what it needs back. That file (the pact) is shared with the provider, whose build replays it against the real API. The provider can change anything consumers don’t use, and gets told immediately if it breaks something they do use.

1. Consumer test runs the consumer’s real client code against a Pact mock server that returns the expected response. Pact records the interaction into a JSON pact file.
2. The pact file is published, usually to a Pact Broker (self-hosted) or PactFlow.
3. Provider verification replays each recorded request against the real provider and compares the real response with the contract, using matchers (type, regex) instead of exact values.
4. can-i-deploy checks the broker before release: “is this version compatible with everything in production?”

Hands-On: A Real Pact Contract with the Practice API

Scenario: a MobileApp consumer needs id, name, role and active from GET /users/1. Install Pact into your Playwright project:

npm install -D @pact-foundation/pact

import { test, expect } from '@playwright/test';
import path from 'node:path';
import { PactV3, MatchersV3, Verifier } from '@pact-foundation/pact';

const { integer, string, regex, boolean } = MatchersV3;
const PACT_DIR = path.join(__dirname, '..', '..', 'pacts');

test.describe.configure({ mode: 'serial' });

test('consumer: mobile app expects this shape from GET /users/1', async () => {
  const provider = new PactV3({ consumer: 'MobileApp', provider: 'SqaUsersApi', dir: PACT_DIR });

  provider
    .given('user 1 exists')
    .uponReceiving('a request for user 1')
    .withRequest({ method: 'GET', path: '/users/1', headers: { Accept: 'application/json' } })
    .willRespondWith({
      status: 200,
      headers: { 'Content-Type': regex('application/json.*', 'application/json') },
      body: {
        data: {
          id: integer(1),
          name: string('Asha Verma'),
          role: regex('admin|editor|viewer', 'admin'),
          active: boolean(true),
        },
      },
    });

  await provider.executeTest(async (mock) => {
    // This is the consumer's real client code calling the Pact mock server
    const res = await fetch(`${mock.url}/users/1`, { headers: { Accept: 'application/json' } });
    expect(res.status).toBe(200);
    const body = await res.json();
    expect(body.data.name).toBe('Asha Verma');
  });
});

test('provider: the real API honours the contract', async () => {
  test.setTimeout(60_000);
  await new Verifier({
    provider: 'SqaUsersApi',
    providerBaseUrl: 'https://sqatools.in/wp-json/sqa-api/v1',
    pactUrls: [path.join(PACT_DIR, 'MobileApp-SqaUsersApi.json')],
    stateHandlers: { 'user 1 exists': async () => { /* data is fixed on the practice API */ } },
    logLevel: 'warn',
  }).verifyProvider();
});

Both tests pass against the live API. Notice the contract only mentions the four fields the app uses; the provider can add fields like city freely. The matchers mean “an integer like 1”, “a string like Asha Verma”, “one of admin/editor/viewer”, so the contract doesn’t break when data values change.

See it fail on purpose

Change name: string('Asha Verma') to full_name: string('Asha Verma') in the consumer test, as if the app expected a renamed field. The consumer test still passes (it talks to the mock), but provider verification fails: the real API has no full_name. The real verifier output:

Verifying a pact between MobileApp and SqaUsersApi
  a request for user 1
     Given user 1 exists
    returns a response which
      has status code 200 (OK)
      includes headers "Content-Type" with value "application/json" (OK)
      has a matching body (FAILED)

1.1) has a matching body
     $.data -> Actual map is missing the following keys: full_name

That’s the early warning you want: precise, in CI, before any user sees “null”.

Provider States

given('user 1 exists') is a provider state: a precondition the provider must set up before replaying the request. In real projects the state handler seeds the database (create user 1, create an order with status “shipped”…). The practice API’s data is fixed, so the handler is empty.

Pact in CI/CD

# Consumer pipeline: run consumer tests, then publish the pact
npx pact-broker publish ./pacts --consumer-app-version "$GIT_SHA" --branch "$GIT_BRANCH" \
  --broker-base-url "$PACT_BROKER_URL" --broker-token "$PACT_BROKER_TOKEN"

# Provider pipeline: verify pacts fetched from the broker (Verifier with pactBrokerUrl), publish results

# Before deploying either side
npx pact-broker can-i-deploy --pacticipant MobileApp --version "$GIT_SHA" --to-environment production

The broker shows a matrix of which consumer and provider versions are compatible, and the can-i-deploy gate prevents incompatible releases.

Choosing an Approach

Common Mistakes

• Using exact values in contracts ("name": "Asha Verma") instead of matchers, so every data change breaks the contract.
• Putting business-logic checks in contracts. Contracts check shape; functional tests check behaviour.
• Consumers listing every field “just in case”: the contract should contain only what the consumer really uses.
• Never running provider verification in the provider’s own pipeline.
• Skipping the can-i-deploy gate, so incompatible versions still reach production.

Interview Questions: API Contract Testing

gRPC API Testing Basics: Protobuf, Status Codes and Streaming

Inside large companies, most API calls never touch a browser: the order service talks to pricing, pricing talks to inventory, inventory talks to the warehouse, millions of times a minute. Many of these internal calls use gRPC because it’s fast and strongly typed. If you test microservices, you’ll meet it sooner or later. This lesson explains gRPC from a tester’s angle and walks through real calls to a public gRPC test server, including streaming and deadlines.

Why gRPC Exists (REST vs gRPC)

The .proto Contract

We’ll test the public grpcb.in test server (plaintext on grpcb.in:9000, TLS on grpcb.in:9001). Its hello service is defined like this:

syntax = "proto3";
package hello;

service HelloService {
  rpc SayHello (HelloRequest) returns (HelloResponse);                // unary
  rpc LotsOfReplies (HelloRequest) returns (stream HelloResponse);    // server streaming
}

message HelloRequest  { string greeting = 1; }
message HelloResponse { string reply = 1; }

Key points for testers:

• Each field has a number (= 1). The number, not the name, is what goes on the wire. Changing or reusing numbers breaks clients.
• In proto3, missing fields get default values (empty string, 0, false). An API can’t tell “not sent” from “sent empty” unless it uses wrappers or optional. Test how the service treats defaults.
• Code generators create typed clients in Java, Go, Python, TypeScript and more from this file.

The Four Types of gRPC Calls

gRPC Status Codes (Know These for Interviews)

Just like 4xx vs 5xx in REST, invalid client input should give INVALID_ARGUMENT, never INTERNAL or UNKNOWN.

Calling gRPC Manually

Postman

1. New → gRPC. Server URL: grpcb.in:9000 (TLS off).
2. Choose Using server reflection (grpcb.in supports it) or import the .proto file.
3. Select hello.HelloService / SayHello, message {"greeting": "Asha"}, click Invoke.
4. Response: {"reply": "hello Asha"} with status 0 OK.

grpcurl (command line)

# List services via server reflection
grpcurl -plaintext grpcb.in:9000 list

# Describe a service
grpcurl -plaintext grpcb.in:9000 describe hello.HelloService

# Unary call
grpcurl -plaintext -d '{"greeting": "Asha"}' grpcb.in:9000 hello.HelloService/SayHello

# Metadata (gRPC's headers), e.g. auth
grpcurl -plaintext -H "authorization: Bearer <token>" -d '{}' my-service:50051 orders.OrderService/ListOrders

Automating gRPC Tests with Playwright + grpc-js

Playwright’s runner, fixtures and reports work for any async test, so you can drive gRPC with the official Node client. Install it in your project:

npm install -D @grpc/grpc-js @grpc/proto-loader

Save the .proto above as hello.proto in the project root, then:

import { test, expect } from '@playwright/test';
import * as grpc from '@grpc/grpc-js';
import * as loader from '@grpc/proto-loader';

const pkg: any = grpc.loadPackageDefinition(loader.loadSync('hello.proto'));
const client = new pkg.hello.HelloService('grpcb.in:9000', grpc.credentials.createInsecure());
const deadline = (ms: number) => new Date(Date.now() + ms);

function sayHello(greeting: string, ms = 8_000): Promise<{ reply: string }> {
  return new Promise((resolve, reject) =>
    client.SayHello({ greeting }, { deadline: deadline(ms) }, (err: grpc.ServiceError | null, res: any) =>
      err ? reject(err) : resolve(res)));
}

test('unary call returns the expected reply', async () => {
  expect(await sayHello('Asha')).toEqual({ reply: 'hello Asha' });
});

test('server streaming returns several messages', async () => {
  const replies: string[] = await new Promise((resolve, reject) => {
    const got: string[] = [];
    const stream = client.LotsOfReplies({ greeting: 'Asha' }, { deadline: deadline(8_000) });
    stream.on('data', (m: any) => got.push(m.reply));
    stream.on('error', reject);
    stream.on('end', () => resolve(got));
  });
  expect(replies.length).toBeGreaterThan(1);
  for (const r of replies) expect(r).toBe('hello Asha');
});

test('a too-short deadline fails with DEADLINE_EXCEEDED', async () => {
  const err = await sayHello('Asha', 1).catch((e: grpc.ServiceError) => e);
  expect((err as grpc.ServiceError).code).toBe(grpc.status.DEADLINE_EXCEEDED);   // 4
});

When written, this returned hello Asha for the unary call, 10 messages from the stream, and status code 4 for the 1 ms deadline. Public test servers can be slow or offline, so for your own services run tests against a test environment you control.

gRPC Test Checklist

Breaking vs Safe .proto Changes

Tools like buf breaking check these rules automatically in CI, the gRPC equivalent of OpenAPI diff tools from the OpenAPI lesson.

Tools for gRPC Testing

• Postman: gRPC requests with reflection or .proto import, streaming support.
• grpcurl: cURL-like command-line client.
• Kreya, Insomnia: GUI clients.
• Code: grpc-js (TypeScript), grpcio (Python), grpc-java; run them in Playwright, pytest or JUnit.
• ghz, k6: load testing.
• buf: linting and breaking-change detection.

Interview Questions: gRPC Testing

WebSocket and Webhook Testing: Real-Time APIs Explained

Not every API waits for you to ask. Live cricket scores, stock tickers, ride tracking and chat are pushed to you over WebSockets; payment confirmations, shipping updates and GitHub events are pushed to your server as webhooks. Both break in ways request-response testing never sees: dropped connections, out-of-order messages, duplicate events, forged signatures. This lesson shows you how to test both, with live echo servers and runnable Playwright code.

Polling vs WebSocket vs Webhook

WebSocket Basics for Testers

1. The client sends an HTTP request with Upgrade: websocket to a wss:// URL.
2. The server answers 101 Switching Protocols. The connection stays open.
3. Both sides exchange messages (text, usually JSON, or binary) in any order.
4. Either side closes with a close code (1000 normal, 1001 going away, 1006 abnormal, 1008 policy violation, 1011 server error).

Many apps add a protocol on top: subscribe/unsubscribe messages, heartbeats (ping/pong), message ids and acknowledgements. Socket.IO, STOMP and GraphQL subscriptions are common higher-level protocols.

Try It: Public Echo Servers

Two free echo servers send back whatever you send, which is perfect for learning:

• wss://echo.websocket.org: first sends a greeting like Request served by 4d896d95b55478, then echoes each message.
• wss://ws.postman-echo.com/raw: echoes immediately.

In Postman

1. New → WebSocket. URL: wss://ws.postman-echo.com/raw. Click Connect.
2. Type {"event":"subscribe","match":"IND-vs-AUS"} and Send.
3. The Messages pane shows sent (up arrow) and received (down arrow) messages with timestamps.
4. Click Disconnect and check the close event.

In the browser console

const ws = new WebSocket('wss://ws.postman-echo.com/raw');
ws.onopen = () => ws.send(JSON.stringify({ ping: 1 }));
ws.onmessage = (e) => console.log('received', e.data);   // received {"ping":1}
ws.onclose = (e) => console.log('closed', e.code);

WebSocket Test Cases

Automating WebSocket Tests in Playwright

For a pure API-level test, use the WebSocket class built into Node.js 22+ inside a Playwright test. A small helper turns events into promises:

import { test, expect } from '@playwright/test';

function openSocket(url: string): Promise<WebSocket> {
  return new Promise((resolve, reject) => {
    const ws = new WebSocket(url);
    ws.onopen = () => resolve(ws);
    ws.onerror = () => reject(new Error(`could not connect to ${url}`));
  });
}

function nextMessage(ws: WebSocket, timeoutMs = 5_000): Promise<string> {
  return new Promise((resolve, reject) => {
    const timer = setTimeout(() => reject(new Error('no message received')), timeoutMs);
    ws.addEventListener('message', (e) => { clearTimeout(timer); resolve(String(e.data)); }, { once: true });
  });
}

test('echo server returns the exact JSON we send', async () => {
  const ws = await openSocket('wss://ws.postman-echo.com/raw');
  const msg = { event: 'subscribe', match: 'IND-vs-AUS', id: Date.now() };

  const reply = nextMessage(ws);
  ws.send(JSON.stringify(msg));
  expect(JSON.parse(await reply)).toEqual(msg);

  ws.close(1000);
});

test('messages come back in order', async () => {
  const ws = await openSocket('wss://ws.postman-echo.com/raw');
  const received: string[] = [];
  const done = new Promise<void>((resolve) => {
    ws.onmessage = (e) => { received.push(String(e.data)); if (received.length === 5) resolve(); };
  });
  for (let i = 1; i <= 5; i++) ws.send(`ball-${i}`);
  await done;
  expect(received).toEqual(['ball-1', 'ball-2', 'ball-3', 'ball-4', 'ball-5']);
  ws.close();
});

For web apps, Playwright can also watch the browser’s own sockets with page.on('websocket', ws => ws.on('framereceived', ...)) and mock a socket server with page.routeWebSocket(), which is useful when testing how the UI reacts to pushed messages.

Webhook Basics for Testers

A typical webhook request your server receives:

POST /webhooks/payments HTTP/1.1
Content-Type: application/json
X-Signature: 5d7c3e...a91f            (HMAC-SHA256 of the raw body with a shared secret)
X-Event-Id: evt_01JD8K2Q
X-Timestamp: 1794482821

{"event":"payment.captured","payment_id":"pay_123","order_id":"ord_987","amount":229746,"currency":"INR"}

Webhook Test Cases

Tools for capturing webhooks

• webhook.site / RequestBin: get a public URL and see every request sent to it. Don’t send real customer data there.
• ngrok / Cloudflare Tunnel: expose your local server to a provider during development.
• Provider CLIs and dashboards: many payment providers can resend events and show delivery logs.

Testing Webhook Signatures in Code

Most providers sign webhooks with HMAC-SHA256 over the raw body (sometimes with the timestamp). Your receiver recomputes the signature and compares it in constant time. Here’s a self-contained test of that logic:

import { test, expect } from '@playwright/test';
import crypto from 'node:crypto';

const SECRET = 'whsec_test_123';   // test secret; real secrets come from environment variables

const sign = (rawBody: string, timestamp: number) =>
  crypto.createHmac('sha256', SECRET).update(`${timestamp}.${rawBody}`).digest('hex');

function verify(rawBody: string, timestamp: number, signature: string, now = Date.now() / 1000): boolean {
  if (Math.abs(now - timestamp) > 300) return false;                       // replay window: 5 minutes
  const expected = Buffer.from(sign(rawBody, timestamp), 'hex');
  const given = Buffer.from(signature, 'hex');
  return given.length === expected.length && crypto.timingSafeEqual(given, expected);
}

const body = JSON.stringify({ event: 'payment.captured', order_id: 'ord_987', amount: 229746 });
const ts = Math.floor(Date.now() / 1000);

test('valid signature is accepted', () => {
  expect(verify(body, ts, sign(body, ts))).toBe(true);
});

test('tampered body is rejected', () => {
  const tampered = body.replace('229746', '100');
  expect(verify(tampered, ts, sign(body, ts))).toBe(false);
});

test('old (replayed) request is rejected', () => {
  const old = ts - 3600;
  expect(verify(body, old, sign(body, old))).toBe(false);
});

test('duplicate delivery is processed once', () => {
  const processed = new Set<string>();
  const handle = (eventId: string) => { if (processed.has(eventId)) return 'duplicate'; processed.add(eventId); return 'processed'; };
  expect(handle('evt_01JD8K2Q')).toBe('processed');
  expect(handle('evt_01JD8K2Q')).toBe('duplicate');
});

In an integration test, the same sign() helper lets you POST signed fake events straight to your own webhook endpoint with request.post(), so you don’t need the real provider for every run.

Common Bugs in Real-Time APIs

• WebSocket server accepts connections without authentication.
• Client never reconnects after the phone switches from Wi-Fi to mobile data.
• Messages arrive out of order during bursts; UI shows an old score.
• Memory leak: server keeps dead connections forever.
• Webhook signature not checked, or checked on a re-serialised body instead of the raw body.
• Duplicate webhook ships an order twice or credits a wallet twice.
• Webhook handler does slow work synchronously and times out, triggering endless retries.

Interview Questions: WebSocket and Webhook Testing

GraphQL API Testing: Queries, Mutations, Errors and Security

GraphQL APIs look simple from the outside, with one URL and one POST, but they hide a surprising number of testing traps. Errors come back with HTTP 200, “not found” is a null instead of a 404, and a single query can ask for so much nested data that it slows the whole server down. In this lesson you’ll test a real public GraphQL API with Postman, cURL and Playwright, and learn the checklist that catches GraphQL-specific bugs.

GraphQL in 2 Minutes

• One endpoint, usually POST /graphql.
• Schema with types and fields defines what can be asked (strongly typed).
• Query reads, mutation writes, subscription streams updates (usually over WebSocket).
• Response is JSON with data and/or errors.

Comparison with REST and SOAP: REST vs SOAP vs GraphQL.

The Public API We’ll Test

We’ll use the free public Countries GraphQL API at https://countries.trevorblades.com/. It needs no key and returns country data. (It’s a third-party demo service; be gentle with it and don’t load test it.)

query {
  country(code: "IN") {
    name
    capital
    currency
    languages { name }
  }
}

Real response (HTTP 200):

{
  "data": {
    "country": {
      "name": "India",
      "capital": "New Delhi",
      "currency": "INR",
      "languages": [ { "name": "Hindi" }, { "name": "English" } ]
    }
  }
}

Sending GraphQL Requests: cURL and Postman

cURL

curl -s https://countries.trevorblades.com/ \
  -H "Content-Type: application/json" \
  -d '{"query":"{ country(code: \"IN\") { name capital currency } }"}'

With variables (the right way)

Never build queries by gluing strings together; pass values as variables. It’s cleaner and avoids injection.

{
  "query": "query GetCountry($code: ID!) { country(code: $code) { name emoji } }",
  "variables": { "code": "JP" },
  "operationName": "GetCountry"
}

Response: {"data":{"country":{"emoji":"🇯🇵","name":"Japan"}}}

Postman

1. New HTTP request, method POST, URL https://countries.trevorblades.com/.
2. Body → GraphQL. Postman fetches the schema and gives autocomplete.
3. Write the query in the QUERY box and variables in the GRAPHQL VARIABLES box. Send.

The Biggest Trap: HTTP 200 Doesn’t Mean Success

We sent three “wrong” requests to the real API. Look at the status codes:

Status code behaviour varies by server (some return 400 for validation errors when using the newer “GraphQL over HTTP” conventions), so check your API’s documented behaviour and test it explicitly.

What to Test in a GraphQL API

GraphQL Security Checks

• Introspection: { __schema { types { name } } } reveals the whole schema. Often disabled in production for internal APIs.
• Field suggestions: “Did you mean …?” messages (like the one above) can leak hidden field names; some teams disable them in production.
• Authorization per resolver: every field that returns sensitive data needs its own check.
• Injection: arguments still reach databases; test them like REST inputs.
• Batching / alias abuse: 1,000 login attempts in one request via aliases can bypass rate limiting.

Automating GraphQL Tests with Playwright

Playwright’s request fixture is all you need. A small helper keeps tests readable:

import { test, expect, APIRequestContext } from '@playwright/test';

const GQL = 'https://countries.trevorblades.com/';

async function gql(request: APIRequestContext, query: string, variables?: Record<string, unknown>) {
  const res = await request.post(GQL, { data: { query, variables } });
  return { status: res.status(), body: await res.json() };
}

test('query with variables returns the right country', async ({ request }) => {
  const { status, body } = await gql(request,
    'query GetCountry($code: ID!) { country(code: $code) { name capital currency } }',
    { code: 'IN' });

  expect(status).toBe(200);
  expect(body.errors).toBeUndefined();                 // the essential GraphQL check
  expect(body.data.country).toEqual({ name: 'India', capital: 'New Delhi', currency: 'INR' });
});

test('unknown record returns null, not an error', async ({ request }) => {
  const { status, body } = await gql(request, '{ country(code: "ZZ") { name } }');
  expect(status).toBe(200);
  expect(body.errors).toBeUndefined();
  expect(body.data.country).toBeNull();
});

test('invalid field returns 200 with a validation error', async ({ request }) => {
  const { status, body } = await gql(request, '{ country(code: "IN") { nam } }');
  expect(status).toBe(200);
  expect(body.data).toBeUndefined();
  expect(body.errors[0].extensions.code).toBe('GRAPHQL_VALIDATION_FAILED');
  expect(body.errors[0].message).toContain('Cannot query field "nam"');
});

test('aliases fetch two records in one request', async ({ request }) => {
  const { body } = await gql(request, '{ in: country(code: "IN") { name } jp: country(code: "JP") { name } }');
  expect(body.data).toEqual({ in: { name: 'India' }, jp: { name: 'Japan' } });
});

These tests ran green against the live API while this lesson was written. For your own APIs, also add schema validation of data (next module) and auth tests per field.

GraphQL Tools for Testers

• Postman / Insomnia: GraphQL body type with schema autocomplete.
• GraphiQL / Apollo Sandbox: in-browser explorer with docs from introspection.
• Playwright, REST Assured, pytest: GraphQL is just HTTP POST, so any API framework works.
• GraphQL Inspector: detects breaking schema changes in CI.
• k6: load testing GraphQL operations.

Mutations: Testing Writes

The countries API is read-only, so here is a typical mutation shape you’ll see in real projects:

mutation AddToCart($input: AddToCartInput!) {
  addToCart(input: $input) {
    cart { id itemCount total }
    errors { field message }         # some APIs return business errors as data
  }
}
# variables: { "input": { "productId": "101", "quantity": 2 } }

Test cases: valid input updates the cart (verify with a follow-up query); quantity 0 / -1 / 1000 returns an error in errors or the typed errors field; unknown product; another user’s cart id (authorization); running the same mutation twice (idempotency where expected).

Common Mistakes When Testing GraphQL

• Asserting only status 200.
• Building queries with string concatenation instead of variables.
• Testing only the fields the UI uses; other clients may request more.
• Ignoring null in nested data (partial failures).
• Not testing depth/complexity limits and introspection settings.

Interview Questions: GraphQL Testing

How to Report API Bugs: Template, Examples and Severity

Finding an API bug is half the job. The other half is reporting it so clearly that a developer can reproduce it in one minute, understand the impact and fix the right thing. A vague ticket like “Order API not working” bounces back and forth for days; a precise one with a cURL command gets fixed the same day. In this lesson you’ll learn the API bug report template that developers love, how to judge severity and priority, and see real examples, including genuine issues found during this course.

Why API Bug Reports Are Different

UI bugs can be shown with a screenshot. API bugs live in requests and responses that nobody can see unless you capture them. That means:

• The exact request matters: one header or one field type changes everything.
• The exact response (status, headers, body) is the evidence.
• Backend logs are searchable by request id and timestamp, so include them.
• The same API serves web, Android, iOS and partners, so state which client or whether it’s reproducible directly.

The API Bug Report Template

Writing a Great Title

A strong title lets a lead triage the bug without opening it.

Capturing the Request Correctly

• Postman: Code (</>) → cURL. Make sure variables are resolved (check the Console).
• Browser: DevTools → Network → right-click → Copy as cURL. Or save a HAR file for multi-request issues.
• Playwright: attach request and response to the test report, or use the trace viewer.
• Mobile: proxy the app through Charles, Proxyman or mitmproxy on a test device.

Example 1: Validation Bug (Full Report)

Title:        POST /objects returns 500 when "ttl" is a JSON array instead of 422

Environment:  QA - https://qa.shop.example/api/v1, build 2026.11.3, 12-Nov-2026 15:20 IST
Preconditions: none (endpoint is public)

Request:
curl -i -X POST "https://qa.shop.example/api/v1/objects" \
  -H "Content-Type: application/json" \
  -d '{"name":"Bag","ttl":[60]}'

Actual:
HTTP/1.1 500 Internal Server Error
X-Request-Id: 9f2c1d7e
{"error":"Unhandled exception"}

Expected:
HTTP/1.1 422 with error.details[0].field = "ttl"
(OpenAPI: NewObject.ttl is integer 10-1800; API guidelines section 4: invalid input -> 422)

Reproducibility: 5/5. Same result for ttl = {}, true.
Severity: Medium (unhandled exception; no data impact)   Priority: P2
Impact: Any client sending a wrong type gets a 500; error monitoring is flooded.
Notes:  ttl = "60" (string) correctly returns 422, so only non-scalar types fail.

Notice the extra analysis in Notes. It narrows the cause to “non-scalar types” and saves the developer time.

Example 2: Security Bug (BOLA)

Title:        GET /orders/{id} returns other customers' orders (BOLA / IDOR)

Environment:  Staging, build 2026.11.2
Preconditions: Customer A (qa_cust_a) and Customer B (qa_cust_b), both role=customer.
               Order 70412 belongs to B.

Request (as A):
curl -i "https://staging.shop.example/api/v1/orders/70412" -H "Authorization: Bearer <TOKEN_CUSTOMER_A>"

Actual:   200 OK with B's name, phone, address and items.
Expected: 404 (or 403) and no data. (Security requirement SEC-12: users access only their own orders.)

Reproducibility: Always. Order ids are sequential, so any order can be read.
Severity: Critical   Priority: P1 (blocker for release)
Impact:   Personal data of all customers exposed to any logged-in user. Data protection risk.
Notes:    PATCH /orders/70412 as A returns 403 correctly, so only the read path lacks the ownership check.

Security bugs often go into a restricted ticket or a separate security queue. Follow your company’s disclosure process.

Example 3: Real Findings From This Course

While writing these lessons we tested real public services. Two genuine, reportable observations:

Stack trace in a public SOAP service

Title:   Divide with intB=0 returns SOAP Fault exposing stack trace and server file path
Service: http://www.dneonline.com/calculator.asmx (public demo calculator)
Actual:  HTTP 500, faultcode soap:Server, faultstring contains
         "System.OverflowException ... at Calculator.Divide(Int32 intA, Int32 intB) in G:\PleskVhosts\...\Calculator.asmx:line 19"
         plus header X-AspNet-Version: 2.0.50727
Expected: soap:Client fault with a safe message such as "intB must not be 0"; no stack trace, paths or version headers.
Severity: Medium (information disclosure, OWASP API8)

Inconsistent error format for unknown versions

The SQA Tools practice API returns its own error shape {"error":{"status","code","message"}} everywhere, except for unknown routes such as a wrong version, where the underlying framework answers with a different shape:

curl -i \
  "https://sqatools.in/wp-json/sqa-api/v1/../v9/users"

GET https://sqatools.in/wp-json/sqa-api/v9/users
Actual:   404 {"code":"rest_no_route","message":"No route was found ...","data":{"status":404}}
Expected: 404 in the documented format {"error":{"status":404,"code":"not_found","message":"..."}}
Severity: Low   Priority: P3
Impact:   Clients that parse error.code break on this path.

Low severity, but exactly the kind of consistency issue a careful tester reports.

Severity vs Priority

Before You Click “Create”: Self-Check

1. Did I reproduce it at least twice, ideally with a fresh token and fresh data?
2. Is it really a bug? Check the spec, the story and recent changes. If the spec is unclear, raise a question rather than a bug.
3. Is it a duplicate? Search existing tickets by endpoint.
4. Did I isolate the minimum request that fails (remove unrelated fields)?
5. Did I include request id/timestamp so logs can be found?
6. Did I remove secrets and personal data?
7. Is the expected result backed by a reference, not my opinion?

Automation Failures vs Real Bugs

When an automated API test fails, triage before reporting:

• Product bug: the API changed behaviour against the spec. Report with the request from the test report.
• Test bug: wrong expectation, hardcoded data, order dependency. Fix the test.
• Environment issue: service down, expired test credentials, data reset. Notify the environment owner, don’t log a product bug.
• Flaky: passes on rerun. Investigate timing, shared data and rate limits; don’t just add retries.

Playwright makes evidence easy: attach the response to the report on failure.

test('POST /objects returns 201', async ({ request }, testInfo) => {
  const res = await request.post(`${BASE}/objects`, { data: { name: 'Bag', ttl: 60 } });
  if (res.status() !== 201) {
    await testInfo.attach('response', {
      body: JSON.stringify({ status: res.status(), headers: res.headers(), body: await res.text() }, null, 2),
      contentType: 'application/json',
    });
  }
  expect(res.status()).toBe(201);
});

Communicating With Developers

• Be factual: “returns 500” not “the API is broken”.
• Offer analysis, not blame: “fails only for non-scalar ttl values” helps more than “validation is missing everywhere”.
• For critical bugs, message the developer or lead directly as well as filing the ticket.
• Retest promptly when fixed, on the same data plus related cases (fixes often break neighbours).
• Close the loop: add an automated regression test so it can’t come back.

Interview Questions: API Bug Reporting

Swagger and OpenAPI for Testers: Read Specs and Generate Tests

On most projects the first thing you get for a new API is a link to its Swagger page. Testers who can read an OpenAPI specification properly start testing on day one, spot gaps in the contract before code is written, and generate collections, mocks and even test cases from it in minutes. This lesson teaches you to read and use OpenAPI as a tester, with a real specification for part of the SQA Tools practice API.

Swagger vs OpenAPI: Names Explained

So “send me the Swagger” usually means “send me the OpenAPI file or the Swagger UI link”. The official demo is the Petstore at petstore3.swagger.io; open it in a new tab to explore while you read.

Anatomy of an OpenAPI File

Here is an OpenAPI 3.0 description of three practice API endpoints. Read it top to bottom:

openapi: 3.0.3
info:
  title: SQA Tools Practice API
  version: 1.0.0
servers:
  - url: https://sqatools.in/wp-json/sqa-api/v1
paths:
  /users/{id}:
    get:
      summary: Get one user
      parameters:
        - name: id
          in: path
          required: true
          schema: { type: integer, minimum: 1 }
      responses:
        '200':
          description: The user
          content:
            application/json:
              schema:
                type: object
                properties:
                  data: { $ref: '#/components/schemas/User' }
        '400': { $ref: '#/components/responses/Error' }
        '404': { $ref: '#/components/responses/Error' }
  /objects:
    post:
      summary: Create a temporary object
      requestBody:
        required: true
        content:
          application/json:
            schema: { $ref: '#/components/schemas/NewObject' }
      responses:
        '201':
          description: Created
          headers:
            Location: { schema: { type: string, format: uri } }
            ETag:     { schema: { type: string } }
        '413': { $ref: '#/components/responses/Error' }
        '422': { $ref: '#/components/responses/Error' }
  /auth/me:
    get:
      security: [ { bearerAuth: [] } ]
      responses:
        '200': { description: Current user }
        '401': { $ref: '#/components/responses/Error' }
components:
  securitySchemes:
    bearerAuth: { type: http, scheme: bearer, bearerFormat: JWT }
  schemas:
    User:
      type: object
      required: [id, name, username, email, role, city, age, active, created_at]
      additionalProperties: false
      properties:
        id:         { type: integer, minimum: 1 }
        name:       { type: string }
        username:   { type: string }
        email:      { type: string, format: email }
        role:       { type: string, enum: [admin, editor, viewer] }
        city:       { type: string }
        age:        { type: integer, minimum: 18 }
        active:     { type: boolean }
        created_at: { type: string, format: date-time }
    NewObject:
      type: object
      required: [name]
      additionalProperties: false
      properties:
        name: { type: string, minLength: 1, maxLength: 100 }
        data: { type: object }
        ttl:  { type: integer, minimum: 10, maximum: 1800, default: 600 }
  responses:
    Error:
      description: Error
      content:
        application/json:
          schema:
            type: object
            properties:
              error:
                type: object
                required: [status, code, message]

Reading it as a tester

Using Swagger UI to Test

1. Open the Swagger UI link. Operations are grouped by tags.
2. Click Authorize and paste your token or key once; it’s applied to secured calls.
3. Expand an operation → Try it out → fill parameters/body → Execute.
4. Swagger UI shows the cURL command, request URL, response code, body and headers.
5. Compare the actual response with the documented one below it. Every mismatch is a bug in the API or in the docs.

Finding Bugs in the Spec Itself (Shift-Left)

Reviewing the spec before development is one of the highest-value things a tester can do. Questions to ask:

• Missing limits: name: { type: string } with no maxLength. What happens with 1 MB?
• Missing error responses: only 200 documented. What about 400, 401, 404, 422, 429?
• Ambiguous types: money as number (float) instead of integer paise or a decimal string.
• Inconsistent naming: userId in one place, user_id in another.
• Missing pagination on list endpoints that will grow.
• Security gaps: sensitive endpoints without security.
• Enum changes: are new values allowed later without breaking mobile apps?

Spec vs Reality: Contract Testing

The spec is only useful if the API matches it. Compare them:

• Schema validation of every response against the spec’s schemas (Ajv in Playwright, jsonSchema in Postman).
• Undocumented endpoints: list what’s deployed and compare. The practice API exposes a discovery endpoint you can diff against docs:

curl -i \
  "https://sqatools.in/wp-json/sqa-api/v1/endpoints"

• Undocumented status codes: the API returns 409 but the spec never mentions it.
• Breaking changes between versions: tools like openapi-diff or oasdiff compare two spec files and flag removed endpoints, new required fields or changed types. Run them in CI on every pull request that touches the spec.

Generating Things from the Spec

# Mock the API locally from the spec
npx @stoplight/prism-cli mock openapi.yaml          # serves on http://127.0.0.1:4010

# Fuzz the real (test) API from the spec
pip install schemathesis
schemathesis run openapi.yaml --url https://qa.example.com/api --checks all

# Detect breaking changes between two versions of the spec
oasdiff breaking old-openapi.yaml new-openapi.yaml

Using the Spec in Playwright Tests

import { test, expect } from '@playwright/test';
import Ajv from 'ajv';
import addFormats from 'ajv-formats';
import fs from 'node:fs';
import YAML from 'yaml';

const spec = YAML.parse(fs.readFileSync('openapi.yaml', 'utf8'));
const ajv = new Ajv({ allErrors: true, strict: false });
addFormats(ajv);
const validateUser = ajv.compile(spec.components.schemas.User);
const BASE = spec.servers[0].url;

test('GET /users/{id} matches the OpenAPI User schema', async ({ request }) => {
  for (const id of [1, 2, 3]) {
    const res = await request.get(`${BASE}/users/${id}`);
    expect(res.status()).toBe(200);
    const ok = validateUser((await res.json()).data);
    expect(ok, JSON.stringify(validateUser.errors)).toBe(true);
  }
});

test('every documented error code for /users/{id} is reachable', async ({ request }) => {
  const documented = Object.keys(spec.paths['/users/{id}'].get.responses);   // ['200','400','404']
  const seen = new Set([
    (await request.get(`${BASE}/users/1`)).status(),
    (await request.get(`${BASE}/users/abc`)).status(),
    (await request.get(`${BASE}/users/999`)).status(),
  ].map(String));
  expect([...seen].sort()).toEqual(documented.sort());
});

The tests now fail automatically when the API drifts from its contract or the contract changes without the tests.

Writing Spec-Based Test Cases: A Quick Recipe

1. For each operation, list every documented response code: one test each.
2. For each parameter and body field: required/optional, type, format, min/max, enum → negative and boundary tests.
3. For each security requirement: no token, invalid token, wrong role.
4. For each response schema: schema validation on positive tests.
5. For each response header: assertion.
6. Then add what the spec can’t express: business rules, state transitions, cross-field rules, performance.

Common Mistakes

• Trusting the spec blindly. It’s often out of date; the tests should prove it.
• Only using Swagger UI for manual clicks and never importing the spec into tools.
• Ignoring required and nullable differences (missing vs null).
• Not versioning the spec in Git alongside the code.

Interview Questions: Swagger and OpenAPI

Negative and Boundary Value Testing for APIs with Real Examples

Any developer can make an API work with perfect input. Your job as a tester is to find what happens with everything else: the empty field, the age of 17, the price of minus one, the 5 MB name, the emoji, the duplicated key. This is where most API bugs and many security issues live. In this lesson you’ll learn a systematic method for negative and boundary testing, and run more than 40 real cases against a live validation endpoint.

Why Negative Testing Matters So Much

• Users make mistakes, old app versions send old formats, partner systems send surprises, and attackers send everything.
• Correct rejection protects data integrity: a bad request must not half-save.
• Clear error messages reduce support tickets and help front-end developers show the right field error.

The Validation Endpoint We’ll Test

POST /validate/user on the practice API validates a user payload without saving it. Its rules:

curl -i \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"name":"A","email":"asha@","age":"30","role":"manager","salary":1}' \
  "https://sqatools.in/wp-json/sqa-api/v1/validate/user"

One request, five errors, all reported together with the field name. That’s the behaviour to expect (and to ask for) from any API.

Technique 1: Boundary Value Analysis

Developers commonly write < where they meant <=. Boundaries catch those off-by-one bugs.

Other boundaries to look for in real APIs: dates (today, past, far future, Feb 29, time zones), list sizes (0 items, 1, max items in a cart), money (0, 0.01, max transaction limit), file sizes and counts (5th vs 6th saved address).

Technique 2: Equivalence Partitioning by Data Type

For each field, test one value from every “class” of wrong input:

Do the same for strings (empty, spaces only, very long, Unicode, emoji, HTML, SQL), enums (wrong value, wrong case “Admin”), and formats (email without domain, phone with country code, date in the wrong format).

Technique 3: Structural (Request-Level) Negative Tests

These break the request itself, not one field:

curl -i \
  -X POST \
  -H "Content-Type: application/json" \
  -d '[{"name":"Asha"}]' \
  "https://sqatools.in/wp-json/sqa-api/v1/validate/user"

Technique 4: Cross-Field and Business Rule Violations

• min_price greater than max_price: the practice API returns 400 “min_price cannot be greater than max_price”.
• End date before start date in a booking.
• Coupon valid only above Rs 500 applied to a Rs 499 cart.
• Delivery pincode outside the service area.
• Body id different from the URL id in a PUT (practice API: 422).
• Transfer amount above the account balance or the daily limit.

curl -i \
  "https://sqatools.in/wp-json/sqa-api/v1/products?min_price=5000&max_price=100"

Technique 5: Special Characters, Unicode and Injection Strings

What a Good Error Response Looks Like

{
  "error": {
    "status": 422,
    "code": "validation_failed",
    "message": "2 field(s) failed validation.",
    "details": [
      { "field": "age",   "message": "age is required: a whole number from 18 to 100 (not a string)." },
      { "field": "phone", "message": "phone must be a 10-digit Indian mobile number starting with 6-9." }
    ]
  }
}

Check every negative response for:

• The specific status code (not a generic 400 for everything, never 200, never 500).
• A machine-readable code and the field name, so apps can highlight the right input.
• All errors at once, not one per attempt.
• A human-readable message without internals (no SQL, stack traces or class names).
• The same error format on every endpoint.
• No side effects: nothing created, updated, charged or emailed.

Automating 40+ Negative Cases with Playwright

import { test, expect } from '@playwright/test';

const URL = 'https://sqatools.in/wp-json/sqa-api/v1/validate/user';
const valid = { name: 'Asha', email: 'asha@example.com', age: 30 };

type Case = { title: string; body: Record<string, unknown>; status: number; field?: string };

const cases: Case[] = [
  // Boundaries
  { title: 'age 17',  body: { ...valid, age: 17 },  status: 422, field: 'age' },
  { title: 'age 18',  body: { ...valid, age: 18 },  status: 200 },
  { title: 'age 100', body: { ...valid, age: 100 }, status: 200 },
  { title: 'age 101', body: { ...valid, age: 101 }, status: 422, field: 'age' },
  { title: 'name 1 char',   body: { ...valid, name: 'A' },            status: 422, field: 'name' },
  { title: 'name 2 chars',  body: { ...valid, name: 'As' },           status: 200 },
  { title: 'name 50 chars', body: { ...valid, name: 'a'.repeat(50) }, status: 200 },
  { title: 'name 51 chars', body: { ...valid, name: 'a'.repeat(51) }, status: 422, field: 'name' },
  // Types
  { title: 'age as string', body: { ...valid, age: '30' }, status: 422, field: 'age' },
  { title: 'age decimal',   body: { ...valid, age: 30.5 }, status: 422, field: 'age' },
  { title: 'age null',      body: { ...valid, age: null }, status: 422, field: 'age' },
  { title: 'name spaces',   body: { ...valid, name: '   ' }, status: 422, field: 'name' },
  // Formats and enums
  { title: 'email no domain',  body: { ...valid, email: 'asha@' },          status: 422, field: 'email' },
  { title: 'unknown role',     body: { ...valid, role: 'manager' },        status: 422, field: 'role' },
  { title: 'phone with +91',   body: { ...valid, phone: '+919876543210' }, status: 422, field: 'phone' },
  { title: 'valid phone',      body: { ...valid, phone: '9876543210' },    status: 200 },
  { title: 'unknown field',    body: { ...valid, salary: 1 },              status: 422, field: 'salary' },
  { title: 'unicode name',     body: { ...valid, name: 'आशा वर्मा' },       status: 200 },
];

for (const c of cases) {
  test(`validate/user: ${c.title} -> ${c.status}`, async ({ request }) => {
    const res = await request.post(URL, { data: c.body });
    expect(res.status()).toBe(c.status);
    if (c.field) {
      const fields = (await res.json()).error.details.map((d: any) => d.field);
      expect(fields).toContain(c.field);
    }
  });
}

test('malformed JSON -> 400 without internals', async ({ request }) => {
  const res = await request.post(URL, { headers: { 'Content-Type': 'application/json' }, data: '{"name": "Asha", ' });
  expect(res.status()).toBe(400);
  expect(await res.text()).not.toMatch(/exception|stack|\.php/i);
});

test('wrong content type -> 415', async ({ request }) => {
  const res = await request.post(URL, { headers: { 'Content-Type': 'text/plain' }, data: 'name=Asha' });
  expect(res.status()).toBe(415);
});

Each row is a test with a readable name in the report, and adding a case is one line.

Negative Testing Checklist

1. Every required field missing, one at a time, and all together.
2. Every field: wrong type, null, empty, too short, too long, boundary values.
3. Every enum: unknown value, wrong case.
4. Every format: invalid email, phone, date, URL, UUID.
5. Body level: malformed, empty, array, oversized, duplicate keys, unknown fields.
6. Headers: wrong or missing Content-Type, Accept, auth.
7. Cross-field and business rules.
8. Unicode, special characters and injection strings.
9. Responses: correct 4xx, clear field errors, no internals, no side effects.

Interview Questions: Negative and Boundary API Testing

SoapUI REST Testing, Property Transfer and Data-Driven Tests

SoapUI isn’t only for SOAP. Many enterprise QA teams use one SoapUI project for both their legacy SOAP services and their newer REST APIs. In this lesson you’ll build a REST project against the live SQA Tools practice API, assert JSON with JSONPath, pass a login token between steps with Property Transfer, write small Groovy scripts and run a data-driven test from a CSV file, all with the free open-source edition.

Create a REST Project

1. File → New REST Project.
2. URI: https://sqatools.in/wp-json/sqa-api/v1/users/1 and click OK.
3. SoapUI splits it into a service (endpoint https://sqatools.in), a resource (/wp-json/sqa-api/v1/users/1), a method (GET) and a request.
4. Click the green arrow. Switch the response view to JSON; check Raw for status and headers.

Alternatively, Import an OpenAPI/Swagger file (File → Import Swagger/OpenAPI, or via the plugin in some versions) to create every resource at once.

curl -i \
  "https://sqatools.in/wp-json/sqa-api/v1/users/1"

Better structure with parameters

Edit the resource path to /wp-json/sqa-api/v1/users/{id}. SoapUI adds a TEMPLATE parameter id. Add QUERY parameters such as role, page, per_page on a /users resource. Parameter styles: QUERY, TEMPLATE (path), HEADER, MATRIX, PLAIN.

Create a TestSuite for the REST API

Right-click a request → Add to TestCase, as you did for SOAP.

JSON Assertions in SoapUI

For older SoapUI versions without JsonPath assertions, SoapUI also exposes JSON responses as XML, so XPath Match works on them too (e.g. //data/name).

Script Assertion (Groovy) for business rules

import groovy.json.JsonSlurper

def json = new JsonSlurper().parseText(messageExchange.responseContent)
def order = json.data

order.items.each { item ->
    assert item.line_total == item.quantity * item.unit_price
}
def subtotal = order.items.sum { it.line_total }
assert order.summary.subtotal == subtotal
assert Math.abs(order.summary.total - (subtotal * 1.18)) < 0.01
assert messageExchange.timeTaken < 2000

Attach it to a GET /orders/5001 step: it proves the totals are internally consistent, which no simple Contains check can.

Property Transfer: Chain Login → Protected Call

1. Step 1, REST Request “Login”: POST /wp-json/sqa-api/v1/auth/login, media type application/json, body:

   { "username": "${#Project#username}", "password": "${#Project#password}" }

2. Add TestCase property token (empty).
3. Step 2, Property Transfer: Source = Login, Property = Response, path language JSONPath, path $.access_token. Target = TestCase, Property = token.
4. Step 3, REST Request “Me”: GET /wp-json/sqa-api/v1/auth/me, add header Authorization = Bearer ${#TestCase#token}.
5. Assert on Step 3: status 200, JsonPath $.data.role = admin.

Set project properties username = admin and password = admin@123 (these are public practice credentials). For real systems, pass passwords at run time with -P on the command line instead of saving them in the project file.

Property expansion syntax

Groovy Script Steps: Small Code, Big Wins

A Groovy Script test step can generate data, read files, call other steps and log. Some everyday snippets:

// Unique test data
def uid = UUID.randomUUID().toString().take(8)
testRunner.testCase.setPropertyValue("objectName", "soapui-" + uid)

// Read a property and log it
def token = testRunner.testCase.getPropertyValue("token")
log.info "Token starts with: " + token?.take(12)

// Read the previous step's JSON response
def resp = context.expand('${Create Object#Response}')
def id = new groovy.json.JsonSlurper().parseText(resp).id
testRunner.testCase.setPropertyValue("objectId", id)

// Run another step by name
testRunner.runTestStepByName("Delete Object")

Data-Driven Testing from a CSV (Open Source)

Goal: test the age boundaries of POST /validate/user (18 to 100) from a file.

age,expected
17,422
18,200
100,200
101,422

TestCase “Validation – data driven” with three steps:

1. Validate (REST Request, disabled so it runs only when called): POST /wp-json/sqa-api/v1/validate/user with body

   { "name": "Asha", "email": "asha@example.com", "age": ${#TestCase#age} }

2. Loop (Groovy Script):

   def rows = new File(context.expand('${#Project#dataDir}') + "/ages.csv").readLines().drop(1)
   def failures = []
   rows.each { line ->
       def (age, expected) = line.split(",")*.trim()
       testRunner.testCase.setPropertyValue("age", age)
       def result = testRunner.runTestStepByName("Validate")
       def status = result.messageExchange.responseStatusCode
       log.info "age=${age} -> ${status} (expected ${expected})"
       if (status != (expected as int)) failures << "age ${age}: got ${status}, expected ${expected}"
   }
   assert failures.isEmpty() : failures.join("\n")

The script fails the TestCase with a clear list of mismatches. Add more columns (name length, email format) to cover other fields.

curl -i \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"name":"Asha","email":"asha@example.com","age":101}' \
  "https://sqatools.in/wp-json/sqa-api/v1/validate/user"

Environments and Endpoints

• Use a project property for the host (${#Project#host}) in the service endpoint.
• Override at run time: testrunner.sh -Phost=https://qa.example.com ....
• ReadyAPI adds an Environments switcher similar to Postman’s.

Common SoapUI Problems and Fixes

Running REST Suites in CI

./testrunner.sh \
  -s"SQA Practice REST" \
  -PbaseHost=https://sqatools.in \
  -Ppassword="$QA_PASSWORD" \
  -r -j -f ./reports \
  SQA-Practice-soapui-project.xml

Publish reports/*.xml with your CI’s JUnit report step. A failing assertion returns a non-zero exit code and fails the build.

SoapUI vs Postman vs Playwright for REST

Interview Questions: SoapUI REST and Data-Driven Testing

SoapUI Tutorial: SOAP Web Service Testing Step by Step

Banks, insurers, airlines, telecom operators and government systems still run on SOAP web services, and SoapUI is the tool their QA teams use to test them. In this SoapUI tutorial you’ll install SoapUI, import a real WSDL, send SOAP requests, add assertions (including XPath and SOAP Fault checks), build a test suite and even find a real bug in a public SOAP service. No coding needed.

Where You’ll Meet SOAP in Real Projects

Service companies working for these clients regularly ask for “SoapUI” in QA job descriptions. If you’ve only tested REST, this lesson closes the gap. Background: REST vs SOAP vs GraphQL.

Key SOAP Terms Before You Start

Step 1: Install SoapUI

1. Download SoapUI Open Source from soapui.org (Windows, macOS, Linux). It’s free; ReadyAPI is the commercial edition.
2. Install and launch it. The left panel is the Navigator (project tree); the main area opens editors for requests and tests.

Step 2: Create a SOAP Project from a WSDL

We’ll use a free public demo calculator service:

http://www.dneonline.com/calculator.asmx?WSDL

1. File → New SOAP Project.
2. Project name: Calculator. Initial WSDL: paste the URL above.
3. Keep Create Requests ticked. Click OK.
4. SoapUI reads the WSDL and creates two bindings, CalculatorSoap (SOAP 1.1) and CalculatorSoap12 (SOAP 1.2), each with four operations: Add, Subtract, Multiply, Divide, and a sample request “Request 1” under each.

Public demo services can be slow or occasionally offline. If it doesn’t respond, try again later; the steps are identical for your company’s WSDL.

Step 3: Send Your First SOAP Request

Expand CalculatorSoap → Add and double-click Request 1. SoapUI generated this template:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:tem="http://tempuri.org/">
   <soapenv:Header/>
   <soapenv:Body>
      <tem:Add>
         <tem:intA>?</tem:intA>
         <tem:intB>?</tem:intB>
      </tem:Add>
   </soapenv:Body>
</soapenv:Envelope>

Replace the ? with 5 and 7, then click the green Submit arrow. The response:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" ...>
   <soap:Body>
      <AddResponse xmlns="http://tempuri.org/">
         <AddResult>12</AddResult>
      </AddResponse>
   </soap:Body>
</soap:Envelope>

Use the tabs on the response side: XML (body), Raw (HTTP status and headers: HTTP/1.1 200 OK, Content-Type: text/xml) and the time in milliseconds at the bottom.

The same call with cURL, so you see what SoapUI sends under the hood:

curl -i -X POST "http://www.dneonline.com/calculator.asmx" \
  -H "Content-Type: text/xml; charset=utf-8" \
  -H 'SOAPAction: "http://tempuri.org/Add"' \
  -d '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://tempuri.org/"><soapenv:Body><tem:Add><tem:intA>5</tem:intA><tem:intB>7</tem:intB></tem:Add></soapenv:Body></soapenv:Envelope>'

Step 4: Build a Test Suite

Requests under the interface are for exploring. Real tests live in test suites:

1. Right-click the Add request → Add to TestCase → create TestSuite Calculator Regression and TestCase Add - positive.
2. The request becomes a TestStep. Add more steps for Subtract, Multiply and Divide the same way.

Step 5: Add Assertions

Open a test step, click the Assertions tab at the bottom, then +:

XPath Match assertion

Click Declare to let SoapUI add the namespaces, then write:

declare namespace soap='http://schemas.xmlsoap.org/soap/envelope/';
declare namespace ns1='http://tempuri.org/';
//ns1:AddResponse/ns1:AddResult

Expected result: 12. Namespace-free alternative that’s handy when prefixes change: //*[local-name()='AddResult'].

Step 6: Negative Tests and SOAP Faults

Create a TestCase Divide - negative and send intA = 5, intB = 0. The real response:

HTTP/1.1 500 Internal Server Error
Content-Type: text/xml; charset=utf-8
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET

<soap:Fault>
  <faultcode>soap:Server</faultcode>
  <faultstring>System.Web.Services.Protocols.SoapException: Server was unable to process request.
    ---> System.OverflowException: Arithmetic operation resulted in an overflow.
    at Calculator.Divide(Int32 intA, Int32 intB) in G:\PleskVhosts\...\Calculator.asmx:line 19 ...</faultstring>
  <detail />
</soap:Fault>

Add assertions SOAP Fault and Valid HTTP Status Codes = 500. Then look closer, as a tester should. This demo service shows three issues you’d report in a real project:

1. Information disclosure (security): the faultstring exposes a .NET stack trace, internal class names and the server’s file path. The X-AspNet-Version header reveals the framework version. (OWASP API8, Security Misconfiguration.)
2. Wrong fault code: dividing by zero is a client input problem, so it should be soap:Client with a clear message such as “intB must not be 0”, not soap:Server.
3. Misleading message: “overflow” instead of “division by zero”.

More negative cases to add: non-numeric input (abc, returns a fault), integer overflow (2147483647 + 1, also a fault), missing elements, empty body, wrong SOAPAction, wrong namespace, and malformed XML. Add a Not Contains assertion for “Exception” and “line” on every negative case to catch stack-trace leaks automatically.

Step 7: Properties: Stop Hardcoding Values

Define values once and reuse them with ${...} expansions:

• Project property endpoint = http://www.dneonline.com/calculator.asmx, then set the request endpoint to ${#Project#endpoint}, so switching environments is one change.
• TestCase property a = 20, b = 5, then the request uses <tem:intA>${#TestCase#a}</tem:intA>.
• Expected values in XPath assertions can also use properties.

Passing values between steps (for example a session id from Login into GetBalance) uses Property Transfer, covered in the next lesson with REST examples: SoapUI REST and Data-Driven Testing.

Step 8: Run Suites and Read Results

• Double-click the TestSuite and press the green Run button; each case turns green or red.
• Open a failed step: the assertion panel shows exactly which assertion failed and why.
• The TestCase Log shows timings per step; Raw tabs show the full HTTP exchange.

Command line and CI

# SoapUI ships testrunner.sh / testrunner.bat in its bin folder
./testrunner.sh -s"Calculator Regression" -r -j -f ./reports Calculator-soapui-project.xml
#  -s  test suite to run      -r  print summary report
#  -j  JUnit-style XML reports -f  output folder
#  -P endpoint=https://qa.bank.example/CoreBanking   override a project property

The whole project is one XML file (Calculator-soapui-project.xml), so keep it in Git and run it in Jenkins or GitHub Actions.

Testing SOAP Without SoapUI

SOAP is just XML over HTTP POST, so Postman (raw XML body + headers), cURL and Playwright can call it too. Playwright example from the same service:

import { test, expect } from '@playwright/test';

const URL = 'http://www.dneonline.com/calculator.asmx';
const envelope = (op: string, a: number | string, b: number | string) =>
  `<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://tempuri.org/">
     <soapenv:Body><tem:${op}><tem:intA>${a}</tem:intA><tem:intB>${b}</tem:intB></tem:${op}></soapenv:Body>
   </soapenv:Envelope>`;

async function call(request: any, op: string, a: number | string, b: number | string) {
  return request.post(URL, {
    headers: { 'Content-Type': 'text/xml; charset=utf-8', SOAPAction: `"http://tempuri.org/${op}"` },
    data: envelope(op, a, b),
  });
}

test('Multiply 6 x 7 = 42', async ({ request }) => {
  const res = await call(request, 'Multiply', 6, 7);
  expect(res.status()).toBe(200);
  expect(await res.text()).toMatch(/<MultiplyResult>42<\/MultiplyResult>/);
});

test('Divide by zero returns a SOAP Fault', async ({ request }) => {
  const res = await call(request, 'Divide', 5, 0);
  expect(res.status()).toBe(500);
  expect(await res.text()).toContain('<soap:Fault>');
});

Use SoapUI when the team relies on WSDL-driven work and non-coders; use code when SOAP tests must live alongside REST and UI tests in one framework.

SOAP Testing Checklist

• WSDL loads; operations match the specification.
• Every operation: valid input → correct values (XPath), Not SOAP Fault, schema compliant, within SLA.
• Invalid input → SOAP Fault with correct faultcode (Client vs Server) and a useful message.
• No stack traces, file paths or versions in faults or headers.
• Both SOAP 1.1 and 1.2 bindings if both are published.
• WS-Security: unsigned, expired or tampered messages are rejected.
• XML attacks: XXE and very large or deeply nested XML are rejected.

Interview Questions: SoapUI and SOAP Testing

cURL for API Testing: Commands Every Tester Should Know

cURL is the universal language of API debugging. Developers paste cURL commands into bug tickets, browsers export failing requests as cURL, Postman imports and exports it, and every Linux server and Windows 10+ machine already has it installed. If you can read and write cURL, you can reproduce any API problem in seconds, on any machine, without opening a tool. This lesson gives you every cURL command a tester needs, each runnable against the live practice API.

Why Testers Should Learn cURL

• Exact, shareable reproduction steps in one line.
• Available on servers and CI machines where there’s no GUI.
• Shows the raw truth: no tool silently adding headers.
• Imports straight into Postman (Import → Raw text) and other tools.

Check your install: curl --version. On Windows, use curl.exe in PowerShell (plain curl may be an alias for Invoke-WebRequest in older PowerShell), or use Git Bash.

Your First cURL Commands (GET)

BASE=https://sqatools.in/wp-json/sqa-api/v1

# Body only
curl $BASE/users/1

# Status line + headers + body  (use this by default when testing)
curl -i $BASE/users/1

# Headers only (HEAD request)
curl -I $BASE/users/1

# Query parameters - quote the URL so the shell doesn't treat & as "run in background"
curl -i "$BASE/users?role=admin&sort=age&order=desc"

# Let curl encode tricky values
curl -i -G "$BASE/users" --data-urlencode "q=gupta" --data-urlencode "per_page=5"

curl -i \
  "https://sqatools.in/wp-json/sqa-api/v1/users?role=admin&sort=age&order=desc"

POST, PUT, PATCH, DELETE

# POST JSON (create)
curl -i -X POST "$BASE/objects" \
  -H "Content-Type: application/json" \
  -d '{"name":"cURL object","data":{"qty":1},"ttl":300}'

# POST JSON from a file (great for big payloads)
curl -i -X POST "$BASE/objects" -H "Content-Type: application/json" -d @order.json

# Newer curl (7.82+): --json sets Content-Type and Accept for you
curl -i --json '{"name":"short form","ttl":60}' "$BASE/objects"

# PUT (create/replace with your own id)
curl -i -X PUT "$BASE/objects/curl-demo-1" \
  -H "Content-Type: application/json" \
  -d '{"name":"Replaced","data":{"qty":5},"ttl":300}'

# PATCH (partial update)
curl -i -X PATCH "$BASE/objects/curl-demo-1" \
  -H "Content-Type: application/json" -d '{"data":{"qty":6}}'

# DELETE
curl -i -X DELETE "$BASE/objects/curl-demo-1"

# Form-encoded body (HTML forms, OAuth token endpoints)
curl -i -X POST "$BASE/echo" -d "name=Asha&city=Pune"

# Multipart file upload
curl -i -X POST https://your-app.test/api/upload -F "file=@invoice.pdf" -F "type=invoice"

Headers and Authentication

# Any header
curl -i "$BASE/users/1" -H "Accept: application/xml"

# Basic auth - curl builds the Base64 header for you
curl -i -u sqa_user:sqa@123 "$BASE/basic/profile"

# API key
curl -i -H "X-API-Key: sqa-key-12345" "$BASE/apikey/profile"

# Bearer token: log in, extract the token with jq, then call
TOKEN=$(curl -s -X POST "$BASE/auth/login" -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"admin@123"}' | jq -r .access_token)
curl -i -H "Authorization: Bearer $TOKEN" "$BASE/auth/me"

# Conditional request with an ETag
curl -i "$BASE/products/101" -H 'If-None-Match: "c2aee9e25c4daa9445361e1bb4a749f7"'

# Cookies: save them, then send them back
curl -c cookies.txt -i -X POST https://your-app.test/login -d "user=qa&pass=..."
curl -b cookies.txt -i https://your-app.test/account

jq is a small JSON command-line tool (jq .data.name, jq '.data | length') that pairs perfectly with cURL.

curl -i \
  -H "X-API-Key: sqa-key-12345" \
  "https://sqatools.in/wp-json/sqa-api/v1/apikey/profile"

Debugging with -v and -w

# Verbose: DNS, TLS handshake, every request and response header
curl -v "$BASE/health"
#  > GET /wp-json/sqa-api/v1/health HTTP/2       (lines with > are what you SENT)
#  < HTTP/2 200                                  (lines with < are what you RECEIVED)

# Print only the status code (perfect for scripts)
curl -s -o /dev/null -w "%{http_code}\n" "$BASE/users/999"      # 404

# Timing breakdown: where is the time spent?
curl -s -o /dev/null -w "dns=%{time_namelookup} connect=%{time_connect} tls=%{time_appconnect} ttfb=%{time_starttransfer} total=%{time_total}\n" "$BASE/delay/2"

# Timeouts
curl -i --max-time 2 "$BASE/delay/5"          # curl gives up after 2 s (exit code 28)
curl -i --connect-timeout 5 "$BASE/health"

Redirects, Retries and Other Useful Flags

A 10-Line Smoke Test Script

#!/usr/bin/env bash
BASE=https://sqatools.in/wp-json/sqa-api/v1
fail=0
check() {   # usage: check EXPECTED METHOD PATH [extra curl args...]
  local expected=$1 method=$2 path=$3; shift 3
  local code=$(curl -s -o /dev/null -w "%{http_code}" -X "$method" "$BASE$path" "$@")
  if [ "$code" = "$expected" ]; then echo "PASS $method $path $code"; else echo "FAIL $method $path got $code expected $expected"; fail=1; fi
}
check 200 GET  /health
check 200 GET  /users/1
check 404 GET  /users/999
check 400 GET  "/users?per_page=51"
check 401 GET  /basic/profile
check 200 GET  /basic/profile -u sqa_user:sqa@123
check 405 POST /users
exit $fail

Run it after every deployment; a non-zero exit code fails the pipeline.

Converting cURL to Postman, Playwright and Back

• Browser → cURL: DevTools → Network → right-click request → Copy → Copy as cURL (bash). Remove cookies and tokens before sharing.
• cURL → Postman: Import → Raw text → paste.
• Postman → cURL: the </> Code button → cURL.
• cURL → Playwright: map flags to options:

// curl -i -X POST "$BASE/objects" -H "Content-Type: application/json" -d '{"name":"A","ttl":60}'
const res = await request.post(`${BASE}/objects`, { data: { name: 'A', ttl: 60 } });

// curl -u sqa_user:sqa@123 "$BASE/basic/profile"
await request.get(`${BASE}/basic/profile`, {
  headers: { Authorization: 'Basic ' + Buffer.from('sqa_user:sqa@123').toString('base64') },
});

// curl --max-time 2 "$BASE/delay/5"
await expect(request.get(`${BASE}/delay/5`, { timeout: 2000 })).rejects.toThrow();

// curl "$BASE/redirect"   (no -L)
await request.get(`${BASE}/redirect`, { maxRedirects: 0 });

Common cURL Mistakes

• Forgetting -i and missing the status code and headers.
• Unquoted URLs with &: the shell cuts the URL at the first &.
• Sending JSON without Content-Type: application/json (cURL’s -d defaults to form-encoded); the practice API answers 415.
• Using -X GET together with -d and expecting query parameters (use -G).
• Pasting real tokens and cookies into tickets or chat.
• Using -k in scripts that later run against production.

cURL Cheat Sheet

Interview Questions: cURL