Human Review, Confidence Calibration and Source Provenance

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objectives 5.5 and 5.6

👋 In plain words: '97% accurate' sounds safe enough to stop checking by hand, but it can hide a type of document where accuracy is much worse. This lesson shows how to prove it's safe to reduce human review, and how to keep track of where every claim in a report came from.

“97% accurate” can still be unsafe to automate. The final two task statements cover how to prove accuracy before removing humans, and how to keep track of where every claim came from (Task Statements 5.5 and 5.6).

📖 What You’ll Learn: You will be able to decide when human review can be reduced and keep every claim in a report traceable to its source.

🔑 Key terms

  • Aggregate accuracy: One overall accuracy number across all documents.
  • Segment: A slice of the data, such as handwritten invoices or a single field.
  • Stratified sampling: Checking a random sample from every segment, not just from the whole pile.
  • Calibrated confidence: Confidence scores tested against known correct answers, so that 0.9 really means about 90% right.
  • Provenance: A record of where each claim came from: source, quote and date.

📖 The story in this lesson

ShopNest's invoice extraction is 97% accurate overall, and finance wants to stop human checks. Ravi is asked to prove whether that is safe.

💡 Think of it like this

A 97% class average can hide one student who is failing. Before you stop marking homework, check each student's grades, not just the average.

🚧 Aggregate accuracy hides weak segments

97% overall can mask 70% on handwritten invoices or on one particular field. Before reducing human review, analyse accuracy by document type and by field.

🌍 Real-world example: Handwritten invoices at 70%

Ravi splits accuracy by document type. Typed invoices are 99% accurate, but handwritten ones from small suppliers are only 70%, and the total field on them is often wrong. Human review stays on for handwritten invoices and is reduced for typed ones.

📏 Measuring and routing

  • Stratified random sampling of high-confidence extractions measures ongoing error rates and catches new failure patterns.
  • Field-level confidence scores must be calibrated against a labelled validation set before you set review thresholds.
  • With limited reviewer capacity, route low-confidence and ambiguous or contradictory documents to humans first.

🌍 Real-world example: Spot checks that catch new problems

Every week a random sample from each invoice type, including high-confidence ones, goes to a human. When a supplier changes its layout, the sample catches the new errors within days. Low-confidence and contradictory invoices always go to reviewers first.

🔗 Provenance in multi-source synthesis

  • Summaries drop attribution — require claim → source mappings (URL or document, excerpt, date) that synthesis must preserve and merge.
  • Conflicting figures from credible sources: keep both, attributed, and annotate the conflict. Never pick one arbitrarily or average them.
  • Publication and collection dates stop a change over time from being misread as a contradiction.
  • Separate well-established from contested findings in the report.
  • Render content by type: financial data as tables, news as prose, technical findings as structured lists.

🌍 Real-world example: Two sources, two numbers

One report says a carrier's on-time rate is 94% and another says 88%. Instead of picking one or averaging them, ShopNest's research summary shows both, with source and date. The dates reveal the 88% figure is a year older.

🚫 Common beginner mistake: Turning off human review because the overall accuracy looks high. Check accuracy for each document type and each field first.

🛠️ Try it yourself (about 8 minutes)

  1. Take any accuracy number from your work, such as test pass rate or data quality.
  2. Split it by two categories, such as browser and platform.
  3. Find the weakest segment and write down its number.

✅ You are done when: you found at least one segment that is clearly worse than the overall figure, or showed that none is.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. How should you measure error rates among high-confidence extractions on an ongoing basis?

  • A. Run monthly full re-extractions
  • B. Review only low-confidence items
  • C. Assume they're correct
  • D. Use stratified random sampling for human verification
Show answer

Answer: D. It detects drift and new error patterns. C is blind trust. B misses confident errors. A is expensive and doesn't measure accuracy.

Question 2. How should field-level confidence thresholds for routing to human review be set?

  • A. Ask the model for the threshold
  • B. Calibrate them against a labelled validation set
  • C. Pick 0.8 by intuition
  • D. Copy another team's value
Show answer

Answer: B. Calibration needs ground truth. A, C and D are arbitrary.

Question 3. A synthesis report converts quarterly revenue figures, news events and technical findings into uniform paragraphs, and readers struggle. What's better?

  • A. Use JSON for everything
  • B. Render each type appropriately: financial data as tables, news as prose, technical findings as structured lists
  • C. Use bullet points for everything
  • D. Keep everything as prose
Show answer

Answer: B. Match the format to the content type. A, C and D force one uniform format.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Break a sample accuracy figure down by document type and field and find the weakest segment.
    ✅ Done when: Your breakdown shows at least two segments and identifies the weakest by name and accuracy.
  2. Task 2: Design a routing rule that sends low-confidence and contradictory documents to reviewers first.
    ✅ Done when: Your rule uses a calibrated threshold and sends contradictory documents to review regardless of confidence.
  3. Stretch: write a synthesis output that keeps two conflicting statistics with sources and dates.
    ✅ Done when: Both statistics appear with their source and date, and the text says plainly that they conflict.

❓ Frequently Asked Questions

🤔 Is 97% accuracy enough to remove human review?

Not until you check accuracy by document type and field, because the overall figure can hide weak segments.

🤔 What is stratified sampling used for?

Measuring error rates in high-confidence outputs over time and spotting new error patterns.

🤔 How should confidence thresholds be set?

Calibrate them against a labelled validation set, not by intuition.

🤔 What if two sources disagree?

Keep both values with attribution and annotate the conflict rather than picking one.

🏁 Summary

  • Segment accuracy by document type and field before automating.
  • Stratified sampling plus calibrated confidence drive review routing.
  • Keep claim-source-date mappings; preserve and annotate conflicts.
📝 Test yourself: Mock Test 4 · Mock Test 5 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Architect Hands-on Labs: 8 Practical Exercises.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Escalation Rules and Error Propagation in Claude Agents

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objectives 5.2 and 5.3

👋 In plain words: A support bot that hands every case to a human is useless, and one that never does is dangerous. This lesson gives clear rules for when to escalate, and for how helper agents should report failures so nothing goes silently missing.

Escalate too often and you miss your resolution target; escalate too rarely and the agent takes actions it shouldn’t. The exam has a clear model of good escalation and error propagation (Task Statements 5.2 and 5.3).

📖 What You’ll Learn: You will be able to calibrate when an agent escalates and design error reporting that lets a coordinator recover.

🔑 Key terms

  • Escalation: Passing a case from the bot to a human.
  • First-contact resolution: Solving the customer's issue without a handoff.
  • Policy gap: A situation your rules don't cover, such as a request to match a competitor's price.
  • Error propagation: How a failure in one agent is reported up to the agent above it.
  • Coverage gap: Part of a report that is weak because a source was unavailable.

📖 The story in this lesson

Asha's bot escalates a quarter of all chats, mostly because customers sound annoyed, and ShopNest misses its 80% resolution target. Meanwhile a research helper silently returns 'nothing found' when its search actually crashed.

💡 Think of it like this

A good bot is like a good junior colleague. They handle routine work themselves, call the manager when the customer asks for one or when the rulebook has no answer, and never hide a mistake by saying 'all fine'.

📞 Valid escalation triggers

EscalateDon’t escalate — resolve
The customer explicitly asks for a human → immediately, without investigating firstThe customer is frustrated but hasn’t asked for a human, and the issue is within the agent’s capability → acknowledge, resolve, escalate if they then ask
Policy gap or exception — the policy is silent or ambiguous (e.g. competitor price-matching when the policy covers only your own site)Standard in-policy cases (damage replacement with photo evidence)
No meaningful progress is possibleMulti-issue requests that are each routine
⚠️ Watch out: Sentiment analysis and the model’s self-reported confidence are unreliable proxies for case complexity. The proportionate first fix for bad escalation calibration is explicit criteria with few-shot examples in the system prompt — before building classifiers.

🌍 Real-world example: Annoyed is not the same as complex

A customer writes angrily about a late but standard delivery. The bot acknowledges the frustration and resolves it, because that's within policy. A different customer asks 'Can I talk to a person?', and the bot hands over immediately. A third asks ShopNest to match a competitor's price, which the policy doesn't cover, so the bot escalates.

Support agent escalation decision flowCustomer messageAsked fora human?Does policycover it?Can the agentmake progress?noyesEscalate nowwith handoff summaryEscalatepolicy gap / exceptionResolve itacknowledge feelingsyesno / silentyesno → escalate✗ Sentiment and self-reported confidence are not escalation triggers · multiple matches → ask for another ID
Figure: when a support agent should escalate and when it should resolve. Swipe sideways to see the whole diagram.

❓ Ambiguous matches

When a lookup returns several customers, ask for an additional identifier. Never pick one by heuristic (most recent, first result).

🌍 Real-world example: Two customers named Priya Shah

get_customer returns two accounts for 'Priya Shah'. Picking the most recent one could refund the wrong person. The bot asks for the order number or postcode instead, then continues.

🚨 Error propagation in multi-agent systems

  • Return structured error context: failure type, the query attempted, partial results and possible alternatives.
  • Distinguish access failures (timeouts needing a retry decision) from valid empty results.
  • Recover transient failures locally; propagate only what can’t be resolved.
  • Anti-patterns: silently suppressing errors (empty result marked success) and terminating the whole workflow on one failure.
  • Annotate synthesis output with coverage gaps where sources were unavailable.

🌍 Real-world example: 'Nothing found' that was really a crash

The search helper now reports the failure properly: what failed, what it tried, and the partial results it did get. The coordinator retries elsewhere, and the final report marks the section that rests on fewer sources.

{"status": "failed", "type": "timeout", "query": "carrier rates 2026",
 "partial_results": 3, "alternatives": ["use cached rates"]}
🚫 Common beginner mistake: Escalating based on sentiment scores or the model's own confidence rating. Use explicit triggers: the customer asked for a human, a policy gap, or no progress.

🛠️ Try it yourself (about 8 minutes)

  1. Write three 'escalate' rules and three 'resolve yourself' rules for a support bot.
  2. Check that no rule depends on tone of voice or a confidence score.
  3. Add one example message for each rule.

✅ You are done when: you have six rules with examples, and every escalate rule is an explicit trigger.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. A frustrated customer has a standard, in-policy issue and hasn't asked for a human. What should the agent do?

  • A. Escalate immediately
  • B. Acknowledge the frustration and offer to resolve it, escalating if they then ask for a human
  • C. Close the ticket
  • D. Ignore the emotion
Show answer

Answer: B. Frustration alone isn't an escalation trigger. A over-escalates and hurts first-contact resolution. D is poor service. C is wrong.

Question 2. Why are sentiment scores and self-reported confidence poor escalation triggers?

  • A. They're illegal
  • B. They require fine-tuning
  • C. They don't reliably reflect how complex a case actually is
  • D. They're expensive
Show answer

Answer: C. They're proxies that are poorly correlated with complexity, and LLM self-confidence is badly calibrated. A, B and D are irrelevant.

Question 3. What's the purpose of a crash-recovery manifest in a multi-agent system?

  • A. Each agent exports its state to a known location, and on resume the coordinator loads the manifest and injects that state into agent prompts
  • B. Logging prompts for audit
  • C. Billing
  • D. Caching prompts
Show answer

Answer: A. Structured state persistence. B, C and D are other concerns.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Write three escalate and three resolve examples for a support system prompt.
    ✅ Done when: Each example states the trigger, and none of the escalate examples relies on sentiment or a confidence score.
  2. Task 2: Write the structured error a timed-out search subagent should return, including partial results.
    ✅ Done when: Your error includes the failure type, the query attempted, partial results and at least one alternative.
  3. Stretch: add a coverage-gaps section to a sample research report.
    ✅ Done when: Your report has a coverage-gaps section that names the missing source and the affected findings.

❓ Frequently Asked Questions

🤔 When should a support agent escalate immediately?

When the customer explicitly asks for a human.

🤔 Is customer frustration a reason to escalate?

Not on its own. Acknowledge it, resolve the issue if you can, and escalate if they then ask for a human.

🤔 What should the agent do with several matching customers?

Ask for another identifier instead of guessing.

🤔 Why not return an empty result when a search fails?

It hides the failure, so the coordinator cannot retry or report the gap.

🏁 Summary

  • Escalate on explicit request, policy gaps or no progress — not on sentiment or confidence.
  • Multiple matches → ask for another identifier.
  • Propagate structured error context; never hide errors or kill the whole run.
📝 Test yourself: Mock Test 4 · Mock Test 5 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Human Review, Confidence Calibration and Source Provenance.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Claude Context Management: Case Facts, Lost-in-the-Middle and /compact

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objectives 5.1 and 5.4

👋 In plain words: Claude only knows what is in the current conversation, and in long conversations important details get lost or blurred. This lesson shows simple habits that keep exact amounts, dates and findings accurate from the first message to the last.

Long conversations and long explorations degrade in predictable ways. The exam tests the patterns that keep critical information intact (Task Statements 5.1 and 5.4).

📖 What You’ll Learn: You will be able to keep amounts, dates and findings accurate across long conversations and long codebase explorations.

🔑 Key terms

  • Context window: Everything Claude can 'see' at once: the conversation, tool results and instructions. It has a size limit.
  • Progressive summarisation: Repeatedly shortening old messages to save space, which can blur exact details.
  • Case-facts block: A small, always-included list of the key facts, such as IDs, amounts and dates.
  • Lost in the middle: Models pay less attention to information buried in the middle of a long input.
  • /compact: A Claude Code command that shrinks the conversation to free up space.

📖 The story in this lesson

A ShopNest customer has a 40-message conversation about a refund. By message 35 the bot remembers 'a refund is pending' but has forgotten it was £184.20, promised by Thursday.

💡 Think of it like this

The context window is a desk. Paper piles up, old sheets get summarised onto sticky notes, and details vanish. A case-facts block is the one index card you keep pinned in the middle of the desk, never summarised and always visible.

⚙️ Progressive summarisation loses precision

Rolling summaries turn “a £184.20 refund on order #A-7710 by Thursday” into “a refund is pending”. Keep a persistent case-facts block — amounts, order IDs, dates, statuses, promises and customer-stated constraints — and include it in every prompt, outside the summarised history.

<case_facts>
customer_id: C-1042 (verified)
order: #A-7710 · refund £184.20 approved · promised by Thu 19 Nov
constraint: technician visits after 17:00 only
</case_facts>

🌍 Real-world example: The £184.20 that became 'a refund'

Asha's bot now keeps a case-facts block, outside the summarised history, in every request. Even on message 40, the exact amount, order and deadline are there.

<case_facts>
order: #A-7710 | refund GBP 184.20 approved | promised by Thu 19 Nov
constraint: deliveries after 17:00 only
</case_facts>

🛠️ Trim tool output before it piles up

An order lookup may return 40+ fields when only 5 matter. Trim results to the relevant fields (for example in a PostToolUse hook) before they accumulate in context. Upstream agents should return structured key facts, citations and relevance scores — not long reasoning chains — when downstream context budgets are tight.

🌍 Real-world example: 40 fields when 5 matter

Each lookup_order call returned 40 fields, including warehouse codes and internal flags. After 10 lookups the context was full of noise. A hook now keeps only status, items, total, delivery date and refund eligibility.

🎯 Lost in the middle

Models reliably use information at the beginning and end of long inputs but may miss findings in the middle. Put a key-findings summary first and organise details under explicit section headers. And always send the complete conversation history — the API is stateless.

🌍 Real-world example: The key finding on page 31

A 60-page research summary hid its most important finding in the middle, and the final report missed it. Putting a short 'key findings' list at the top, with clear section headings below, fixed it.

🗂️ Long codebase explorations

  • Symptoms of degradation: inconsistent answers, and talk of “typical patterns” instead of the specific classes found earlier.
  • Scratchpad files record key findings for later questions.
  • Subagents investigate focused questions (“find all test files”) while the main agent keeps the big picture.
  • Phase summaries are injected before the next phase begins.
  • /compact reduces context use when verbose discovery output fills it.
  • Crash-recovery manifests: each agent exports its state to a known location; on resume the coordinator loads the manifest and injects that state into agent prompts.

🌍 Real-world example: Ravi's scratchpad

During a two-hour exploration of the payment service, Claude starts saying 'typical patterns' instead of naming the real classes it found earlier. Ravi has it write findings to notes/scratchpad.md, runs /compact, and Claude carries on with the exact class names.

🚫 Common beginner mistake: Trusting a running summary to keep exact numbers and dates. Keep critical facts in a separate block that is never summarised.

🛠️ Try it yourself (about 8 minutes)

  1. Take a long support or chat thread you have handled.
  2. Write a case-facts block of 5 lines or fewer: ID, amount, date, status, one constraint.
  3. Re-read the thread and check that nothing critical is missing from the block.

✅ You are done when: a colleague could act on the case using only your case-facts block.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. Aggregated subagent results put the key findings in the middle of a 60-page input, and the synthesis misses them. What helps?

  • A. Remove the headers
  • B. Increase the temperature
  • C. Shuffle the order
  • D. Put a key-findings summary at the beginning and organise the details under clear section headers
Show answer

Answer: D. It mitigates the "lost in the middle" effect. A, B and C don't address position effects.

Question 2. A long exploration fills context with noisy grep output. Which built-in command reduces context usage?

  • A. /mcp
  • B. /init
  • C. /compact
  • D. /memory
Show answer

Answer: C. D inspects memory files. B bootstraps CLAUDE.md. A manages MCP servers.

Question 3. A downstream agent has a small context budget. What should upstream agents return?

  • A. Structured key facts, citations and relevance scores
  • B. Raw HTML
  • C. Nothing until asked
  • D. Full reasoning chains
Show answer

Answer: A. It's compact and preserves what synthesis needs. B and D bloat context. C adds round trips.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Design a case-facts block for a support conversation with order ID, amount, deadline and one customer constraint.
    ✅ Done when: Your block keeps exact amounts, IDs, dates and the customer constraint, and fits in under 80 words.
  2. Task 2: Trim a 40-field API response down to the five fields a refund decision needs.
    ✅ Done when: Your trimmed response keeps only the five fields the refund decision needs, with names unchanged.
  3. Stretch: run a long Claude Code exploration with a scratchpad file and use /compact when context fills.
    ✅ Done when: Claude still names the specific classes it found earlier after /compact, using your scratchpad file.

❓ Frequently Asked Questions

🤔 Why do summaries lose important details?

Progressive summarisation compresses exact numbers, dates and promises into vague phrases.

🤔 What is the lost-in-the-middle effect?

Models use the start and end of long inputs more reliably than the middle, so put key findings first.

🤔 What does /compact do?

It reduces context use in Claude Code when verbose output has filled the conversation.

🤔 How do multi-agent systems recover from a crash?

Each agent exports its state to a known location, and the coordinator reloads a manifest on resume.

🏁 Summary

  • Case facts live outside the summary, in every prompt.
  • Trim tool output; summarise key findings first.
  • Scratchpads, subagents, /compact and manifests keep long work accurate.
📝 Test yourself: Mock Test 4 · Mock Test 5 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Escalation Rules and Error Propagation in Claude Agents.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Claude Message Batches API and Multi-Pass Review Design

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objectives 4.5 and 4.6

👋 In plain words: If you have thousands of requests and nobody is waiting for the answers right now, the Message Batches API runs them at half the cost, in exchange for waiting up to 24 hours. This lesson shows when that trade-off makes sense and when it doesn't.

Batch processing questions are pure judgement: when is cheaper-but-slower acceptable? The exam also asks how to design reviews that stay accurate at scale (Task Statements 4.5 and 4.6).

📖 What You’ll Learn: You will be able to decide when batches fit, plan submissions around an SLA, and recover from batch failures efficiently.

🔑 Key terms

  • Synchronous call: A normal request: you send it and wait for the answer.
  • Message Batches API: Send many requests at once. They are processed within 24 hours at a 50% discount.
  • custom_id: Your own label on each request, used to match results, because results can come back in any order.
  • SLA (service-level agreement): A promised deadline, such as 'results within 30 hours'.

📖 The story in this lesson

Every night ShopNest classifies 10,000 product reviews, and every day developers want instant feedback on their pull requests. One of these jobs suits batches, the other does not.

💡 Think of it like this

Batches are like posting parcels by economy mail: half the price, arrives within a day, and parcels may arrive in any order, so you label each one. Synchronous calls are a courier: dearer, but there in minutes when someone is waiting.

📦 Message Batches API facts

  • 50% cost savings compared with synchronous calls.
  • Processing can take up to 24 hours; no latency SLA.
  • Results are correlated with custom_id — they can arrive in any order.
  • No multi-turn tool calling inside a single request (tools can’t run mid-request).
  • You poll for completion.

🌍 Real-world example: 10,000 reviews overnight

ShopNest submits the nightly review-classification job as one batch, each request labelled with a custom_id like 'review-48213'. Results arrive by morning in a jumbled order. They match each result to its review by ID, and the bill is half what synchronous calls would cost.

{"custom_id": "review-48213", "params": {"model": "...", "max_tokens": 200, "messages": [...]}}

📦 Batch or synchronous?

Use batchesUse synchronous calls
Overnight tech-debt reports, weekly audits, nightly test generation, historical backfillsBlocking pre-merge checks, live chat, anything a person is waiting on

🌍 Real-world example: Nightly report yes, PR check no

The nightly tech-debt report and the weekly test-generation job go to batches. The pre-merge check that blocks a developer's pull request stays synchronous, because a developer is sitting there waiting for it.

⏱️ SLA submission maths

If results are due within 30 hours and a batch can take 24 hours, a document arriving just after a submission waits until the next one. Submitting every 4 hours gives a worst case of 4 + 24 = 28 hours — inside the SLA. Every 8 hours (32 h) or 12 hours (36 h) can miss it.

🌍 Real-world example: Meeting a 30-hour promise

Finance needs each invoice processed within 30 hours, and a batch can take up to 24. If ShopNest submits every 4 hours, the worst case is 4 hours waiting plus 24 processing, which is 28 hours: safe. Submitting every 8 hours risks 32 hours: too late.

🚨 Handling failures

Resubmit only the failed custom_ids, with a fix applied (for example, chunk documents that exceeded the context limit). Before a large run, refine the prompt on a representative sample to maximise first-pass success.

🌍 Real-world example: Resubmit only what failed

120 of the 10,000 requests fail because the reviews were too long. ShopNest splits those long reviews into chunks and resubmits only those 120 custom_ids, not the whole batch.

🔍 Multi-instance and multi-pass review

  • Independent reviewer: a second Claude instance without the generator’s reasoning context catches more than self-review instructions or extended thinking in the same session.
  • Multi-pass: per-file passes for local issues plus a cross-file integration pass.
  • Confidence per finding: useful for routing reviewer attention once calibrated — not as ground truth.

🌍 Real-world example: A second opinion for nightly code review

In the nightly code-review batch, one pass checks each file separately and a second pass checks how the files fit together. The reviewing instance is independent of the one that generated the code, so it doesn't just agree with its own earlier reasoning.

🚫 Common beginner mistake: Assuming batch results come back in the order you sent them. Always match them by custom_id.

🛠️ Try it yourself (about 5 minutes)

  1. List three jobs in your team that use or could use an AI model.
  2. For each one, ask: is a person waiting for the answer?
  3. Mark it 'batch' if nobody is waiting and up to 24 hours is fine, otherwise 'synchronous'.
  4. For one batch job, work out a submission cadence for a 30-hour deadline.

✅ You are done when: each job is labelled, and your cadence plus 24 hours stays under 30.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. Which properties apply to the Message Batches API? (Choose two)

  • A. Guaranteed 5-minute latency
  • B. Multi-turn tool calling within a request
  • C. 50% cost savings
  • D. Results returned in submission order
  • E. Up to 24-hour processing window
Show answer

Answer: C, E. A: there's no latency SLA. B: tools can't be executed mid-request. D: results can come back in any order, so correlate with custom_id.

Question 2. Results are due within 30 hours, and batches can take up to 24 hours to process. Which submission cadence safely meets the SLA?

  • A. Every 8 hours
  • B. Every 12 hours
  • C. Every 4 hours
  • D. Once a day
Show answer

Answer: C. The worst case is 4 h waiting plus 24 h processing = 28 h, which is under 30. B (36 h), A (32 h) and D (48 h) can miss the SLA.

Question 3. Why is an independent Claude instance better than self-review for generated code?

  • A. It's cheaper
  • B. Self-review is disallowed
  • C. The generating session keeps its own reasoning context and is less likely to question its own decisions
  • D. It uses a different tokenizer
Show answer

Answer: C. It removes anchoring on the original reasoning. A, B and D are false or irrelevant.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Submit a batch of 20 requests with custom_id values and match the results by ID.
    ✅ Done when: All 20 results are matched to their requests by custom_id, even though they arrive in a different order.
  2. Task 2: Calculate the submission cadence needed for a 30-hour SLA with 24-hour batch processing.
    ✅ Done when: Your answer is every 4 hours or more often, with the worst-case arithmetic written out.
  3. Stretch: resubmit only failed requests after chunking the oversized inputs.
    ✅ Done when: Only the failed custom_ids are resubmitted, and the resubmitted requests succeed.

❓ Frequently Asked Questions

🤔 How much cheaper is the Message Batches API?

About 50% compared with standard synchronous calls.

🤔 How long can a batch take?

Up to 24 hours, with no latency guarantee.

🤔 Can I use batches for pre-merge checks?

No. Developers are waiting on those, so they need synchronous calls.

🤔 How do I match batch results to requests?

Use the custom_id you set on each request, because results can arrive in any order.

🏁 Summary

  • Batches: 50% off, ≤ 24 h, no SLA, custom_id, no mid-request tools.
  • Never put a blocking workflow on batches.
  • Resubmit only failures; refine prompts on a sample first.
📝 Test yourself: Mock Test 3 · Mock Test 5 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Context Management: Case Facts, Lost-in-the-Middle and /compact.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Claude Structured Output with tool_use and JSON Schema

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 8 min read🎯 Official objectives 4.3 and 4.4

👋 In plain words: When another program has to use Claude's answer, you need clean JSON in the exact same shape every time, not a friendly paragraph. This lesson shows how to get reliable JSON and how to design fields so Claude never invents data that isn't in the document.

Structured extraction is a whole exam scenario. You need to know how to guarantee valid JSON, how to design schemas that don’t push Claude to invent data, and when a retry will — and won’t — fix a problem (Task Statements 4.3 and 4.4).

📖 What You’ll Learn: You will be able to get reliable JSON from Claude, design schemas that do not force invented values, and retry only when it helps.

🔑 Key terms

  • Structured output: Claude's answer in a fixed, machine-readable shape such as JSON.
  • JSON schema: A template listing each field, its type, and which fields are required.
  • Nullable field: A field allowed to be null (empty) when the information isn't in the source.
  • Enum: A field limited to a fixed list of values, such as hardware, software or other.
  • Validation-retry loop: Checking the output and, if a rule fails, asking Claude again with the exact error.

📖 The story in this lesson

ShopNest's finance team receives supplier invoices as PDFs. Claude turns each one into JSON for the accounting system. Some invoices have no purchase-order number, and an early version invented one.

💡 Think of it like this

A schema is a paper form with labelled boxes. Claude fills the boxes instead of writing a letter. A nullable box has a 'not on the invoice' tick option. Without that option, a helpful assistant feels pressure to write something in every box.

🛠️ tool_use as the structured-output mechanism

Define an “extraction tool” whose input_schema is your output schema. Claude’s tool call arguments are your structured data — syntactically valid and schema-compliant.

{
  "name": "record_invoice",
  "description": "Record fields extracted from ONE supplier invoice.",
  "input_schema": {
    "type": "object",
    "properties": {
      "vendor": {"type": "string"},
      "invoice_date": {"type": ["string", "null"], "description": "ISO-8601; null if absent"},
      "po_number": {"type": ["string", "null"]},
      "category": {"type": "string", "enum": ["hardware", "software", "services", "other", "unclear"]},
      "category_detail": {"type": ["string", "null"]},
      "stated_total": {"type": "number"},
      "calculated_total": {"type": "number"},
      "conflict_detected": {"type": "boolean"}
    },
    "required": ["vendor", "category", "stated_total", "calculated_total", "conflict_detected"]
  }
}
⚠️ Watch out: Schema compliance ≠ correctness. Tool use removes syntax errors, not semantic ones such as line items that don’t add up or values in the wrong field.

🌍 Real-world example: Invoices into clean JSON

ShopNest defines a record_invoice tool whose input schema is the invoice format. Claude 'calls' the tool, and the tool's arguments are the extracted data, always valid JSON in the right shape. But the totals still need checking: valid JSON can still contain wrong numbers.

📏 Schema design rules

  • Nullable for possibly-absent data. A required field the document doesn’t contain pushes the model to fabricate a value.
  • Enum + “other” + detail string for extensible categories, and “unclear” for genuinely ambiguous cases.
  • Self-check fields such as calculated_total next to stated_total, plus conflict_detected.
  • Format rules in the prompt (dates to ISO-8601, amounts as numbers) alongside the schema.

🌍 Real-world example: The invented PO number

po_number was required, so Claude made one up for invoices that had none. Making it nullable lets Claude return null honestly. A category enum with 'other' plus a detail field handles unusual purchases without forcing a wrong category.

"po_number": {"type": ["string", "null"]},
"category": {"type": "string", "enum": ["hardware", "software", "services", "other", "unclear"]}

🔁 Validation and retry loops

When validation fails, send a follow-up with the original document, the failed extraction and the specific validation error.

Retries fixRetries can’t fix
Wrong date or number formatInformation absent from the provided source
Missing or misplaced keysData that lives only in an external document you didn’t supply
Structural output errors— route these to null + human review

In review pipelines, add a detected_pattern field to findings so you can analyse which code constructs developers keep dismissing.

🌍 Real-world example: Fixing a date format, not a missing value

An extraction returns '12/11/26' where ISO format is required. ShopNest sends back the document, the bad output and the exact error, and the retry returns '2026-11-12'. When the supplier simply left the date off the invoice, retrying can't help, so the field stays null and a human checks it.

Structured extraction validation and retry loopSourcedocumentClaude + extractiontool (JSON schema,nullable fields)Validate schema+ semanticsDownstreamsystem / ERPpassRetry with: original doc +failed output + exact error(format / structure errors)Info missing from source?return null and route tohuman review — don’t retryfixableabsentTool use removes JSON syntax errors — semantic checks (totals, field placement) are still your job
Figure: retry only fixable errors; route missing information to a human. Swipe sideways to see the whole diagram.
🚫 Common beginner mistake: Making every field required 'to be safe'. It pushes Claude to invent values. Make fields that may be missing nullable.

🛠️ Try it yourself (about 10 minutes)

  1. Write a small schema for a receipt: shop name (required), date (nullable), total (required).
  2. Give Claude a receipt text with no date in it.
  3. Ask it to fill the schema using a tool.
  4. Check what comes back for the date.

✅ You are done when: the date comes back as null, not an invented date.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. Which schema design reduces fabrication?

  • A. Make fields that may be missing from the source nullable or optional
  • B. Remove all enums
  • C. Make every field required
  • D. Use free-text fields only
Show answer

Answer: A. Required fields push the model to invent values. C causes exactly that. B and D lose structure.

Question 2. What can a retry with validation-error feedback not fix?

  • A. A value that doesn't exist anywhere in the provided document
  • B. A date in the wrong format
  • C. A string where a number was expected
  • D. A missing required key
Show answer

Answer: A. Absent information can't be produced by retrying, which only invites fabrication. B, C and D are format or structure errors that retries do fix.

Question 3. Why use tool_use with a JSON schema for extraction?

  • A. It reliably produces schema-compliant, syntactically valid structured output
  • B. It removes the need for validation
  • C. It's cheaper than text
  • D. It makes outputs semantically correct
Show answer

Answer: A. B and D are wrong: semantic errors are still possible, so you still validate. C isn't the reason.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Define an extraction tool with one nullable field and an enum with other and unclear values.
    ✅ Done when: Your schema has a nullable field, an enum containing other and unclear, and a detail string for the other value.
  2. Task 2: Process a document missing one field and confirm you get null, not a made-up value.
    ✅ Done when: The missing field comes back as null in three runs out of three.
  3. Stretch: build a validation-retry loop that sends the document, the failed output and the exact error.
    ✅ Done when: A deliberately wrong date format is corrected on the retry, and your log shows the exact error you sent back.

❓ Frequently Asked Questions

🤔 How do I guarantee valid JSON from Claude?

Define a tool whose input schema is your output schema and read Claude's tool_use arguments.

🤔 Does a JSON schema guarantee correct data?

No. It stops syntax errors, but values can still be wrong, such as totals that don't add up.

🤔 Why make fields nullable?

A required field that is missing from the document pushes the model to invent a value.

🤔 When does retrying fail?

When the information is not in the provided source. Return null and route the document to a human.

🏁 Summary

  • tool_use + JSON schema = reliable structure; validate semantics separately.
  • Nullable fields prevent fabrication; enum “other” + detail keeps categories extensible.
  • Retry with specific feedback for format errors; never retry missing information.
📝 Test yourself: Mock Test 3 · Mock Test 5 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Message Batches API and Multi-Pass Review Design.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Prompt Engineering for Claude: Explicit Criteria and Few-Shot Examples

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objectives 4.1 and 4.2

👋 In plain words: If you ask Claude to 'be careful' or 'only report important issues', every run interprets that differently. Clear rules and a few good examples make its output consistent. This lesson shows how to write both, using a code-review bot as the example.

Two prompt-engineering skills dominate the exam: writing explicit criteria instead of vague instructions, and using few-shot examples when instructions alone give inconsistent results (Task Statements 4.1 and 4.2).

📖 What You’ll Learn: You will be able to replace vague instructions with explicit criteria and write few-shot examples that make Claude's output consistent.

🔑 Key terms

  • Prompt: The instructions you give Claude.
  • Explicit criteria: Concrete rules about what to report and what to skip, instead of vague words like 'important'.
  • False positive: A reported problem that isn't really a problem.
  • Few-shot examples: A handful of worked examples in the prompt that show Claude exactly what you want.
  • Severity rubric: Definitions of each severity level, each with an example.

📖 The story in this lesson

Ravi's AI reviewer floods pull requests with comments about naming and harmless code comments. Developers start ignoring it, including its real security warnings.

💡 Think of it like this

Saying 'be conservative' to a new reviewer is like telling a new chef to 'season it well'. Saying 'salt: one teaspoon per litre, never on desserts' and showing them one finished plate gets the same dish every time.

⚙️ Explicit criteria beat vague instructions

Vague (doesn’t improve precision)Explicit (does)
“Check that comments are accurate.”“Flag a comment only when the behaviour it describes contradicts what the code actually does.”
“Be conservative.”“Report: logic bugs, security vulnerabilities, data loss. Skip: formatting, naming, patterns consistent with local conventions.”
“Only report high-confidence findings.”A severity rubric with a concrete code example for each level.
⚠️ Watch out: “Be conservative” and “only high-confidence findings” are listed in the official guide as instructions that fail to improve precision. Pick the option with categorical criteria.

🌍 Real-world example: From 'be careful' to clear rules

Ravi replaces 'only report high-confidence issues' with a report and skip list. False alarms drop by more than half, because Claude no longer has to guess what 'high confidence' means.

REPORT: logic bugs, security holes, data loss.
SKIP: formatting, naming, code that matches local conventions.
Flag a comment only if it contradicts what the code does.

🤝 Protect developer trust

A category with many false positives makes developers ignore all findings, including accurate security ones. Temporarily disable the noisy category while you improve its prompt, then re-enable it.

🌍 Real-world example: Switching off the noisy category

Naming comments were dismissed 80% of the time, and developers stopped reading any comments at all. Ravi temporarily disables the naming category while he improves its prompt. Developers start reading the security findings again.

🧪 Few-shot examples

  • Few-shot examples are the most effective way to get consistently formatted, actionable output when detailed instructions still vary.
  • Use 2–4 targeted examples of ambiguous cases, and show the reasoning for why one option beat the plausible alternatives.
  • Good examples help Claude generalise to new cases rather than just pattern-match.
  • Show the exact output format (location, issue, severity, suggested fix).
  • Contrast acceptable patterns with genuine issues to cut false positives.
  • For extraction, show documents with different structures (inline citations vs bibliography, tables vs narrative) to reduce empty fields and hallucinations.
<example>
Code: user = repo.find(id)  # reflection-based DI injects repo
Finding: none
Reasoning: repo looks unused but is injected via reflection — acceptable pattern.
</example>
<example>
Code: total = price * qty; return round(total)
Finding: {location: "cart.py:42", issue: "rounds currency with float", severity: "high",
          fix: "use Decimal and quantize to 2 places"}
Reasoning: float rounding loses cents in real orders.
</example>

🌍 Real-world example: Three examples beat three paragraphs

The bot's output format kept changing between runs. Ravi adds three short examples: one real bug, and two cases that look suspicious but are fine, each with a line of reasoning. Output becomes consistent and the 'looks odd but is fine' false alarms disappear.

🚫 Common beginner mistake: Adding more and more adjectives ('very', 'only', 'carefully') to fix inconsistent output. Use explicit criteria, and add examples if it is still inconsistent.

🛠️ Try it yourself (about 8 minutes)

  1. Find one vague instruction in a prompt you use, such as 'be thorough'.
  2. Rewrite it as a 'REPORT:' line and a 'SKIP:' line.
  3. Add one short example of the output format you want.
  4. Run it twice and compare the results.

✅ You are done when: both runs use the same format and follow your report and skip rules.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. A review prompt says "check that comments are accurate" and flags many harmless comments. What's the best rewrite?

  • A. "Only report high-confidence issues."
  • B. "Ignore comments."
  • C. "Be conservative."
  • D. "Flag a comment only when the behaviour it describes contradicts what the code actually does."
Show answer

Answer: D. Explicit categorical criteria improve precision. A and C are vague and proven ineffective. B throws away the category entirely.

Question 2. Detailed instructions still give inconsistently formatted findings. What's the most effective technique?

  • A. Lower max_tokens
  • B. A larger model
  • C. More instructions
  • D. Few-shot examples showing the exact format (location, issue, severity, fix)
Show answer

Answer: D. Examples beat added prose for format consistency. A, B and C don't target the cause.

Question 3. The "naming" findings category has an 80% dismissal rate, and developers have started ignoring all findings. What's the best immediate action?

  • A. Raise the severity of everything
  • B. Temporarily disable the naming category while you improve its prompt, to restore trust
  • C. Switch models
  • D. Keep everything and add a disclaimer
Show answer

Answer: B. Noisy categories erode trust in the accurate ones. D doesn't fix the trust problem. C isn't targeted. A makes it worse.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Rewrite a vague review instruction from your team as explicit report and skip criteria.
    ✅ Done when: Your rewrite lists what to report and what to skip, with no words such as 'be careful' or 'high confidence'.
  2. Task 2: Write a severity rubric with one code example per level.
    ✅ Done when: Each severity level has one short code example that a teammate would classify the same way.
  3. Stretch: add three few-shot examples to a prompt and measure the change in format consistency over ten runs.
    ✅ Done when: You recorded format consistency for ten runs before and after adding the examples.

❓ Frequently Asked Questions

🤔 Does 'only report high-confidence findings' reduce false positives?

No. The exam guide notes that vague confidence instructions don't improve precision; explicit categorical criteria do.

🤔 How many few-shot examples should I use?

Two to four targeted examples of ambiguous cases, with the reasoning for each choice.

🤔 Why disable a noisy review category?

High false-positive categories make developers ignore all findings, including accurate ones.

🤔 Do few-shot examples help extraction?

Yes. Examples that cover different document layouts reduce empty and invented fields.

🏁 Summary

  • Replace vague instructions with categorical report/skip criteria.
  • Disable noisy categories temporarily to restore trust.
  • 2–4 targeted few-shot examples with reasoning give consistency and generalisation.
📝 Test yourself: Mock Test 3 · Mock Test 5 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Structured Output with tool_use and JSON Schema.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Claude Code in CI/CD: -p Flag, JSON Output and Automated PR Review

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objective 3.6

👋 In plain words: Claude Code can review every pull request automatically inside your CI pipeline, which is great news for testers. But CI has no human to answer questions, so it must run non-interactively and produce output a script can read. This lesson shows the exact setup.

Running Claude Code in CI is where QA and automation engineers get the most value — and where the exam asks very practical questions about flags and review design (Task Statement 3.6).

📖 What You’ll Learn: You will be able to run Claude Code in a pipeline, get structured findings for PR comments and design reviews developers trust.

🔑 Key terms

  • CI pipeline: Automated jobs (GitHub Actions, Jenkins and so on) that run on every code change.
  • Non-interactive mode: Claude Code takes one prompt, prints the result and exits. You turn it on with -p.
  • JSON output: Machine-readable results, so a script can turn findings into PR comments.
  • JSON schema: A template that fixes the exact shape of the JSON, for example file, line and severity.
  • Independent reviewer: A separate Claude run that didn't write the code, so it reviews with fresh eyes.

📖 The story in this lesson

Ravi adds an AI reviewer to ShopNest's GitHub Actions. On its first day the job hangs for an hour. By the end of this lesson it posts clean, non-repeated comments on every PR.

💡 Think of it like this

Interactive mode is a phone call: Claude may ask you something and wait for your answer. Non-interactive mode is a letter: you send the full request, it sends back a full reply, and nobody waits on the line.

⚙️ Non-interactive mode

claude -p "Review the diff in $DIFF_FILE for bugs and security issues only" \
  --output-format json \
  --json-schema ./review-findings.schema.json
  • -p / --print runs non-interactively: process the prompt, print, exit. Without it, the job hangs waiting for input.
  • --output-format json with --json-schema gives machine-parseable findings you can post as inline PR comments.
⚠️ Watch out: CLAUDE_HEADLESS=true, --batch and redirecting stdin from /dev/null are distractors — not documented ways to run in CI.

🌍 Real-world example: The job that hung for an hour

Ravi's first pipeline ran claude "review this", and Claude sat waiting for input that never came. Adding -p makes it process the prompt, print and exit. Adding JSON output with a schema gives findings his script can post as inline comments.

claude -p "Review the diff for bugs only" \
  --output-format json \
  --json-schema ./findings.schema.json
Claude Code CI pull-request review pipelinePull requestopened orupdatedCI jobchecks out diff+ CLAUDE.mdclaude -p "…"--output-format json--json-schema findingsScript postsinline PRcommentsNew commit → re-run with previous findings in contextreport only new or still-open issues (no duplicates)Independentreviewer instance
Figure: a pull-request review pipeline with Claude Code in non-interactive mode. Swipe sideways to see the whole diagram.

⚙️ Give CI the project context

CI-invoked Claude Code reads the project CLAUDE.md. Document testing standards, what makes a test valuable, available fixtures and review criteria there. For test generation, include the existing test files so Claude doesn’t suggest scenarios you already cover.

🌍 Real-world example: Reviews that know the team's rules

The first reviews flagged ShopNest's custom test fixtures as 'unused'. Ravi documents the fixtures and review criteria in the project CLAUDE.md, which Claude Code also reads in CI. The false alarms stop.

🔍 Independent review instances

The session that generated code keeps its own reasoning context and is less likely to question its own decisions. Review generated code with a separate, independent Claude invocation.

🌍 Real-world example: Don't mark your own homework

When Claude generated a test and then reviewed it in the same session, it approved its own mistakes. Ravi runs the review as a separate Claude invocation that never saw the generation reasoning, and it catches two real bugs.

🔍 Re-running reviews after new commits

Include the earlier findings in context and ask Claude to report only new or still-unaddressed issues. Otherwise every push re-posts the same comments.

🌍 Real-world example: No more 14 duplicate comments

After each push, the reviewer reposted the same 14 comments. Ravi now passes the previous findings into the prompt and asks only for new or still-unfixed issues. Developers see one fresh comment instead of 14 repeats.

🚫 Common beginner mistake: Running Claude Code in CI without -p. The job waits for input that never comes and hangs until it times out.

🛠️ Try it yourself (about 10 minutes)

  1. In a repository with a small change, run claude -p "List possible bugs in this diff" with --output-format json.
  2. Save the output to a file.
  3. Open the file and check it is valid JSON, for example with python -m json.tool.

✅ You are done when: the command exits by itself and the saved file parses as JSON.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. A CI job running claude "review this" hangs. What's the fix?

  • A. CLAUDE_HEADLESS=true
  • B. Pipe in yes
  • C. claude -p "review this"
  • D. claude --batch
Show answer

Answer: C. -p/--print is non-interactive mode. A and D don't exist. B is a hack that doesn't address the mode.

Question 2. After a developer pushes fixes, the re-run review repeats 14 already-resolved comments. What's the best fix?

  • A. Include the prior findings in context and ask for only new or still-unaddressed issues
  • B. Delete the old comments automatically
  • C. Lower max_tokens
  • D. Run reviews weekly
Show answer

Answer: A. Context lets Claude reconcile old and new findings. B loses history and may delete valid comments. D delays feedback. C truncates output.

Question 3. Your CI posts Claude's review findings as inline PR comments. What output configuration should you use?

  • A. Markdown output
  • B. Default text output parsed with regexes
  • C. --verbose
  • D. --output-format json with --json-schema defining the findings structure
Show answer

Answer: D. Machine-parseable and schema-enforced. A and B are brittle. C is for debugging.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Run claude -p with --output-format json on a sample diff and save the output.
    ✅ Done when: The command exits on its own and the saved file is valid JSON.
  2. Task 2: Write a JSON schema for findings with file, line, severity, category and suggested fix, and use it with --json-schema.
    ✅ Done when: The output validates against your schema, and every finding has a file, line and severity.
  3. Stretch: add a GitHub Actions step that posts findings as PR comments and skips issues already reported.
    ✅ Done when: A second run after a new commit posts only new or still-open findings, with no duplicates.

❓ Frequently Asked Questions

🤔 Why does Claude Code hang in my CI job?

It is running interactively. Add -p (or --print) so it processes the prompt, prints and exits.

🤔 How do I get structured output from Claude Code in CI?

Use --output-format json together with --json-schema.

🤔 Should the same session review the code it generated?

No. An independent instance without the generator's context catches more issues.

🤔 How do I avoid duplicate review comments after new commits?

Pass the earlier findings in context and ask only for new or still-unaddressed issues.

🏁 Summary

  • claude -p + --output-format json + --json-schema.
  • CLAUDE.md carries the review criteria and testing standards into CI.
  • Use an independent reviewer; pass earlier findings to avoid duplicate comments.
📝 Test yourself: Mock Test 1 · Mock Test 4 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Prompt Engineering for Claude: Explicit Criteria and Few-Shot Examples.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Claude Code Plan Mode vs Direct Execution and Iterative Refinement

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objectives 3.4 and 3.5

👋 In plain words: For a small fix you just want Claude to make the change. For a big change, letting it edit 40 files before you agree on the approach is risky. This lesson shows when to plan first, when to act directly, and how to steer Claude when its first attempt isn't what you meant.

Choosing the right working mode saves hours of rework. The exam tests when to plan first, when to just make the change, and how to steer Claude when its output isn’t what you meant (Task Statements 3.4 and 3.5).

📖 What You’ll Learn: You will be able to choose plan mode or direct execution for a task and steer Claude with examples, tests and questions.

🔑 Key terms

  • Plan mode: Claude explores and proposes a plan without changing any files until you approve.
  • Direct execution: Claude makes the change straight away.
  • Explore subagent: A helper that does noisy searching separately and returns only a summary.
  • Iterative refinement: Improving the result step by step, using examples, tests or questions.
  • Interview pattern: Asking Claude to question you first, so hidden requirements come out early.

📖 The story in this lesson

Ravi has two jobs: fix a typo in an error message, and move ShopNest's whole test suite from Selenium to Playwright, which touches about 60 files.

💡 Think of it like this

Plan mode is an architect's drawing before builders start: cheap to change on paper, expensive once walls are up. Direct execution is fixing a squeaky hinge: you just do it.

🗺️ Plan mode or direct execution?

Plan modeDirect execution
Large-scale or multi-file changes (e.g. a library migration touching 45+ files)Clear, well-scoped changes
Architectural decisions (monolith → microservices)Single-file bug fix with a stack trace
Several valid approaches to compareAdding one validation check

Combine them: plan the migration, agree the approach, then execute the plan directly.

⚠️ Watch out: “Start in direct execution and switch to plan mode if it gets complex” is wrong when the scenario already tells you the task is complex.

🌍 Real-world example: Typo now, migration after a plan

The typo is one line in one file, so Ravi uses direct execution and it's done in seconds. For the Playwright migration he uses plan mode. Claude proposes the order (shared fixtures first, then page objects, then tests), and Ravi corrects it before a single file changes.

🤖 The Explore subagent

Explore isolates verbose discovery (file listings, grep output) and returns a summary, preserving the main conversation’s context during multi-phase tasks.

🌍 Real-world example: Mapping 60 files without flooding the chat

Before planning the migration, Claude needs to know where Selenium is used. The Explore subagent runs the searches separately and returns a one-page summary, so the main conversation stays focused on the plan.

🔄 Iterative refinement techniques

  • Concrete input/output examples — when prose descriptions are interpreted inconsistently, give 2–3 examples of input and expected output.
  • Test-driven iteration — write tests first (including edge cases and performance), then iterate by sharing failures.
  • The interview pattern — ask Claude to question you first in unfamiliar domains, surfacing issues like cache invalidation and failure modes.
  • Interacting vs independent issues — put interacting problems in one detailed message; fix independent ones one at a time.

🌍 Real-world example: Showing beats telling

Ravi's prose description of the test-naming style keeps being misunderstood. He gives two examples of input and expected output, and Claude gets it right first time. For the date-parsing helper, he writes failing tests first and shares each failure with Claude until all of them pass.

login_with_bad_password  ->  test_login_rejects_bad_password
cart_empty_checkout      ->  test_checkout_blocked_when_cart_empty
🚫 Common beginner mistake: Starting a large, multi-file change in direct execution 'to see how it goes'. If you already know the task is big, plan first.

🛠️ Try it yourself (about 10 minutes)

  1. In Claude Code, ask for a one-line change directly, such as fixing a comment.
  2. Then switch to plan mode and ask for a plan to rename a function used in several files.
  3. Read the plan and correct one step before approving.

✅ You are done when: the small fix is done directly, and the rename happens only after you approved a plan.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. Which tasks suit plan mode? (Choose two)

  • A. Restructuring a monolith into services
  • B. Renaming a local variable
  • C. Choosing between two message-queue integrations with different infrastructure needs
  • D. Adding a single if validation
  • E. Fixing a typo in a log message
Show answer

Answer: A, C. Architectural, multi-approach work. B, D and E are clear, scoped changes suited to direct execution.

Question 2. You're building a caching layer in a domain you don't know well. Which technique surfaces the considerations you haven't thought of?

  • A. The interview pattern: have Claude ask you questions before implementing
  • B. Direct execution
  • C. Few-shot examples
  • D. /compact
Show answer

Answer: A. Claude's questions surface invalidation, consistency and failure modes. B jumps straight to code. C needs known examples. D is irrelevant.

Question 3. Three bugs interact: fixing the parser changes the validator's input, which changes the formatter. How should you raise them?

  • A. One at a time, in separate messages
  • B. Only the first; the rest will resolve themselves
  • C. Through three parallel sessions
  • D. All in one detailed message, explaining the interactions
Show answer

Answer: D. Interacting issues need a joint fix. A and C suit independent issues. B is wishful thinking.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Do a one-line fix in direct execution and a ten-file rename in plan mode, and compare the experience.
    ✅ Done when: You wrote two sentences comparing time spent and rework for the two modes.
  2. Task 2: Give Claude two input and output examples for a data transformation and compare the result with a prose-only request.
    ✅ Done when: The example-driven request gives the expected output on the first or second try, and you noted how many tries prose needed.
  3. Stretch: start an unfamiliar feature with the interview pattern and list the questions Claude asked you.
    ✅ Done when: You listed at least three questions Claude asked and one design change they led to.

❓ Frequently Asked Questions

🤔 When should I use plan mode in Claude Code?

For large, multi-file or architectural changes and when there are several valid approaches to compare.

🤔 When is direct execution better?

For clear, scoped changes such as a single-file bug fix with a stack trace.

🤔 What is the Explore subagent for?

It runs noisy discovery in isolation and returns a summary, keeping the main context clean.

🤔 What is the interview pattern?

Asking Claude to question you before building, so it surfaces design issues you had not considered.

🏁 Summary

  • Plan for architecture and multi-file work; act directly on clear, scoped fixes.
  • Explore keeps discovery noise out of the main context.
  • Examples and tests beat longer prose instructions.
📝 Test yourself: Mock Test 1 · Mock Test 4 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Code in CI/CD: -p Flag, JSON Output and Automated PR Review.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

Claude Code Slash Commands and Skills (SKILL.md Frontmatter)

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 7 min read🎯 Official objective 3.2

👋 In plain words: If your team keeps typing the same long instruction ('review this using our checklist…'), you can save it as a slash command or a skill that anyone can run. This lesson covers where to save them so the team shares them, and how a few settings keep them safe and tidy.

Custom slash commands and skills package repeatable workflows for your team. The exam checks where they live, how they are shared, and how frontmatter controls isolation, permissions and arguments (Task Statement 3.2).

📖 What You’ll Learn: You will be able to create shared commands and skills and control their isolation, permissions and arguments with frontmatter.

🔑 Key terms

  • Slash command: A saved prompt you run by typing /name, for example /review.
  • Skill: A reusable workflow in a SKILL.md file that Claude can run when needed, with its own settings.
  • Frontmatter: Settings at the top of a file, between two --- lines.
  • context: fork: Runs the skill in a separate side conversation, so its output doesn't clutter yours.
  • allowed-tools: Limits which tools the skill may use.

📖 The story in this lesson

Ravi's team reviews every pull request against the same 10-point checklist. He turns it into a shared /review command and an 'explore-codebase' skill for new joiners.

💡 Think of it like this

A slash command is a speed-dial button: one press runs a long instruction. A skill is a trained helper with a job description, a toolbox it is allowed to use, and its own desk, so it doesn't spread papers over yours.

⌨️ Commands: project vs personal

  • .claude/commands/review.md — project command, committed, available to everyone who clones the repo.
  • ~/.claude/commands/ — personal commands, not shared.
⚠️ Watch out: Commands are not defined in CLAUDE.md, and there is no .claude/config.json commands array.

🌍 Real-world example: One /review for the whole team

Ravi saves the checklist as .claude/commands/review.md and commits it. Every tester who pulls the repository can type /review. His personal /standup command stays in ~/.claude/commands/.

🎯 Skills and SKILL.md

---
name: explore-codebase
description: Map an unfamiliar area of the codebase and summarise entry points
context: fork
allowed-tools: Read, Grep, Glob
argument-hint: <directory or feature to explore>
---
Explore $ARGUMENTS. Return: purpose, entry points, key data flows, risks.
FrontmatterWhat it doesUse when
context: forkRuns the skill in an isolated sub-agent contextVerbose output (codebase analysis) or exploratory brainstorming would pollute the main session
allowed-toolsRestricts tools during the skillPrevent destructive actions (e.g. allow only file writes, no Bash)
argument-hintPrompts for required parametersUsers often invoke the skill without arguments

🌍 Real-world example: A safe explorer for new joiners

The explore skill produces long file listings, so it runs with context: fork and keeps the main chat clean. It may only read and search, never edit or run commands. Its argument hint reminds people to say which folder to explore.

---
name: explore-codebase
context: fork
allowed-tools: Read, Grep, Glob
argument-hint: <folder or feature>
---

📘 Personal variants

To customise a team skill for yourself, create a copy under a different name in ~/.claude/skills/. Editing the team skill — even locally — affects others or gets shadowed.

🌍 Real-world example: Ravi's stricter review

Ravi wants an extra security check in his own reviews. Instead of editing the team's command, he copies it to ~/.claude/skills/ under a new name, review-strict. The team version stays unchanged for everyone else.

📄 Skills vs CLAUDE.md

Skills are invoked on demand for task-specific workflows; CLAUDE.md is always loaded for universal standards. Conventions that must always apply belong in CLAUDE.md or rules, not in a skill.

🌍 Real-world example: Always-on rules vs on-demand jobs

'Never commit secrets' must apply all the time, so it goes in CLAUDE.md. 'Generate a release-notes draft' is needed once a sprint, so it is a skill that runs only when someone asks for it.

🚫 Common beginner mistake: Defining commands inside CLAUDE.md, or in a made-up config file. Commands are Markdown files in .claude/commands/.

🛠️ Try it yourself (about 8 minutes)

  1. Create .claude/commands/hello.md containing 'Summarise the README in 3 bullet points.'
  2. Start Claude Code in that repository.
  3. Type /hello and run it.
  4. Commit the file so teammates get it too.

✅ You are done when: /hello appears in the command list and prints a 3-bullet summary.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. Where does a project slash command that every developer gets on clone belong?

  • A. .claude/commands/ in the repo
  • B. CLAUDE.md
  • C. .claude/config.json
  • D. ~/.claude/commands/
Show answer

Answer: A. Project-scoped and version-controlled. D is personal. B holds instructions, not commands. C doesn't exist.

Question 2. A skill should only write files and never run Bash. Which frontmatter enforces this?

  • A. context: fork
  • B. paths
  • C. argument-hint
  • D. allowed-tools: Write, Edit (or similar, excluding Bash)
Show answer

Answer: D. It restricts the tools available during the skill. A isolates context. C prompts for arguments. B belongs to rules.

Question 3. Developers often run /migrate-schema with no arguments and get poor output. What helps?

  • A. Plan mode
  • B. context: fork
  • C. A longer CLAUDE.md
  • D. argument-hint: <table-name> <target-version> in the skill frontmatter
Show answer

Answer: D. It prompts for the required parameters. A, B and C don't address missing arguments.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Create .claude/commands/review.md with your team's review checklist and run it.
    ✅ Done when: A teammate can run /review after pulling the repository, and it follows your checklist.
  2. Task 2: Create a skill with context: fork, allowed-tools and argument-hint, and run it with and without arguments.
    ✅ Done when: Running the skill without arguments shows the argument hint, and its output does not flood the main conversation.
  3. Stretch: make a personal variant of a team skill under a different name in ~/.claude/skills/.
    ✅ Done when: Your personal variant has a different name, and the team skill still behaves exactly as before.

❓ Frequently Asked Questions

🤔 Where do team slash commands live?

In .claude/commands/ in the repository, so everyone gets them after cloning.

🤔 What does context: fork do?

It runs the skill in an isolated sub-agent context so its output does not crowd the main conversation.

🤔 How do I stop a skill running shell commands?

List only the tools it needs in allowed-tools, leaving out Bash.

🤔 Skill or CLAUDE.md?

Use a skill for on-demand, task-specific workflows and CLAUDE.md for standards that must always apply.

🏁 Summary

  • .claude/commands = team; ~/.claude/commands = personal.
  • context: fork isolates, allowed-tools restricts, argument-hint prompts.
  • Personal variants get a different name at user level.
📝 Test yourself: Mock Test 1 · Mock Test 4 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Code Plan Mode vs Direct Execution and Iterative Refinement.

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.

CLAUDE.md Configuration Guide: Hierarchy, @import and Path-Specific Rules

📚 Exam guide v1.0 (July 2026)🔄 Checked against official sources: 11 Nov 2026✏️ Updated: 12 Nov 2026⏱️ 8 min read🎯 Official objectives 3.1 and 3.3

👋 In plain words: CLAUDE.md is a plain text file where you write instructions Claude Code reads every time, such as 'use pytest' or 'never edit generated files'. Where you put that file decides who gets the rules. This lesson shows how to set it up so the whole team gets the same behaviour.

CLAUDE.md is how you give Claude Code persistent project knowledge. The exam tests where instructions live, who receives them, and how to keep them modular so the right conventions load at the right time (Task Statements 3.1 and 3.3).

📖 What You’ll Learn: You will be able to set up CLAUDE.md files so the right conventions reach every teammate and load only when they are relevant.

🔑 Key terms

  • CLAUDE.md: A Markdown file of standing instructions that Claude Code loads automatically.
  • User level: ~/.claude/CLAUDE.md on your own machine. Only you get these rules.
  • Project level: CLAUDE.md in the repository. Everyone who clones it gets these rules.
  • @import: A line in CLAUDE.md that pulls in another file's contents.
  • Path-specific rule: A rule file in .claude/rules/ that loads only when Claude edits matching files.

📖 The story in this lesson

A new tester joins Ravi's team and Claude Code ignores the team's test conventions on their laptop. The reason: Ravi had put the rules in his personal CLAUDE.md.

💡 Think of it like this

CLAUDE.md levels are like notices in a company. A sticky note on your own monitor (user level) is for you alone. A notice on the office board (project level) reaches everyone. A sign on the lab door (directory or path rules) applies only to people working in that room.

🏛️ The three-level hierarchy

LevelLocationShared with team?
User~/.claude/CLAUDE.mdNo — only you
Project./CLAUDE.md or ./.claude/CLAUDE.mdYes, via git
Directorypackages/api/CLAUDE.mdYes; applies when working there
⚠️ Watch out: “New team members don’t get our conventions” → the conventions are in a user-level file. Move them to the project CLAUDE.md.

🌍 Real-world example: The rules the new hire never got

Ravi moves 'Use Playwright fixtures, never hard-coded waits' from ~/.claude/CLAUDE.md into the repository's CLAUDE.md and commits it. The next morning the new tester pulls the repository and Claude follows the rule on their laptop too.

CLAUDE.md configuration hierarchyUser level~/.claude/CLAUDE.md · ~/.claude/skills · ~/.claude/commandsonly you — not sharedProject level./CLAUDE.md (+ @imports) · .claude/commands · .claude/skills · .mcp.jsonwhole team via gitDirectory levelpackages/payments/CLAUDE.mdwhen working in that folderPath-scoped rules.claude/rules/testing.md → paths: [“**/*.test.tsx”]only for matching filesAll applicable files load together into the session — run /memory to see exactly which onesTeam conventions belong at project level; a new teammate never receives your user-level file
Figure: where each CLAUDE.md and rule file lives, and who receives it. Swipe sideways to see the whole diagram.

📎 Keep it modular with @import

# packages/payments/CLAUDE.md
Payments service conventions.
@docs/standards/pci.md
@docs/standards/error-handling.md

Each package imports only the standards relevant to it, instead of loading everything everywhere.

🌍 Real-world example: Payment rules only where they matter

The payments package needs PCI security rules, but the storefront doesn't. The payments CLAUDE.md imports only the files it needs, so storefront work never loads irrelevant rules.

# packages/payments/CLAUDE.md
@docs/standards/pci.md
@docs/standards/error-handling.md

📏 .claude/rules/ and path-specific rules

Split a huge CLAUDE.md into topic files such as testing.md, api-conventions.md and deployment.md in .claude/rules/. Add YAML frontmatter with paths globs so a rule loads only when Claude edits matching files:

---
paths: ["**/*.test.tsx", "**/*.test.ts"]
---
Use React Testing Library. One behaviour per test. No snapshot tests.

Path-scoped rules beat directory CLAUDE.md files when matching files are scattered (tests next to components, Terraform in many service folders), and they save context because irrelevant rules never load.

🌍 Real-world example: Test rules that follow the test files

ShopNest's test files sit next to the code they test, in dozens of folders. One rule file with a paths pattern applies the testing conventions wherever a test file lives, and only when Claude edits a test.

---
paths: ["**/*.spec.ts"]
---
Use page objects. One behaviour per test.

🧠 Debug with /memory

When two developers see different behaviour, run /memory in each session to see exactly which memory files are loaded. It is the first diagnostic for hierarchy problems.

🌍 Real-world example: Why do our laptops behave differently?

Two testers get different results from the same request. Each runs /memory and compares the list of loaded files. One laptop still has an old personal rule file that overrides the team setup.

🚫 Common beginner mistake: Putting team rules in your personal ~/.claude/CLAUDE.md. Only you get them. Team rules belong in the repository.

🛠️ Try it yourself (about 8 minutes)

  1. In a test repository, run /init in Claude Code to create a CLAUDE.md.
  2. Add one rule, such as 'Write tests with pytest'.
  3. Run /memory and find your file in the list.
  4. Ask Claude to write a small test and check it follows the rule.

✅ You are done when: /memory shows the project CLAUDE.md and the generated test follows your rule.

🧪 Exam-Style Practice Questions

Try these before opening the answers. They use the same scenario style as the real exam.

Question 1. Your packages/payments/CLAUDE.md should include the PCI standards file but not the frontend style guide. What's the cleanest method?

  • A. @import the PCI standards file in that package's CLAUDE.md
  • B. Copy-paste the PCI text into it
  • C. Create a skill
  • D. Put everything in the root
Show answer

Answer: A. Selective, modular imports. B duplicates text that will drift. D loads irrelevant rules everywhere. C isn't always-on.

Question 2. Test files sit next to their sources throughout the repo. What's the best way to apply testing conventions automatically?

  • A. Root CLAUDE.md
  • B. A CLAUDE.md in each folder
  • C. A /test command
  • D. A .claude/rules/testing.md with paths: ["**/*.test.*"]
Show answer

Answer: D. Glob-scoped rules follow file types wherever they are. B is impractical for scattered files. C needs manual invocation. A loads the rules always and relies on inference.

Question 3. What's the key difference between a skill and CLAUDE.md?

  • A. CLAUDE.md can't be shared
  • B. Skills are invoked on demand for specific tasks; CLAUDE.md holds always-loaded standards
  • C. Skills can't restrict tools
  • D. Skills are always loaded
Show answer

Answer: B. D is reversed. A: project CLAUDE.md is shared. C: allowed-tools restricts them.

📝 Assignment

Do these tasks in order. Each one has a Done when check, so you know when it is complete. The stretch task is optional but close to what the exam expects from an architect. Keep your answers in one notes file or Git repository, and revisit them in your final week of revision together with the cheat sheet in the last lesson of this course.

  1. Task 1: Run /init in a repository, then move one team convention from your user-level file into the project CLAUDE.md.
    ✅ Done when: A teammate who pulls the repository gets the convention without changing their own machine, and /memory shows the project file.
  2. Task 2: Create .claude/rules/testing.md with a paths glob for test files and confirm it loads only when you edit a test.
    ✅ Done when: The testing rule appears when you edit a test file and does not appear when you edit an unrelated source file.
  3. Stretch: split a long CLAUDE.md into three rule files and use @import for one shared standards document.
    ✅ Done when: The original CLAUDE.md is shorter, the three rule files each cover one topic, and the imported file loads where you expect.

❓ Frequently Asked Questions

🤔 Why doesn't my teammate get my CLAUDE.md rules?

They are probably in ~/.claude/CLAUDE.md, which is personal. Put team rules in the project CLAUDE.md.

🤔 What does @import do in CLAUDE.md?

It pulls another file's contents into CLAUDE.md, so each package can include only the standards it needs.

🤔 When should I use .claude/rules with paths?

When conventions apply to files spread across many folders, such as all test files.

🤔 How do I see which memory files are loaded?

Run /memory in the Claude Code session.

🏁 Summary

  • User-level = personal; project-level = team.
  • @import for modular standards; .claude/rules/ for topic files.
  • paths globs load conventions only for matching files, wherever they live.
  • /memory shows what’s loaded.
📝 Test yourself: Mock Test 1 · Mock Test 4 (30 scenario-based questions in 60 minutes, with explanations).

Next, continue with Claude Code Slash Commands and Skills (SKILL.md Frontmatter).

🎬 Prefer video? Watch QA, automation and AI testing tutorials on the SQA Tools YouTube channel.

Independent study material by SQA Tools. Not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic. Mock questions are original practice items, not real exam content. Exam facts checked against Anthropic's official exam guides and certification FAQ on 11 November 2026.