Generating Test Cases with LLMs
Prompts that turn requirements into test cases and test data, and the review checklist that keeps them correct.
Why use an LLM for test design?
Turning a user story into a complete set of test cases takes time, and it's easy to miss edge cases when you're tired or rushed. A large language model can produce a strong first draft in seconds — positive, negative, boundary and security cases — which you then review, correct and prioritise.
The key is a good prompt and a disciplined review.
A prompt that produces useful test cases
Vague prompts ("write test cases for login") produce generic lists. Give the model the requirement, the techniques to apply and the output format:
You are a senior QA engineer. Design test cases for the requirement below.
Requirement:
A discount code field at checkout. Codes are 6–12 characters, letters and
digits only, case-insensitive. A code can be used once per customer, only
before its expiry date, and only on orders of at least $20. Only one code
per order. The discount can never make the total negative.
Instructions:
- First list any ambiguities or missing rules as questions. Do not invent answers.
- Apply equivalence partitioning, boundary value analysis and a decision table.
- Include positive, negative, boundary, security and usability cases.
- Mark each case with priority P1 (must run every release) to P3.
Output a Markdown table with columns:
ID | Title | Technique | Preconditions | Steps | Test data | Expected result | Priority
Asking for questions first is the most valuable part: the model will flag gaps such as "Are spaces around the code trimmed?" or "What happens when a code is applied and the cart then drops below $20?" — exactly the questions a good tester asks in refinement.
Generating test cases from code (Python + Claude)
You can automate the drafting step. This script uses the official Anthropic Python SDK to turn a requirement into structured test cases you can import into a test management tool. Any capable LLM works the same way; this example uses Claude.
pip install anthropic
export ANTHROPIC_API_KEY="your-key" # Windows PowerShell: $env:ANTHROPIC_API_KEY="your-key"
# generate_tests.py — run with: python generate_tests.py
import csv
import json
import anthropic
REQUIREMENT = """Discount codes are 6-12 letters/digits, case-insensitive, single use per
customer, valid until their expiry date, for orders of at least $20, one code per order,
and the total can never go below $0."""
PROMPT = f"""You are a senior QA engineer. Design test cases for this requirement:
{REQUIREMENT}
Apply equivalence partitioning, boundary value analysis and decision tables. Include
negative and security cases. Do not invent rules that are not in the requirement.
Respond with JSON only, no prose, in this shape:
{{"questions": ["..."],
"test_cases": [{{"id": "TC-01", "title": "...", "technique": "...",
"steps": "...", "test_data": "...", "expected": "...", "priority": "P1"}}]}}"""
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from the environment
message = client.messages.create(
model="claude-opus-5-5",
max_tokens=4000,
messages=[{"role": "user", "content": PROMPT}],
)
FENCE = "`" * 3 # a Markdown code fence
text = message.content[0].text.strip()
if text.startswith(FENCE): # tolerate a fenced JSON block
text = text.strip("`").removeprefix("json").strip()
result = json.loads(text)
print("Open questions for the product owner:")
for q in result["questions"]:
print(" -", q)
with open("generated_test_cases.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=["id", "title", "technique", "steps", "test_data", "expected", "priority"])
writer.writeheader()
writer.writerows(result["test_cases"])
print(f"\nWrote {len(result['test_cases'])} draft test cases to generated_test_cases.csv — review before use.")
The output is a draft. Review it before anything is imported or automated.
The review checklist
For every generated test case, check:
- Traceable — does it test something actually in the requirement, or an invented rule?
- Correct expected result — would the product owner agree?
- Specific — exact data and exact expected messages, not "an error is shown".
- Not a duplicate — LLMs often repeat the same idea with different wording.
- Right priority — the model doesn't know your business risk; you do.
- Missing cases — what did it skip? (Concurrency, two browser tabs, time zones and permissions are often missed.)
Expect to delete 10–30% of the drafts and add a few of your own. That's still much faster than starting from a blank page.
Generating test data
LLMs are good at varied, realistic edge-case data that's tedious to invent:
Generate 15 values for a "full name" field to test a registration form.
Include: very short and very long names, apostrophes and hyphens,
Urdu and Arabic script, names with diacritics, a single word, leading
and trailing spaces, an emoji, and an HTML/script injection attempt.
Return one value per line with a short note on what each tests.
For large volumes of ordinary data (10,000 customers), use a library such as Faker instead — it's free, fast and repeatable:
# pip install faker
from faker import Faker
fake = Faker(["en_US", "ar_AA"])
Faker.seed(42) # same data every run → reproducible tests
for _ in range(3):
print(fake.name(), "|", fake.email(), "|", fake.phone_number())
Use the LLM for the clever edge cases, Faker for bulk data.
Prompting tips that improve results
- Give context: user roles, platform, business rules, known risky areas.
- Ask for techniques by name (EP, BVA, decision tables, state transitions) — it changes the output noticeably.
- Ask for questions before answers to surface ambiguity.
- Request a structured format (table or JSON) so results are easy to review and import.
- Iterate: "Add five more negative cases around expiry dates and time zones."
- Never include real customer data or secrets in prompts.
The Prompt Engineering track covers these techniques in depth. Next: letting AI help write the automation code itself — AI-Assisted Test Automation.