Advanced Prompting

Requesting structured output#

By default, an LLM produces free-form prose — great for a chat interface, unusable for code that needs to parse the result reliably. Asking explicitly for a structured format, and describing that format precisely, gets far more consistent, machine-parseable output.

Prompt:

Extract the following fields from this customer message and return ONLY valid
JSON matching this exact schema, with no extra commentary:

{
  "customer_name": string,
  "issue_category": "billing" | "technical" | "account" | "other",
  "urgency": "low" | "medium" | "high"
}

Message: "Hi, this is Maria Chen. I've been charged twice for my subscription
this month and I need this fixed today."

Expected output:

{
  "customer_name": "Maria Chen",
  "issue_category": "billing",
  "urgency": "high"
}

Two details matter more than they might seem: naming the exact schema (field names and allowed values) rather than describing it loosely, and explicitly saying "return ONLY valid JSON, no extra commentary." Without that instruction, a model will often add a friendly preamble ("Sure, here's the extracted information:") that breaks a naive JSON.parse() call on the raw output. Production systems typically add a further validation step after parsing (for example, checking the response against a JSON Schema) rather than trusting the model to never deviate.

Function/tool calling as a prompting pattern#

Function calling (also called tool calling) is a pattern where, instead of asking the model to directly answer a question, you describe a set of available functions/tools (each with a name, a description, and its expected parameters), and let the model decide whether to call one, which one, and with what arguments — as structured data your application code then actually executes.

Conceptually: given a question like "What's the weather in Tokyo right now?", a model with no tools can only guess or admit it doesn't know (it has no live data). A model given a get_weather(location: string) tool description can instead respond with a structured request to call that function:

{
  "tool_call": {
    "name": "get_weather",
    "arguments": { "location": "Tokyo" }
  }
}

Your application code executes the actual get_weather function, gets a real result, and feeds that result back to the model, which then produces a final natural-language answer grounded in real data. The model itself never touches the network or a database directly — it only ever decides when and how to ask your code to do so, which is what makes this pattern safe to put behind real, permissioned application logic.

This is the same underlying mechanism that powers autonomous "agents" (see this site's LangChain tutorials for a fuller treatment): a loop where the model can repeatedly decide to call a tool, observe the result, and decide what to do next.

Self-consistency and multiple attempts#

For tasks where a single generation can be inconsistent, especially reasoning-heavy ones, one useful technique is generating multiple independent answers to the same prompt — often with chain-of-thought reasoning and some randomness (temperature) enabled — and then taking the most common final answer across them ("self-consistency"), or having a separate pass compare the candidates and pick the best one. This trades extra compute and latency for meaningfully higher reliability on problems where any single attempt has a non-trivial chance of a reasoning slip, and is most worth the added cost on high-stakes or hard-to-verify tasks rather than routine ones.

Common mistakes#

• Trusting a model's structured output without validating it against your actual schema in code — models occasionally add extra fields, omit a required one, or produce a value outside your allowed enum, especially under prompt or content edge cases.
• Giving a tool-calling model vague or overlapping tool descriptions — if two tools sound like they could both handle a request, the model's choice between them becomes unreliable.
• Using self-consistency/multiple-attempts on every request regardless of cost — it meaningfully increases latency and API spend, so it's worth reserving for genuinely hard or high-stakes cases rather than applying it uniformly.