Skip to content

1.5 The Power of Prompt Chaining

Prompt chaining solves complex tasks through a sequence of simple, interconnected steps. Instead of one “monolithic” request, you build a chain of small prompts: each step solves a specific subtask and prepares context for the next. This reduces errors, makes model behavior more controllable, and increases observability: it’s easier to see where and why a mistake happened and to intervene precisely. It’s like cooking a complex dish step by step or using modular architecture in software — it’s always easier to debug and maintain a series of small, clear operations than a single spaghetti‑like step.

Practical benefits are clear: you can orchestrate the workflow by checkpointing state at each step and adapting the next step to the previous result; save context and budget, since long prompts cost more while each chain step uses only the minimum needed; reduce errors by isolating the problem; and load only relevant information, respecting LLM context limits. Methodologically, this means decomposing the task, explicitly managing state between steps, designing each prompt for a narrow focus, adding tools for loading and pre‑processing data, and dynamically injecting only the context fragments needed right now. Best practices are simple: don’t overcomplicate when a single prompt suffices; be clear; keep and update external context; think about efficiency (quality, cost, latency); and test the chain end to end.

Below is a sequential example that assembles an end‑to‑end scenario: entity extraction, querying a simple “database”, parsing JSON, and composing a user‑facing answer — then tying it all together into a single support flow.

import os
from openai import OpenAI
from dotenv import load_dotenv, find_dotenv

_ = load_dotenv(find_dotenv())
client = OpenAI()

Now extract entities from a user request. The first step sets the task and output format in a system instruction. The user input is bounded by delimiters, which makes it easier to control data boundaries and pass the result along the chain.

def retrieve_model_response(message_sequence, model="gpt-4o-mini", temperature=0, max_tokens=500):
    response = client.chat.completions.create(
        model=model,
        messages=message_sequence,
        temperature=temperature,
        max_tokens=max_tokens,
    )
    return response.choices[0].message["content"]

system_instruction = """
You will receive support requests. The request is delimited by '####'.
Return a Python list of objects, each representing a product or category mentioned in the request.
"""

user_query = "#### Tell me about the SmartX ProPhone and the FotoSnap DSLR Camera, and also about your televisions ####"

message_sequence = [
    {'role': 'system', 'content': system_instruction},
    {'role': 'user', 'content': user_query},
]

extracted_info = retrieve_model_response(message_sequence)
print(extracted_info)

Next, plug in a data source and find the specific products or categories. Even an in‑memory “database” demonstrates the idea: extract entities → move to structured data → prepare facts for the answer.

product_database = {
    "TechPro Ultrabook": {
        "name": "TechPro Ultrabook",
        "category": "Computers and Laptops",
    },
    # ...
}

def get_product_details_by_name(product_name):
    return product_database.get(product_name, None)

def get_products_in_category(category_name):
    return [p for p in product_database.values() if p["category"] == category_name]

print(get_product_details_by_name("TechPro Ultrabook"))
print(get_products_in_category("Computers and Laptops"))

On the next step, convert JSON strings that the model might return during entity extraction into Python objects for downstream chain steps.

import json

def json_string_to_python_list(json_string):
    if json_string is None:
        return None
    try:
        json_string = json_string.replace("'", '"')
        return json.loads(json_string)
    except json.JSONDecodeError:
        print("Error: Invalid JSON string")
        return None

json_input = "[{\"category\": \"Smartphones and Accessories\", \"products\": [\"SmartX ProPhone\"]}]"
python_list = json_string_to_python_list(json_input)
print(python_list)

Finally, compose a concise user‑facing answer from the resulting structures. You can swap this formatting layer for templates, localization, or generation tuned to your UX.

def generate_response_from_data(product_data_list):
    response_string = ""
    if product_data_list is None:
        return response_string
    for data in product_data_list:
        response_string += json.dumps(data, indent=4) + "\n"
    return response_string

response_instruction = """
You are a support assistant. Answer briefly and ask clarifying questions when needed.
"""

final_response = generate_response_from_data(python_list)
print(final_response)

We’ll finish with an end‑to‑end support scenario: first detect interest in photography, then provide troubleshooting, clarify warranty coverage, and end with accessory recommendations — four steps in one chain, each building on the previous result.

system_instruction = """
You will receive support requests delimited by '####'.
Return a list of objects: the mentioned products/categories.
"""

user_query_1 = "#### I'm interested in upgrading my photography gear. Can you tell me about the latest DSLR cameras and compatible accessories? ####"

message_sequence_1 = [
    {'role': 'system', 'content': system_instruction},
    {'role': 'user', 'content': user_query_1},
]
response_1 = retrieve_model_response(message_sequence_1)
print("Product query response:", response_1)

troubleshooting_query = "#### I just bought the FotoSnap DSLR Camera you recommended, but I'm having trouble connecting it to my smartphone. What should I do? ####"
system_instruction_troubleshooting = "Provide step‑by‑step troubleshooting advice for the customer’s issue."
message_sequence_2 = [
    {'role': 'system', 'content': system_instruction_troubleshooting},
    {'role': 'user', 'content': troubleshooting_query},
]
response_2 = retrieve_model_response(message_sequence_2)
print("Troubleshooting response:", response_2)

follow_up_query = "#### Also, could you clarify what the warranty covers for the FotoSnap DSLR Camera? ####"
system_instruction_follow_up = "Provide detailed information about the product’s warranty coverage."
message_sequence_3 = [
    {'role': 'system', 'content': system_instruction_follow_up},
    {'role': 'user', 'content': follow_up_query},
]
response_3 = retrieve_model_response(message_sequence_3)
print("Warranty information response:", response_3)

additional_assistance_query = "#### Given your interest in photography, would you like recommendations for lenses and tripods compatible with the FotoSnap DSLR Camera? ####"
system_instruction_additional_assistance = "Suggest accessories that complement the user’s existing products."
message_sequence_4 = [
    {'role': 'system', 'content': system_instruction_additional_assistance},
    {'role': 'user', 'content': additional_assistance_query},
]
response_4 = retrieve_model_response(message_sequence_4)
print("Additional assistance response:", response_4)

In the end, prompt chaining gives you a robust and understandable workflow: it saves context and budget, localizes errors more precisely, and preserves flexibility to tailor the answer to the user’s task.

Theory Questions

  1. What is prompt chaining and how does it differ from using one long prompt?
  2. Provide two analogies and explain how they map to chaining.
  3. How does chaining help manage workflow?
  4. Where do the savings come from when using chaining?
  5. How does chaining reduce errors on complex tasks?
  6. Why is dynamic data loading useful given LLM context limits?
  7. Describe a step‑by‑step methodology for chaining and the role of each step.
  8. List best practices that ensure chaining efficiency.
  9. Which libraries are used in the example and for what?
  10. How does the system message guide the model’s answer?
  11. What is the role of the product database, and how do you query it?
  12. Why convert JSON strings to Python objects, and how?
  13. How does formatting answers from processed data improve service quality?
  14. How does the end‑to‑end scenario demonstrate adapting to user needs via chaining?

Practical Tasks

  1. Implement retrieve_model_response with model, temperature, and max_tokens parameters.
  2. Show an entity‑extraction example using a system instruction.
  3. Create a mini product database and functions to query by name or category.
  4. Implement JSON‑to‑Python list conversion with error handling.
  5. Write generate_response_from_data that formats a data list into a user‑friendly answer.
  6. Compose an end‑to‑end support scenario (query → troubleshooting → warranty → recommendations) based on the functions above.