Skip to content

1.4 Advanced Machine Reasoning: Strategies

Advanced machine reasoning brings together a set of practices that help language models solve complex tasks more reliably and transparently. Chain of Thought (CoT) encourages step‑by‑step solutions, breaking the problem into logical stages. This approach improves accuracy and makes the reasoning auditable: the user can see how the model arrived at the answer, which is especially helpful for multi‑constraint tasks, comparative analysis, and calculations. In education, CoT mimics a tutor who guides you through each step rather than giving a finished answer. In customer support, it helps unpack complicated requests: clarify details, check assumptions, fix misunderstandings, and provide a correct conclusion.

In parallel with CoT, teams often use Inner Monologue, where intermediate reasoning is hidden and only the result (or a minimal slice of logic) is shown. This is appropriate when exposing internal steps could harm learning (avoiding “spoilers”), when sensitive information is involved, or when extra details would degrade the user experience.

To make the examples reproducible, start by preparing the environment and API client.

# Import libraries and load keys
import os
from openai import OpenAI
from dotenv import load_dotenv, find_dotenv

load_dotenv(find_dotenv())
client = OpenAI()

def get_response_for_queries(query_prompts,
                             model_name="gpt-4o-mini",
                             response_temperature=0,
                             max_response_tokens=500):
    """
    Returns the model response based on a list of messages (system/user...).
    """
    model_response = client.chat.completions.create(
        model=model_name,
        messages=query_prompts,
        temperature=response_temperature,
        max_tokens=max_response_tokens,
    )
    return model_response.choices[0].message["content"]

Next, we’ll set up a wrapper for requests and move to CoT prompting, where the reasoning is structured into steps under a special delimiter. The system message describes the analysis stages, and the user input is wrapped in delimiters, simplifying parsing and later post‑processing.

step_delimiter = "####"

system_prompt = f"""
Follow the steps, separating them with the '{step_delimiter}' marker.

Step 1:{step_delimiter} Check whether the question is about a specific product (not a category).

Step 2:{step_delimiter} If yes, match it to the product list (brand, specs, price).

[Insert your product list here]

Step 3:{step_delimiter} Identify the user’s assumptions (comparisons/specifications).

Step 4:{step_delimiter} Verify those assumptions against the product data.

Step 5:{step_delimiter} Correct inaccuracies using only the list and respond politely.
"""

example_query_1 = "How does the BlueWave Chromebook compare to the TechPro Desktop in terms of cost?"
example_query_2 = "Are televisions available for sale?"

query_prompts_1 = [
    {'role': 'system', 'content': system_prompt},
    {'role': 'user', 'content': f"{step_delimiter}{example_query_1}{step_delimiter}"},
]

query_prompts_2 = [
    {'role': 'system', 'content': system_prompt},
    {'role': 'user', 'content': f"{step_delimiter}{example_query_2}{step_delimiter}"},
]

response_to_query_1 = get_response_for_queries(query_prompts_1)
print(response_to_query_1)

response_to_query_2 = get_response_for_queries(query_prompts_2)
print(response_to_query_2)

To compare approaches, first print the full answer with intermediate CoT steps, then apply an Inner Monologue variant where only the final portion is shown to the user. If the model returns text with steps separated by step_delimiter, you can keep just the final segment — keeping the interface succinct where the “inner workings” aren’t needed.

try:
    final_response = response_to_query_2.split(step_delimiter)[-1].strip()
except Exception:
    final_response = "Sorry, there was a problem. Please try another question."

print(final_response)

The result is two modes of the same solution: a detailed one, with a visible chain of steps, and a concise one showing only the outcome. Clear prompt design helps in both cases; keep refining your prompts based on observed behavior. When UI clarity matters and extra details are undesirable, prefer Inner Monologue for display while still leveraging internal step‑by‑step analysis for quality control.

Theory Questions

  1. What is Chain of Thought (CoT) and why is it useful for multi‑step tasks?
  2. How does CoT transparency increase user trust in model answers?
  3. How does CoT help in educational scenarios?
  4. How does a reasoning chain improve the quality of support chatbot answers?
  5. What is Inner Monologue, and how does it differ from CoT in terms of what the user sees?
  6. Why is Inner Monologue important when dealing with sensitive information?
  7. How can Inner Monologue help in learning scenarios without revealing “spoilers”?
  8. What steps are needed to prepare the environment for OpenAI API examples?
  9. How is the get_response_for_queries function structured?
  10. How does CoT prompting simplify handling complex queries?
  11. How does the system/user prompt structure help answer product questions?
  12. Why is extracting only the final part of the answer useful when using Inner Monologue?

Practical Tasks

  1. Implement chain_of_thought_prompting(query), which generates a system prompt with step structure and wraps the user query in a delimiter.
  2. Write get_final_response(output, delimiter) to extract the last part of the answer and handle possible errors.
  3. Create a script that sends two queries — one with CoT and one with Inner Monologue — and prints both responses.
  4. Implement validate_response_structure(resp, delimiter) to check that the answer contains the required number of steps.
  5. Build a QueryProcessor class that encapsulates CoT and Inner Monologue logic (key loading, prompt assembly, request sending, post‑processing, and error handling).