Structured output

LLM providers offer functionality for defining a structure of the messages generated by LLMs, AG2 enables this functionality by propagating a response_format, in the LLM configuration for your agents, to the underlying client.

LLM providers offer functionality for defining a structure of the messages generated by LLMs, AG2 enables this functionality by propagating a response_format, in the LLM configuration for your agents, to the underlying client.

Structured outputs are available for a number of Model Providers, see the Supported model providers section below. In this example we will use OpenAI as the model provider.

Install ag2:

pip install -U ag2[openai]

Note: If you have been using autogen or ag2, all you need to do is upgrade it using:

pip install -U autogen

or

pip install -U ag2

as autogen, and ag2 are aliases for the same PyPI package.

For more information, please refer to the installation guide.

Supported model providers

AG2 has structured output support for following model providers:

• OpenAI (openai)

• Anthropic (anthropic)

• Google Gemini (google)

• Ollama (ollama)

Set your API Endpoint

The LLMConfig.from_json function loads a list of configurations from an environment variable or a json file. Structured Output is supported by OpenAI’s models from gpt-4-0613 and gpt-3.5-turbo-0613.

import autogen

Learn more about configuring LLMs for agents here.

Example: math reasoning

Using structured output, we can enforce chain-of-thought reasoning in the model to output an answer in a structured, step-by-step way.

Define the reasoning model

First we will define the math reasoning model. This model will indirectly force the LLM to solve the posed math problems iteratively through math reasoning steps.

from pydantic import BaseModel


class Step(BaseModel):
    explanation: str
    output: str


class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

Define chat actors

Now we can define the agents that will solve the posed math problem. We will keep this example simple; we will use a UserProxyAgent to input the math problem and an AssistantAgent to solve it.

The AssistantAgent will be constrained to solving the math problem step-by-step by using the MathReasoning response format we defined above.

The response_format is added to the LLM configuration and then this configuration is applied to the agent.

llm_config = autogen.LLMConfig.from_json(
    path="OAI_CONFIG_LIST",
    cache_seed=42,
    response_format=MathReasoning,
).where(tags=["gpt-4o", "gpt-4o-mini"])


user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="A human admin.",
    human_input_mode="NEVER",
)

assistant = autogen.AssistantAgent(
    name="Math_solver",
    llm_config=llm_config,  # Response Format is in the configuration
)

Start the chat

Let’s now start the chat and prompt the assistant to solve a simple equation. The assistant agent should return a response solving the equation using a step-by-step MathReasoning model.

user_proxy.initiate_chat(assistant, message="how can I solve 8x + 7 = -23", max_turns=1, summary_method="last_msg")

Formatting a response

When defining a response_format, you have the flexibility to customize how the output is parsed and presented, making it more user-friendly. To demonstrate this, we’ll add a format method to our MathReasoning model. This method will define the logic for transforming the raw JSON response into a more human-readable and accessible format.

Define the reasoning model

Let’s redefine the MathReasoning model to include a format method. This method will allow the underlying client to parse the return value from the LLM into a more human-readable format. If the format method is not defined, the client will default to returning the model’s JSON representation, as demonstrated in the previous example.

from pydantic import BaseModel


class Step(BaseModel):
    explanation: str
    output: str


class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

    def format(self) -> str:
        steps_output = "\n".join(
            f"Step {i + 1}: {step.explanation}\n  Output: {step.output}" for i, step in enumerate(self.steps)
        )
        return f"{steps_output}\n\nFinal Answer: {self.final_answer}"

Define chat actors and start the chat

The rest of the process is the same as in the previous example: define the actors and start the chat.

Observe how the Math_solver agent now communicates using the format we have defined in our MathReasoning.format method.

for config in llm_config.config_list:
    config.response_format = MathReasoning

user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="A human admin.",
    human_input_mode="NEVER",
)

assistant = autogen.AssistantAgent(
    name="Math_solver",
    llm_config=llm_config,
)

print(
    user_proxy.initiate_chat(
        assistant, message="how can I solve 8x + 7 = -23", max_turns=1, summary_method="last_msg"
    ).summary
)

Normal function calling still works alongside structured output, so your agent can have a response format while still calling tools.

@user_proxy.register_for_execution()
@assistant.register_for_llm(description="You can use this function call to solve addition")
def add(x: int, y: int) -> int:
    return x + y


user_proxy.initiate_chat(
    assistant, message="solve 3 + 4 by calling appropriate function", max_turns=2, summary_method="last_msg"
)

Edit this page