autogen.agentchat.contrib.capabilities.transforms.MessageTokenLimiter

MessageTokenLimiter

MessageTokenLimiter(
    max_tokens_per_message: int | None = None,
    max_tokens: int | None = None,
    min_tokens: int | None = None,
    model: str = 'gpt-3.5-turbo-0613',
    filter_dict: dict[str, Any] | None = None,
    exclude_filter: bool = True
)

Truncates messages to meet token limits for efficient processing and response generation.
This transformation applies two levels of truncation to the conversation history:
1. Truncates each individual message to the maximum number of tokens specified by max_tokens_per_message.
2. Truncates the overall conversation history to the maximum number of tokens specified by max_tokens.
NOTE: Tokens are counted using the encoder for the specified model. Different models may yield different token counts for the same text.
NOTE: For multimodal LLMs, the token count may be inaccurate as it does not account for the non-text input (e.g images).
The truncation process follows these steps in order:
1. The minimum tokens threshold (min_tokens) is checked (0 by default). If the total number of tokens in messages is less than this threshold, then the messages are returned as is. In other case, the following process is applied.
2. Messages are processed in reverse order (newest to oldest).
3. Individual messages are truncated based on max_tokens_per_message. For multimodal messages containing both text and other types of content, only the text content is truncated.
4. The overall conversation history is truncated based on the max_tokens limit. Once the accumulated token count exceeds this limit, the current message being processed get truncated to meet the total token count and any remaining messages get discarded.
5. The truncated conversation history is reconstructed by prepending the messages to a new list to preserve the original message order.

Parameters:

Name	Description
max_tokens_per_message	Type: int | None Default: None
max_tokens	Type: int | None Default: None
min_tokens	Type: int | None Default: None
model	Type: str Default: ‘gpt-3.5-turbo-0613’
filter_dict	Type: dict[str, typing.Any] | None Default: None
exclude_filter	Type: bool Default: True

Instance Methods

apply_transform

apply_transform(self, messages: list[dict[str, Any]]) -> list[dict[str, Any]]

Applies token truncation to the conversation history.

Parameters:

Name	Description
messages	The list of messages representing the conversation history. Type: list[dict[str, typing.Any]]

Returns:

Type	Description
list[dict[str, typing.Any]]	List[Dict]: A new list containing the truncated messages up to the specified token limits.

get_logs

get_logs(
    self,
    pre_transform_messages: list[dict[str, Any]],
    post_transform_messages: list[dict[str, Any]]
) -> tuple[str, bool]

Parameters:

Name	Description
pre_transform_messages	Type: list[dict[str, typing.Any]]
post_transform_messages	Type: list[dict[str, typing.Any]]