autogen.code_utils.eval_function_completions

eval_function_completions

eval_function_completions(
    responses: list[str],
    definition: str,
    test: str | None = None,
    entry_point: str | None = None,
    assertions: str | Callable[[str], tuple[str, float]] | None = None,
    timeout: float | None = 3,
    use_docker: bool | None = True
) -> dict

(openai<1) Select a response from a list of responses for the function completion task (using generated assertions), and/or evaluate if the task is successful using a gold test.

Parameters:

Name	Description
responses	The list of responses. Type: list[str]
definition	The input definition. Type: str
test	The test code. Type: str | None Default: None
entry_point	The name of the function. Type: str | None Default: None
assertions	The assertion code which serves as a filter of the responses, or an assertion generator. When provided, only the responses that pass the assertions will be considered for the actual test (if provided). Type: str | Callable[[str], tuple[str, float]] | None Default: None
timeout	The timeout for executing the code. Type: float | None Default: 3
use_docker	Whether to use docker for code execution. Type: bool | None Default: True

Returns:

Type	Description
dict	dict: The success metrics.