opto.trainer.algorithms.aggregator¶

AggregatedUpdate ¶

AggregatedUpdate(
    agent,
    optimizer,
    use_asyncio: bool = True,
    logger=None,
    llm: AbstractModel = None,
    max_tokens: int = 4096,
    *args,
    **kwargs
)

Bases: Minibatch

The algorithm applies the optimizer to propose updates for each instance in the minibatch independently. The updates are then aggregated using an LLM and applied to the agent.

aggregator_system_prompt `class-attribute` `instance-attribute` ¶

aggregator_system_prompt = f'You are an expert in aggregating suggestions. You will see a list of suggestions of parameters from different people (denoted as #SuggestedValue_i). A parameter is represented as a dict, where the key is the name of a parameter component, and the value is the component value.

        Your task is to aggregate the suggestions and provide a new value for the parameter. Please consider the following:
        1. Make sure the new values in the dict is in the same format as the values in the dict of the suggested parameters.
        2. Provide a new value to consolidate the suggestions considering on their confidence scores. The suggestions can be wrong (especially the ones with low confidence).
        3. When aggregating, try to find the common ground between the suggestions.


        Output_format: Your output should be in the following json format, satisfying the json syntax:

            {
            "reasoning_<component_1>": <Your reasoning>,
            "reasoning_<component_2>": <Your reasoning>,
            "suggestion": {
                <component_1>: <suggested_value_1>,
                <component_2>: <suggested_value_2>,
            }
            }

            In "reasoning", explain the problem your thought process and how you arrive at the new value.

            In "suggestion", write down the suggested values. For each key in #CurrentValue, you should write the new value in the format of python code without syntax errors. If you don't want to change a variable, just write down its current value.

            If no changes or answer are needed, just output TERMINATE.
        '

llm `instance-attribute` ¶

llm = llm or LLM()

max_tokens `instance-attribute` ¶

max_tokens = max_tokens

train ¶

train(
    guide,
    train_dataset,
    *,
    stepsize=0.5,
    num_epochs: int = 1,
    batch_size: int = 1,
    test_dataset=None,
    eval_frequency: int = 1,
    log_frequency: Union[int, None] = None,
    min_score: Union[int, None] = None,
    verbose: Union[bool, str] = False,
    **kwargs
)

forward ¶

forward(agent, x, guide, info, verbose=False)

Run the agent, compute feedback and return the new parameters for an instance in the minibatch.

to_param_dict ¶

to_param_dict(update_dict)

Convert the update the dict {ParameterNode:Any} to a dict {str:Any}.

update ¶

update(outputs, verbose=False)

Ask LLM to aggregate the new parameter suggestions.

construct_update_dict ¶

construct_update_dict(
    parameters: List[ParameterNode],
    suggestion: Dict[str, Any],
    ignore_extraction_error: bool = True,
) -> Dict[ParameterNode, Any]

Convert the suggestion in text into the right data type.

extract_llm_suggestion ¶

extract_llm_suggestion(
    response: str, ignore_extraction_error: bool = True
) -> Dict[str, Any]

Extract the suggestion from the response.