Agent Run API

Request content type. Needs to be application/json

Replace <YOUR_API_KEY> with your user/org API key.

LLM API key for the LLM being used in the request.

interface AgentRunOptions {
	model: string;
	input: string | Array<InputMessage>;
    instructions?: string;
	stream?: boolean;
	tools?: Tool[];
	tool_choice?: 'auto' | 'required' | ToolChoice;
	parallel_tool_calls?: boolean;
	top_p?: number;
	max_tokens?: number;
	temperature?: number;
	presence_penalty?: number;
	frequency_penalty?: number;
	stop?: string[];
	customModelParams?: Record<string, any>;
}

Following are the properties of the options object.

LLM model. Combination of model provider and model id, like openai:gpt-4o-mini

Format: provider:model_id

You can copy the ID of a model from the list of supported LLM models at Langbase.

A string (for simple text queries) or an array of input messages.

When using a string, it will be treated as a single user message. Use it for simple queries. For example:

langbase.agent.run({
    input: 'What is an AI Agent?',
    ...
});

When using an array of input messages InputMessage[]. Each input message should include the following properties:

interface InputMessage {
	role: 'user' | 'assistant' | 'system'| 'tool';
	content: string | ContentType[] | null;
	name?: string;
	tool_call_id?: string;
}

langbase.agent.run({
    input: [
        {
            role: 'user',
            content: 'What is an AI Agent?',
        },
    ],
    ...
});

Used to give high level instructions to the model about the task it should perform, including tone, goals, and examples of correct responses.

This is equivalent to a system/developer role message at the top of LLM's context.

Whether to stream the response or not. If true, the response will be streamed.

A list of tools the model may call.

interface ToolsOptions {
	type: 'function';
	function: FunctionOptions
}

Tool usage configuration.

Call multiple tools in parallel, allowing the effects and results of these function calls to be resolved in parallel.

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random. Lower values like 0.2 will make it more focused and deterministic.

Default: 0.7

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

Default: 1

Maximum number of tokens in the response message returned.

Default: 1000

Penalizes a word based on its occurrence in the input text.

Default: 0

Penalizes a word based on how frequently it appears in the training data. Default: 0

Up to 4 sequences where the API will stop generating further tokens.

Additional parameters to pass to the model as key-value pairs. These parameters are passed on to the model as-is.

interface CustomModelParams {
	[key: string]: any;
}

Example:

{
	"logprobs": true,
	"service_tier": "auto",
}

The generated text response (also called completion) from the agent. It can be a string or null if the model called a tool.

The ID of the raw response.

The object type name of the response.

The timestamp of the response creation.

The model used to generate the response.

A list of chat completion choices. Can contain more than one elements if n is greater than 1.

interface ChoiceGenerate {
	index: number;
	message: Message;
	logprobs: boolean | null;
	finish_reason: string;
}

The usage object including the following properties.

interface Usage {
	prompt_tokens: number;
	completion_tokens: number;
	total_tokens: number;
}

This fingerprint represents the backend configuration that the model runs with.

The different headers of the response.

The different headers of the response.

Stream is an object with a streamed sequence of StreamChunk objects.

type StreamResponse = ReadableStream<StreamChunk>;

StreamChunk