Solving LLM Limitations & Building a Serverless AI Agent Locally

In this guide we’ll cover what AI agents are and how an AI agent pipe on Langbase is different from a typical agent. Followed by how a pipe overcomes LLMs’ limitations and will also create a serverless AI agent pipe locally.

What is an AI Agent?

An AI agent is an autonomous software that uses LLMs to handle more than just text generation. We can call them purposely built for real-time interaction that LLMs can’t do.

AI agents can perceive their environment, make decisions, and act autonomously or semi-autonomously, handling tasks like scheduling, managing smart devices, or offering customer support. While LLMs and RAG models excel at processing and generating text, they fall short when it comes to real-world actions that require decision-making and dynamic responses.

AI agents fill this gap by combining decision-making algorithms, task execution, and real-time interaction. They analyze information, adapt to changing conditions, and perform tasks in real time, making them essential for scenarios where quick, informed decisions are needed. Unlike LLMs, agents can take actions based on context, bridging the gap between language understanding and practical execution.

In short, AI agents complement LLMs and RAG models by providing the decision-making and real-time interaction needed for complex, real-world tasks. Together, they form a more complete solution to tackle a wider range of challenges.

What is an AI Agent Pipe and How Is It Different?

A pipe on Langbase is a serverless AI agent as an API, purpose-built to be fast, flexible, and easy to integrate into any app. It simplifies building AI agents/features or RAG apps by abstracting away the complexity of servers, GPUs, and infrastructure. You can connect it to any LLM to assist you.

While typical AI agents might be limited in scope, a pipe is highly customizable. You can fine-tune it with variables, safety prompts, and real-time interactions, making it more dynamic and adaptable to specific tasks. It also supports RAG (Retrieval-Augmented Generation), meaning it can pull from your custom datasets to deliver even smarter results.

In short, a pipe is not just another AI agent—it’s a full AI-powered system that’s developer-first, configurable, and designed to ship AI apps quickly. It stands for:

P → Prompt: Prompt engineering and orchestration.
I → Instructions: Instruction training, few-shot, persona, character, etc.
P → Personalization: Knowledge base, variables, and safety hallucination engine.
E → Engine: Experiments, API Engine, evals, and enterprise governance.

LLMs Limitations

Synchronous Processing: LLMs handle one input at a time, limiting real-time interaction or handling multiple queries simultaneously.
Hallucinations: LLMs sometimes generate incorrect or nonsensical information due to learning or training patterns, not facts.
No Web Access: LLMs can’t retrieve live data from the internet, making their responses outdated for real-time needs.
Weak at Reasoning Queries: LLMs struggle with complex calculations or multi-step math problems.
Non-Deterministic Output: LLM responses can vary even with the same input, making consistent formatting difficult.

How AI Agent Pipes on Langbase Overcome These Limitations:

Serverless RAG: A pipe can be connected to serverless RAG called AI memory which is a managed search engine as an API for developers having vector store, file storage, attribution data, parsing + chunking, and semantic similarity search engine—all at one place. This enables accurate, continuous, and coherent conversations over time.
Asynchronous Processing: Pipes can run tasks simultaneously, improving responsiveness and handling multiple requests in parallel.
Fact-Checking and Real-Time Data: Pipes can connect to live data sources and verify LLM-generated outputs, reducing hallucinations and improving accuracy.
Tool Calls: Pipe lets you add a tool definition schema for self-healing agentic tool calling. Your AI not only uses tools but also detects and corrects its own errors—re-querying or seeking additional data when needed. This makes your AI more reliable and trustworthy. This way an AI can even solve complex mathematical problems instead of hallucinating.
Consistent Output Formatting: Pipes can enforce standard output formats, such as JSON, ensuring uniformity in responses.
Personalized Interactions: Pipes personalize user experiences by retaining context and adjusting responses to user preferences, creating more relevant and customized outputs.

Create a Serverless AI Agent Pipe

You can create a serverless AI Agent pipe either by using:

The Langbase’s AI studio
Create it locally using the web AI framework called BaseAI

Note

Create serverless AI agent pipe

To create an AI agent pipe using Langbase’s AI Studio follow the steps here.

Create it Locally using BaseAI

BaseAI is the first web AI framework built for developers. For more details visit our BaseAI docs.

Let’s start creating a serverless AI agent pipe locally to generate titles:

Step #1: Create a Pipe

Create a project directory in your local machine. Navigate to your project directory, and initialize npm and install packages.

# create a project directory
mkdir my-ai-project

# intialize npm
npm init -y

# install dotenv package
npm install dotenv

Then create a new pipe using the baseai pipe command. It will ask you for the name, description, and other details of the pipe step-by-step. Add the name, description, and set status to private:

npx baseai@latest pipe

Step #2: Add your prompt

It will ask for the system prompt next. We will use the following system prompt to generate title ideas:

You're a helpful AI assistant. Give me 5 title ideas for an article about the topic given by the user.

Enter this prompt in the terminal. It will create a pipe in your current directory under baseai/pipes/ai-title-generator.ts.

Step #3: View and Configure your pipe

Your pipe is created successfully at /baseai/pipes/ai-title-generator.ts. Open the file, it will look like this:

import {PipeI} from '@baseai/core';

const pipeAiTitleGenerator = (): PipeI => ({
	apiKey: process.env.LANGBASE_API_KEY!, // Replace with your API key https://langbase.com/docs/api-reference/api-keys
	name: 'ai-title-generator',
	description: 'Pipe to create title ideas with AI',
	status: 'private',
	model: 'openai:gpt-4o-mini',
	stream: true,
	json: false,
	store: true,
	moderate: true,
	top_p: 1,
	max_tokens: 1000,
	temperature: 0.7,
	presence_penalty: 1,
	frequency_penalty: 1,
	stop: [],
	tool_choice: 'auto',
	parallel_tool_calls: true,
	messages: [
		{
			role: 'system',
			content: `You're a helpful AI assistant. Give me 5 title ideas for an article about the topic given by the user.`,
		},
	],
	variables: [],
	memory: [],
	tools: [],
});

export default pipeAiTitleGenerator;

In the file, you can see the configuration of the pipe. You can modify the configuration as per your requirements. For instance, you can change the model, temperature, max_tokens, etc.

Step #4: Run the Pipe

Now that you have created the pipe, let's see it in action.

Create an index.ts file in your project directory.

touch index.ts

We will use the pipe primitive from @baseai/core to run the pipe. Add the following code to index.ts file:

import {Pipe} from '@baseai/core';
import { pipeAiTitleGenerator } from './baseai/pipes/ai-title-generator';

const pipe = new Pipe(pipeAiTitleGenerator());

async function main() {
	const userMsg = 'Generate 5 blog title ideas for an article about Large Language Models';

	const response = await pipe.run({
		messages: [
			{
				role: 'user',
				content: userMsg,
			},
		],
	});
	console.log('response: ', response);
}

main();

This will generate and print several title ideas for the given topic.

Or to run in streaming mode, check the API reference for the run method.

We have added a demo user message to generate 5 blog title ideas for an article about LLMs. You can change this user message as per your requirements.

Generate 5 blog title ideas for an article about Large Language Models

Since we are using an OpenAI model, your OpenAI key is required. In your project, you will find .env.baseai.example, which contains the example env variables. Create anenv file using the example like this:

cp .env.baseai.example .env

In the file, add your OpenAI API key like this.

# Adding the OpenAI API key in .env file

OPENAI_API_KEY="your-open-ai-key"

Step #5: Start BaseAI Server

To run the pipe locally, you need to start the BaseAI server. Run the following command in your terminal:

npx baseai@latest dev

This command will start the BaseAI server. You will also see the real-time logs in your terminal for observability.

Step #6: Run the Code

In another terminal tab, run the index.ts file using the following command:

npx tsx index.ts

It will prompt the LLM model to get the response for your title generation query. It will look something like this:

Here are five blog title ideas for an article about Large Language Models (LLMs):

1. **"Unlocking the Power of Large Language Models: How They’re Shaping the Future of AI"**
2. **"The Rise of Large Language Models: Transforming Communication and Knowledge Processing"**
3. **"Exploring the Potential of LLMs: From GPT to BERT and Beyond"**
4. **"Large Language Models Explained: What They Are and Why They Matter in AI"**
5. **"Demystifying LLMs: How These AI Models Are Revolutionizing Industries"**

This all happens locally on your machine and the response should be streamed in your terminal.

After testing locally, you can deploy this pipe seamlessly to Langbase using the deploy command. But before that, make sure you are authenticated with the BaseAI CLI.

npx baseai@latest deploy

And your AI agent pipe will be deployed on Langbase. For more details read here.

General Guides