In this blog, we will build an AI Video Wisdom Extraction Tool using the Langbase SDK. This tool will:
- Extract wisdom from a YouTube video
- Answer questions related to the video content
- Generate a summary of the video
- List main ideas and key points
- Extract quotes and key phrases
- Provide a list of references and resources
- Highlight the wow moments in the video
- Write Tweets from the video content
We will create a basic Next.js application that will use the Langbase SDK to connect to the AI Pipes and stream the final response back to user.
Let's get started!
Step 0: Create a Next.js Application
To build the agent, we need to have a Next.js starter application. If you don't have one, you can create a new Next.js application using the following command:
1npx create-next-app@latest video-wisdom
2
3# or with pnpm
4pnpx create-next-app@latest video-wisdomThis will create a new Next.js application in the video-wisdom directory. Navigate to the directory and start the development server:
1cd video-wisdom
2npm run dev
3
4# or with pnpm
5pnpm run devStep 1: Install Langbase SDK
Install the Langbase SDK in this project using npm or pnpm.
1npm install langbase
2
3# or with pnpm
4pnpm add langbaseStep 2: Fork the AI pipes
Fork the following AI Pipes in ⌘ Langbase dashboard. These Pipes will power the Video Wisdom Extraction Tool:
- YouTube Videos Q/A Pipe - Answers questions related to the video content.
- Summarize YouTube Video Pipe - Generates a summary of the video.
- Main Ideas Extractor Pipe - Lists main ideas and key points.
- List Interesting Facts Pipe - Extracts interesting facts from the video.
- Wow Moments Extractor Pipe - Highlights the wow moments in the video.
- Video Tweets Extractor Pipe - Writes Tweets from the video content.
- Video Recommendations Extractor Pipe - Provides a list of references and resources from the video.
- List Quotes from Video Pipe - Extracts quotes and key phrases from the video.
When you fork a Pipe, navigate to the API tab located in the Pipe's navbar. There, you'll find API keys specific to each Pipe, which are essential for making calls to the Pipes using the Langbase SDK.
Create a .env.local file in the root directory of your project and add the following environment variables:
1# Replace `LB_SUMMARIZE_PIPE_KEY` with your API from the forked Summary Pipe
2LB_SUMMARIZE_PIPE_KEY=""
3
4# Replace `LB_GENERATE_PIPE_KEY` with your API from the forked YouTube Videos Q/A Pipe
5LB_GENERATE_PIPE_KEY=""
6
7# Replace `LB_MAIN_IDEAS_PIPE_KEY` with your API from the forked Main Ideas Extractor Pipe
8LB_MAIN_IDEAS_PIPE_KEY=""
9
10# Replace `LB_FACTS_PIPE_KEY` with your API from the forked List Interesting Facts Pipe
11LB_FACTS_PIPE_KEY=""
12
13# Replace `LB_WOW_PIPE_KEY` with your API from the forked Wow Moments Extractor Pipe
14LB_WOW_PIPE_KEY=""
15
16# Replace `LB_TWEETS_PIPE_KEY` with your API from the forked Video Tweets Extractor Pipe
17LB_TWEETS_PIPE_KEY=""
18
19# Replace `LB_RECOMMENDATION_PIPE_KEY` with your API from the forked Video Recommendations Extractor Pipe
20LB_RECOMMENDATION_PIPE_KEY=""
21
22# Replace `LB_QUOTES_PIPE_KEY` with your API from the forked List Quotes from Video Pipe
23LB_QUOTES_PIPE_KEY=""Step 3: Create Wisdom Extraction API Route
Create a new file app/api/langbase/wisdom/route.ts. This API route will call the Langbase AI Pipes to extract wisdom from the YouTube video.
First we define the GenerationType enum and the getEnvVar function to get the environment variable based on the type of the Pipe we want to call. We will specify the type of the Pipe in the request body from the UI. This function will return the environment variable based on the type.
1// Enum for type.
2enum GenerationType {
3 Generate = 'generate',
4 Summarize = 'summarize',
5 Quotes = 'quotes',
6 Recommendation = 'recommendation',
7 MainIdeas = 'mainIdeas',
8 Facts = 'facts',
9 Wow = 'wow',
10 Tweets = 'tweets'
11}
12
13// Get the environment variable based on type.
14const getEnvVar = (type: GenerationType) => {
15 switch (type) {
16 case GenerationType.Generate:
17 return process.env.LB_GENERATE_PIPE_KEY;
18 case GenerationType.Summarize:
19 return process.env.LB_SUMMARIZE_PIPE_KEY;
20 case GenerationType.Quotes:
21 return process.env.LB_QUOTES_PIPE_KEY;
22 case GenerationType.Recommendation:
23 return process.env.LB_RECOMMENDATION_PIPE_KEY;
24 case GenerationType.MainIdeas:
25 return process.env.LB_MAIN_IDEAS_PIPE_KEY;
26 case GenerationType.Facts:
27 return process.env.LB_FACTS_PIPE_KEY;
28 case GenerationType.Wow:
29 return process.env.LB_WOW_PIPE_KEY;
30 case GenerationType.Tweets:
31 return process.env.LB_TWEETS_PIPE_KEY;
32 default:
33 return null;
34 }
35};We also define the RequestBody type and the requestBodySchema schema for the request body of the API route.
1import z from 'zod'; // For schema validation
2
3// Schema for request body
4const requestBodySchema = z.object({
5 prompt: z.string(),
6 transcript: z.string().trim().min(1),
7 type: z.enum([
8 GenerationType.Generate,
9 GenerationType.Summarize,
10 GenerationType.Quotes,
11 GenerationType.Recommendation,
12 GenerationType.MainIdeas,
13 GenerationType.Facts,
14 GenerationType.Wow,
15 GenerationType.Tweets
16 ])
17});
18
19// Type for request body
20type RequestBody = z.infer<typeof requestBodySchema>;Add the route code to the app/api/langbase/wisdom/route.ts file:
1/**
2 * This API route calls the Langbase AI Pipes to extract wisdom from the YouTube video.
3 *
4 * @param {NextRequest} req - The request object.
5 * @returns {Response} The response object streaming the final response back to the frontend.
6 */
7export async function POST(req: NextRequest) {
8 try {
9 // Extract the prompt from the request body
10 const reqBody: RequestBody = await req.json();
11 const parsedReqBody = requestBodySchema.safeParse(reqBody);
12
13 // If the request body is not valid
14 if (!parsedReqBody.success || !parsedReqBody.data) {
15 throw new Error(parsedReqBody.error.message);
16 }
17
18 // Extract the prompt from the request body
19 const { prompt, transcript, type } = parsedReqBody.data;
20
21 // Get the environment variable based on type.
22 const pipeKey = getEnvVar(type);
23
24 // If the Pipe API key is not found, throw an error.
25 if (!pipeKey) {
26 throw new Error('Pipe API key not found');
27 }
28
29 // Generate the response and stream from Langbase Pipe.
30 return await generateResponse({ prompt, transcript, pipeKey });
31 } catch (error: any) {
32 return new Response(error.message, { status: 500 });
33 }
34}
35
36/**
37 * Generates a response by initiating a Pipe, constructing the input for the stream,
38 * generating a stream by asking a question, and returning the stream in a readable stream format.
39 * @param {Object} options - The options for generating the response.
40 * @param {string} options.transcript - The transcript to be used as user input or variable value.
41 * @param {string} options.prompt - The prompt to be used as user input or variable value.
42 * @param {string} options.pipeKey - The API key for the Pipe.
43 * @returns {Response} The response stream in a readable stream format.
44 */
45async function generateResponse({
46 transcript,
47 prompt,
48 pipeKey
49}: {
50 transcript: string;
51 prompt: string;
52 pipeKey: string;
53}) {
54 // 1. Initiate the Pipe.
55 const pipe = new Pipe({
56 apiKey: pipeKey
57 });
58
59 // 2. Construct the input for the stream
60 // 2a. If we have prompt, we pass 'transcript' as a variable.
61 // This is useful when we want to use the transcript as a variable in the prompt.
62 // Used with the question answers Pipe.
63 // 2b. Otherwise we pass 'transcript' as user input.
64 let streamInput: StreamOptions;
65 if (!prompt) {
66 streamInput = {
67 messages: [{ role: 'user', content: transcript }]
68 };
69 } else {
70 streamInput = {
71 messages: [{ role: 'user', content: prompt }],
72 variables: { transcript: transcript }
73 };
74 }
75
76 // 2. Generate a stream by asking a question
77 const { stream } = await pipe.streamText(streamInput);
78
79 // 3. Done, return the stream in a readable stream format.
80 return new Response(stream.toReadableStream());
81}Here is a quick explanation of what's happening in the code above:
- We extract the prompt, transcript, and type from the request body.
- We get the environment variable based on the type of the Pipe we want to call.
- We initiate the Pipe with the API key using the Langbase SDK.
- We construct the input for the stream. If we have a prompt, we pass the transcript as a variable. Otherwise, we pass the transcript as user input.
- We generate a stream by asking a question using the Langbase SDK.
- We return the stream in a readable stream format.
That's it! You have successfully created an AI Video Wisdom Extraction Tool using the Langbase SDK. You can connect the API routes to the frontend and start extracting wisdom from YouTube videos.
Live demo
You can try out the live demo of the VideoWisdom here.
