Streaming & Vercel AI SDK

Why Streaming Matters

Streaming Architecture

LLM generation is slow. Waiting 10 seconds for a full paragraph to generate before displaying it results in a poor user experience. Streaming allows tokens to be rendered on the UI the moment they are generated by the model.

The Vercel AI SDK

The Vercel AI SDK has become the industry standard for building AI interfaces in TypeScript/React. It abstracts away the complexity of handling HTTP Server-Sent Events (SSE) and managing UI state.

Core Modules

• ai/core: Unified API (generateText, streamText) that works across OpenAI, Anthropic, Google, etc.
• ai/react: React hooks (useChat, useCompletion) for managing streaming state.
• ai/rsc: React Server Components integration for streaming UI components directly from the server.

1// app/api/chat/route.ts
2import { openai } from '@ai-sdk/openai';
3import { streamText } from 'ai';
4
5export async function POST(req: Request) {
6  const { messages } = await req.json();
7
8  const result = await streamText({
9    model: openai('gpt-4-turbo'),
10    messages,
11  });
12
13  return result.toDataStreamResponse();
14}
15
16// app/page.tsx
17'use client';
18import { useChat } from 'ai/react';
19
20export default function Chat() {
21  const { messages, input, handleInputChange, handleSubmit } = useChat();
22
23  return (
24    <div>
25      {messages.map(m => (
26        <div key={m.id}>{m.role}: {m.content}</div>
27      ))}
28      <form onSubmit={handleSubmit}>
29        <input value={input} onChange={handleInputChange} />
30      </form>
31    </div>
32  );
33}