Tool Architecture

The TanStack AI tool system provides a powerful, flexible architecture for enabling AI agents to interact with external systems:

Server Tools execute securely on the backend with automatic handling
Client Tools execute in the browser for UI updates and local operations
The Agentic Cycle enables multi-step reasoning and complex workflows
Tool States provide real-time feedback and enable robust UIs
Approval Flow gives users control over sensitive operations This architecture enables building sophisticated AI applications that can:
Fetch data from APIs and databases
Perform calculations and transformations
Update UIs and manage state
Execute multi-step workflows
Require user approval for sensitive actions

Call Flow: Client to LLM Service

When a user sends a message that requires tool usage, the following flow occurs:

mermaid

sequenceDiagram
    participant User
    participant Browser
    participant Server
    participant LLM Service
    
    User->>Browser: Types message
    Browser->>Server: POST /api/chat<br/>{messages, ...}
    Server->>Server: Build tool definitions<br/>from tool array
    Server->>LLM Service: Send request with:<br/>- messages<br/>- tool definitions<br/>- model config
    
    Note over LLM Service: Model analyzes tools<br/>and decides to use one
    
    LLM Service-->>Server: Stream chunks:<br/>tool_call, content, done
    Server-->>Browser: Forward chunks via SSE/HTTP
    Browser->>Browser: Parse chunks &<br/>update UI
    Browser->>User: Show response

Step-by-Step Breakdown

User Input: The user types a message in the chat interface
Client Request: The browser sends a POST request to the server with:
- Current conversation history (messages)
- Optional data payload (body)
Server Processing: The server:
- Receives the request
- Extracts messages from the request body
- Converts tool definitions into the LLM's expected format
- Sends the request to the LLM service (OpenAI, Anthropic, etc.)
LLM Decision: The LLM service:
- Analyzes the conversation and available tools
- Decides whether to call a tool based on the user's request
- Generates tool calls with arguments
Streaming Response: The LLM streams back chunks:
- tool_call chunks with tool name and arguments
- content chunks with text responses
- done chunk when complete
Client Updates: The browser receives chunks and updates the UI in real-time

Code Example

Server (API Route):

import { chat, toServerSentEventsResponse } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { getWeather, sendEmail } from "./tools";

export async function POST(request: Request) {
  const { messages } = await request.json();

  // Create streaming chat with tools
  const stream = chat({
    adapter: openaiText("gpt-5.5"),
    messages,
    tools: [getWeather, sendEmail], // Tool definitions passed here
  });

  return toServerSentEventsResponse(stream);
}

Client (React Component):

tsx

import { useState } from "react";
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

function ChatComponent() {
  const [input, setInput] = useState("");
  const { messages, sendMessage, isLoading } = useChat({
    connection: fetchServerSentEvents("/api/chat"),
  });

  return (
    <div>
      {messages.map((message) => (
        <div key={message.id}>{/* Render message */}</div>
      ))}
      <form
        onSubmit={(e) => {
          e.preventDefault();
          sendMessage(input);
          setInput("");
        }}
      >
        <input value={input} onChange={(e) => setInput(e.target.value)} />
        <button type="submit" disabled={isLoading}>
          Send
        </button>
      </form>
    </div>
  );
}

States and Lifecycle

Tools progress through different states during their lifecycle. Understanding these states helps build robust UIs and debug tool execution.

Two parts, two state sets — this page is the canonical reference. Call states (awaiting-input, input-streaming, input-complete, approval-requested, approval-responded) live on the tool-call part as part.state. There is no complete/error/executing/cancelled value on the call part. The result lives on a separate sibling tool-result part whose own state is streaming, complete, or error; the resolved value is also mirrored onto the call part's part.output.

The diagram below is conceptual: the nodes after approval-responded (executing, success, error, cancelled) are not ToolCallState values — they correspond to the sibling tool-result part's state (complete / error) and the call part's output field.

mermaid

stateDiagram-v2
    state "tool-call part (ToolCallState)" as Call {
        [*] --> AwaitingInput: tool_call received
        AwaitingInput --> InputStreaming: partial arguments
        InputStreaming --> InputComplete: all arguments received
        InputComplete --> ApprovalRequested: needsApproval=true
        ApprovalRequested --> ApprovalResponded: user approves / denies
    }
    InputComplete --> ResultComplete: needsApproval=false, success
    InputComplete --> ResultError: parsing, validation, or execution error
    ApprovalResponded --> ResultComplete: approved + success (output set)
    ApprovalResponded --> ResultError: approved + error
    ApprovalResponded --> Denied: user denied (no execution)
    state "tool-result part" as Results {
        ResultComplete: complete
        ResultError: error
    }
    ResultComplete --> [*]
    ResultError --> [*]
    Denied --> [*]

Call States

State	Description	Client Action
awaiting-input	Tool call received, no arguments yet	Show loading
input-streaming	Partial arguments being received	Show progress
input-complete	All arguments received	Ready to execute
approval-requested	Waiting for user approval	Show approval UI
approval-responded	User has approved/denied	Execute or cancel

Result States

State	Description	Client Action
streaming	Result being streamed (future feature)	Show progress
complete	Result is complete	Show result
error	Error occurred before or during execution	Show error message

Monitoring Tool States in React

tsx

import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
import { createChatClientOptions } from "@tanstack/ai-client";
import { getWeather, sendEmail } from "./tools";
import { ApprovalUI } from "./approval-ui";

// Wiring `tools` is what lets `part.name` / `part.input` / `part.output`
// narrow to each tool's types below.
const chatOptions = createChatClientOptions({
  connection: fetchServerSentEvents("/api/chat"),
  tools: [getWeather, sendEmail],
});

function ChatComponent() {
  const { messages } = useChat(chatOptions);

  return (
    <div>
      {messages.map((message) => (
        <div key={message.id}>
          {message.parts.map((part) => {
            if (part.type === "tool-call") {
              return (
                <div key={part.id} className="tool-status">
                  {/* Show state-specific UI */}
                  {part.state === "awaiting-input" && (
                    <div>🔄 Calling {part.name}...</div>
                  )}
                  {part.state === "input-streaming" && (
                    <div>📥 Receiving arguments...</div>
                  )}
                  {part.state === "input-complete" && (
                    <div>✓ Arguments ready</div>
                  )}
                  {part.state === "approval-requested" && (
                    <ApprovalUI part={part} />
                  )}
                </div>
              );
            }
            if (part.type === "tool-result") {
              return (
                <div key={part.toolCallId}>
                  {part.state === "complete" && (
                    <div>✓ Tool completed</div>
                  )}
                  {part.state === "error" && (
                    <div>❌ Error: {part.error}</div>
                  )}
                </div>
              );
            }
            return null;
          })}
        </div>
      ))}
    </div>
  );
}

Approval Flow

For sensitive operations, tools can require user approval before execution:

mermaid

sequenceDiagram
    participant User
    participant Client
    participant Server
    participant LLM
    participant Tool
    
    LLM->>Server: tool_call: send_email
    Server->>Server: Check needsApproval
    Server->>Client: approval-requested chunk
    Client->>Client: Show approval UI
    User->>Client: Clicks "Approve"
    Client->>Server: POST approval response
    Server->>Tool: execute(args)
    Tool-->>Server: result
    Server->>LLM: tool_result
    LLM-->>Client: Generate response

Define tool with approval:

import { toolDefinition } from "@tanstack/ai";
import { z } from "zod";
import { emailService } from "./email-service";

const sendEmailDef = toolDefinition({
  name: "send_email",
  description: "Send an email",
  inputSchema: z.object({
    to: z.string().email(),
    subject: z.string(),
    body: z.string(),
  }),
  needsApproval: true, // Requires user approval
});

const sendEmail = sendEmailDef.server(async ({ to, subject, body }) => {
  await emailService.send({ to, subject, body });
  return { success: true };
});

Handle approval in client:

tsx

const { messages, addToolApprovalResponse } = useChat({
  connection: fetchServerSentEvents("/api/chat"),
});

// In your render (guard `type` and `approval` so `part.approval.id` is safe):
{part.type === "tool-call" &&
  part.state === "approval-requested" &&
  part.approval && (
    <div>
      <p>Approve sending email to {part.input.to}?</p>
      <button
        onClick={() =>
          addToolApprovalResponse({
            id: part.approval.id,
            approved: true,
          })
        }
      >
        Approve
      </button>
      <button
        onClick={() =>
          addToolApprovalResponse({
            id: part.approval.id,
            approved: false,
          })
        }
      >
        Deny
      </button>
    </div>
  )}

Hybrid Tools (Server + Client)

Some tools need to execute in both environments:

import { toolDefinition } from "@tanstack/ai";
import { z } from "zod";
import { db } from "./db";
import { i18n } from "./i18n";

// Server: Fetch data from database
const fetchUserPrefsDef = toolDefinition({
  name: "fetch_user_preferences",
  description: "Get user preferences from server",
  inputSchema: z.object({
    userId: z.string(),
  }),
});

const fetchUserPreferences = fetchUserPrefsDef.server(async ({ userId }) => {
  const prefs = await db.userPreferences.findUnique({ where: { userId } });
  return prefs;
});

// Client: Apply preferences to UI
const applyPrefsDef = toolDefinition({
  name: "apply_preferences",
  description: "Apply user preferences to the UI",
  inputSchema: z.object({
    theme: z.string(),
    language: z.string(),
  }),
});

// On client, create client implementation
const applyPreferences = applyPrefsDef.client(async ({ theme, language }) => {
  // Update UI state with preferences
  document.body.className = theme;
  i18n.changeLanguage(language);
  return { applied: true };
});

// Usage: LLM can chain these together
// 1. Call fetchUserPreferences (server)
// 2. Call applyPreferences with the result (client)

Parallel Tool Execution

The LLM can call multiple tools in parallel for efficiency:

mermaid

graph TD
    A[LLM decides to call 3 tools] --> B[tool_call index: 0]
    A --> C[tool_call index: 1]
    A --> D[tool_call index: 2]
    B --> E[Execute in parallel]
    C --> E
    D --> E
    E --> F[Collect all results]
    F --> G[Continue with results]

Example:

plaintext

User: "Compare the weather in NYC, SF, and LA"

LLM calls:
- get_weather({city: "NYC"}) [index: 0]
- get_weather({city: "SF"}) [index: 1]
- get_weather({city: "LA"}) [index: 2]

All execute simultaneously, then LLM generates comparison.

The server tools of one model turn start together, and the model gets their results in the order of its calls. To run them one at a time, see Run tools one at a time.