Skip to main content

Streaming Chatbot on CloudBase: Three Implementation Paths

In one sentence: There are three main paths to connect a "stream as it generates" chat interface to a CloudBase backend — the official cloudbase-agent-ui Mini Program component, hand-written SSE + Cloud Function, or Vercel AI SDK. This guide helps you pick one based on your frontend type and protocol requirements, and provides minimal runnable code for each.

Estimated time: 30–60 minutes (depending on which path you choose) | Difficulty: Advanced

Applicable Scenarios​

  • You want a streaming chat experience where users read output as it generates, not waiting several seconds for a full response
  • Your backend is on CloudBase (Cloud Function / CloudBase Run)
  • Your frontend can be WeChat Mini Program / Web / Next.js / Vue / React Native
  • Your LLM can be Hunyuan / DeepSeek / any OpenAI-compatible provider

Three Paths Overview​

PathBest forStrengthsLimitations
A. cloudbase-agent-ui (official component)WeChat Mini Program developersOfficial component; streaming response, multi-turn conversation, file upload, voice, and tool calling all encapsulated; three chatMode options: Hunyuan / DeepSeek / AgentCurrently only the WeChat Mini Program version is released — no Web or React version yet
B. Hand-written SSE + Cloud FunctionExisting Web / Vue / Next frontend with custom protocol requirementsFully controllable; free choice of frontend-backend protocol; compatible with any clientStream parsing, reconnection, and abort handling are your responsibility
C. Vercel AI SDKExisting Next.js / Vue projects; full-stack React teamsuseChat hook provides great DX; error handling and abort are built into the SDKNot usable in Mini Programs; ai@4 and ai@5+ have incompatible protocols — pin the version

Each path has its own dedicated section below. Read only the one you need.

Important: cloudbase-agent-ui is currently distributed as WeChat Mini Program source code components (not an npm package, and no React/Web version exists). This section covers Mini Program scenarios only. If your frontend is not a Mini Program, skip to Path B or C.

Step 1: Create an AI service in the CloudBase Console​

Open WeChat DevTools and click "CloudBase" at the top to enable the service, or go to the CloudBase Platform. Two AI service types are available:

  • Direct LLM access (chatMode: 'model'): connects directly to Hunyuan / DeepSeek, works out of the box
  • Framework-based Agent (chatMode: 'bot'): build an Agent on the Agent platform, bind a prompt / tools / Knowledge Base, and get a botId like agent-xxxxxxx

See Agent Development Documentation for details.

Step 2: Copy the component into your Mini Program project​

cloudbase-agent-ui is not published to npm — it is distributed as source code:

  1. git clone https://github.com/TencentCloudBase/cloudbase-agent-ui or download agent-ui.zip from the GitHub Releases page

  2. Copy the components/agent-ui directory from the repo into your Mini Program project at components/agent-ui/

  3. Initialize the CloudBase environment in miniprogram/app.js:

    App({
      onLaunch() {
        wx.cloud.init({
          env: 'your-env-id', // CloudBase environment ID
          traceUser: true,
        });
      },
    });

Step 3: Register and use the component in a page​

Register in the page .json:

{
  "usingComponents": {
    "agent-ui": "/components/agent-ui/index"
  }
}

Reference in the page .wxml:

<view>
  <agent-ui
    chatMode="{{chatMode}}"
    agentConfig="{{agentConfig}}"
    modelConfig="{{modelConfig}}"
    showBotAvatar="{{showBotAvatar}}"
  ></agent-ui>
</view>

Configure in the page .js (pick one of the two chatMode options):

// Option 1: Direct LLM access (simplest)
Page({
  data: {
    chatMode: 'model',
    showBotAvatar: true,
    modelConfig: {
      modelProvider: 'cloudbase',
      quickResponseModel: 'deepseek-v4-flash',
      logo: '',
      welcomeMsg: 'Hello, I am your assistant',
    },
  },
});
// Option 2: Framework-based Agent (create the Agent in the Console first)
Page({
  data: {
    chatMode: 'bot',
    showBotAvatar: true,
    agentConfig: {
      botId: 'agent-xxxxxxx',  // copy from the Agent list in the Console
      tools: [
        // optional: frontend tools that the Agent can call
        {
          name: 'get_weather',
          description: 'Get the weather for a specified city',
          parameters: {
            type: 'object',
            properties: { city: { type: 'string' } },
            required: ['city'],
          },
          handler: ({ city }) => `${city}: sunny, 25°C`,
        },
      ],
    },
  },
});

The full props (chatMode / agentConfig / modelConfig / showBotAvatar) and their accepted values are defined in the repo README for the current version. Streaming response, multi-turn conversation, file upload, voice input, and tool calling are all handled inside the component — your business code never touches SSE.

Step 4: Configure the Mini Program server domain (allowlist)​

In WeChat Public Platform admin: Development Management → Server domain → request legal domain, add:

https://{your-envid}.api.tcloudbasegateway.com

Without this, file upload and multi-session features will not work.

Path B: Hand-written SSE + Cloud Function (custom scenario)​

Use this path when: the protocol is not AG-UI protocol, the frontend is not a Mini Program, or you need deep customization of the frontend-backend communication.

Backend Web Cloud Function​

mkdir chat-backend && cd chat-backend
npm init -y
npm install express

index.js:

const express = require('express');
const app = express();
app.use(express.json());

app.post('/api/chat', async (req, res) => {
  const { messages } = req.body;

  // Call upstream LLM (reuse the proxy URL from connect-openai-api-cloud-function)
  const upstream = await fetch(process.env.LLM_PROXY_URL + '/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${process.env.LLM_PROXY_TOKEN}`,
    },
    body: JSON.stringify({
      model: 'gpt-4o-mini',
      messages,
      stream: true,
    }),
  });

  if (!upstream.ok) {
    res.status(upstream.status).json({ error: await upstream.text() });
    return;
  }

  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  const reader = upstream.body.getReader();
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    res.write(value); // upstream is already in SSE format, proxy it as-is
  }
  res.end();
});

app.listen(process.env.PORT || 9000);

Deploy to CloudBase Web Cloud Function:

tcb fn deploy chat-backend --httpFn -e your-env-id

After deployment, configure LLM_PROXY_URL / LLM_PROXY_TOKEN in the Console under "Cloud Functions → chat-backend". Note the HTTP trigger URL, e.g. https://your-env.service.tcloudbase.com/chat-backend/api/chat.

Frontend​

Web (plain React / Vue, using fetch + ReadableStream):

const res = await fetch('/api/chat', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ messages: [{ role: 'user', content: 'Hello' }] }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  // SSE events are separated by \n\n
  const events = buffer.split('\n\n');
  buffer = events.pop(); // last chunk may be incomplete — carry it to the next iteration

  for (const event of events) {
    for (const line of event.split('\n')) {
      if (!line.startsWith('data:')) continue;
      const data = line.slice(5).trim();
      if (data === '[DONE]') return;
      try {
        const obj = JSON.parse(data);
        const delta = obj.choices?.[0]?.delta?.content;
        if (delta) console.log(delta); // accumulate into state
      } catch {}
    }
  }
}

WeChat Mini Program (wx.request + enableChunked, when not using cloudbase-agent-ui):

const requestTask = wx.request({
  url: 'https://your-env.service.tcloudbase.com/chat-backend/api/chat',
  method: 'POST',
  enableChunked: true,
  data: { messages: [{ role: 'user', content: 'Hello' }] },
  responseType: 'arraybuffer',
});

requestTask.onChunkReceived((res) => {
  const text = String.fromCharCode.apply(null, new Uint8Array(res.data));
  // parse SSE as above, accumulate delta into page data
});

See the wx.request enableChunked documentation.

Path C: Vercel AI SDK (existing Next.js projects)​

Version note: This code is based on ai@4.x. ai@5+ is released and introduces breaking changes to useChat / streamText protocols. For new projects, refer to the current version of the Vercel AI SDK official docs.

Use this path when: you have an existing Next.js / Vue project using Vercel AI SDK 4.x, or you are migrating from Vercel to CloudBase.

Install dependencies​

npm install ai@4 @ai-sdk/openai

Backend: Next.js Route Handler​

app/api/chat/route.ts:

import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';

export const runtime = 'nodejs'; // Do not use edge — CloudBase Run runs a Node container
export const maxDuration = 60;

const llm = createOpenAI({
  baseURL: process.env.LLM_PROXY_URL,   // https://xxx.service.tcloudbase.com/llm-proxy/v1
  apiKey: process.env.LLM_PROXY_TOKEN,  // proxy auth token, not a real OpenAI key
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: llm('gpt-4o-mini'),
    system: 'You are a helpful assistant.',
    messages,
  });

  return result.toDataStreamResponse();
}

createOpenAI({ baseURL }) redirects requests to your own proxy. The real OpenAI key lives inside the Cloud Function proxy layer (see connect-openai-api-cloud-function) — invisible to the frontend or Next.js.

Backend: Web Cloud Function version (for non-Next.js frontends)​

const express = require('express');
const { createOpenAI } = require('@ai-sdk/openai');
const { streamText } = require('ai');

const app = express();
app.use(express.json({ limit: '5mb' }));

const llm = createOpenAI({
  baseURL: process.env.LLM_PROXY_URL,
  apiKey: process.env.LLM_PROXY_TOKEN,
});

app.post('/api/chat', async (req, res) => {
  const { messages } = req.body;
  const result = await streamText({
    model: llm('gpt-4o-mini'),
    messages,
    onFinish: async ({ text }) => {
      // optional: persist to database after streaming ends
    },
  });
  result.pipeDataStreamToResponse(res);
});

app.listen(process.env.PORT || 9000);

Deploy:

tcb fn deploy chat-backend --httpFn -e your-env-id

Configure LLM_PROXY_URL / LLM_PROXY_TOKEN in the Console.

Frontend: useChat​

app/chat/page.tsx:

'use client';
import { useChat } from 'ai/react';

export default function ChatPage() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, error, stop } = useChat({
    api: '/api/chat', // for cross-service calls, use the Web Cloud Function URL
  });

  return (
    <div style={{ maxWidth: 720, margin: '0 auto', padding: 24 }}>
      {messages.map((m) => (
        <div key={m.id} style={{ margin: '12px 0' }}>
          <strong>{m.role === 'user' ? 'You' : 'Assistant'}:</strong>
          <span>{m.content}</span>
        </div>
      ))}

      {error && <div style={{ color: 'red' }}>Error: {error.message}</div>}

      <form onSubmit={handleSubmit} style={{ display: 'flex', gap: 8 }}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Say something..."
          disabled={isLoading}
          style={{ flex: 1, padding: 8 }}
        />
        <button type="submit" disabled={isLoading}>Send</button>
        {isLoading && <button type="button" onClick={stop}>Stop</button>}
      </form>
    </div>
  );
}

For Vue, use useChat from @ai-sdk/vue — the API shape is identical.

Deploy​

Deploy Next.js as a single unit to CloudBase Run:

tcb cloudrun deploy --port 3000

See Deploy Next.js to CloudBase Run. In CloudBase Run "Service Settings → Environment Variables", add LLM_PROXY_URL (must end with /v1 because @ai-sdk/openai automatically appends /chat/completions) and LLM_PROXY_TOKEN.

How to Choose​

Your situationPick
WeChat Mini Program frontendA (cloudbase-agent-ui)
Want the fastest chat UI without implementing the protocol yourselfA (Mini Program) or C (Web)
Protocol must be custom, or frontend is non-Next.js Web / React NativeB
Existing Next.js + Vercel AI SDK projectC
Trying everything — where to startMini Program: A; Web: start with C, switch to B if customization is needed

Common Errors​

SymptomCauseFix
No token-by-token display — output appears all at once after several secondsSSE not actually active / intermediate layer is buffering the streamConfirm backend sends Content-Type: text/event-stream + Cache-Control: no-cache; disable response buffering on custom domain / CDN; verify first with the default *.service.tcloudbase.com domain
Path A: component not showing or "agent-ui not registered" errorPage .json missing usingComponents declaration, or components/agent-ui not copiedCheck directory structure + page usingComponents path
Path A: botId wrong, Agent not foundAgent not created in Console, or ID mistypedCloudBase Console → AI → Agent list, copy the actual ID (format: agent-xxxxxxx)
Path A: request not in domain listServer domain not added to allowlistWeChat Public Platform admin → Server domain → add https://{envid}.api.tcloudbasegateway.com
Path B: Mini Program wx.request error url not in domain listSame as aboveSame fix; also add the Web Cloud Function access domain (*.service.tcloudbase.com)
Path B: backend stream drops without the frontend knowingUpstream 5xx causes SSE to breakIn the backend catch, write data: {"error":"..."}\n\n then call end() to notify the frontend
Path C: useChat is not exported from 'ai/react'ai is not v4 (v5+ changed import paths)npm install ai@^4 @ai-sdk/openai@^1; do not mix v5 import paths
Path C: useChat sends GET instead of POST, returns 404App Router Route Handler missing the POST exportCheck route.ts for export async function POST spelling
Path C: Failed to fetch cross-origin errorFrontend domain != backend domain, CORS not configuredAdd CORS middleware to Express: Access-Control-Allow-Origin / Methods / Headers, handle OPTIONS
Path C: long conversation hits context_length_exceededFull message history sent every timeFrontend limits to the last N messages, or backend compresses with summarization

Next Steps​