Skip to content
WP EngineDocumentation

Stream a chat completion

Set "stream": true on a chat completion request to receive the response as Server-Sent Events (SSE) instead of a single JSON payload. You can then render tokens in your user interface as the model produces them.

SSE chunks don’t arrive aligned on event boundaries. A single read can contain several complete data: lines plus a partial line, or only a fragment of one. The steps below read the stream incrementally and parse only complete lines.

Prerequisites

Section titled “Prerequisites”
  • An API key with Chat set to Write. Full access includes this permission. Either key type works. An Account key (wpe_) suits a backend service because it doesn’t depend on one person’s access.
  • The key, stored in an AI_API_KEY environment variable on the server that runs this code. Never send the key to a browser.
  • For the JavaScript example, Node.js 18 or later. For the PHP example, PHP 8.0 or later and Guzzle.

1. Request the stream

Section titled “1. Request the stream”

Set stream: true in the request body. The API responds with Content-Type: text/event-stream and sends the body as a sequence of data: { ... } events instead of a single JSON object.

const BASE_URL = "https://api.ai.wpengine.com";


const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.AI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "google/gemini-3.8-flash",
    stream: true,
    messages: [{ role: "user", content: prompt }],
  }),
});


if (!res.ok || !res.body) {
  throw new Error(`stream request failed: ${res.status}`);
}

2. Read the response incrementally

Section titled “2. Read the response incrementally”

Read the body as it arrives instead of waiting for it to complete.

Each read can return zero or more complete lines plus a trailing partial line. Buffer that remainder and prepend it to the next read.

Decode with the streaming TextDecoder instead of converting each chunk independently. It preserves multi-byte UTF-8 sequences across chunk boundaries.

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";


while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });


  // continue to step 3 with `buffer`
}

3. Parse SSE lines and detect completion

Section titled “3. Parse SSE lines and detect completion”

Handle each complete line as follows:

  • Skip any line that doesn’t start with data: .
  • Treat the sentinel data: [DONE] as the end of the stream.
  • Parse any other line as a JSON chat.completion.chunk and pass it to the caller.

If the stream fails after it starts, the gateway sends a chunk with an error field instead, followed by [DONE]. Throw on that chunk so your code doesn’t mistake a failed response for a complete one. For details, see streaming errors.

Split the buffer on newlines. Hold back the last element, which can be incomplete, for the next read.

const lines = buffer.split("\n");
buffer = lines.pop() ?? "";


for (const line of lines) {
  if (!line.startsWith("data: ")) continue;
  const data = line.slice(6).trim();
  if (data === "[DONE]") return;
  const chunk = JSON.parse(data);
  if (chunk.error) throw new Error(chunk.error.message);
  onDelta(chunk);
}

4. Call it and handle each delta

Section titled “4. Call it and handle each delta”

The function below combines steps 1–3. It takes a prompt and a callback. The function throws if the gateway sends an error chunk, or if the body ends without data: [DONE]. It never returns a partial answer as if it were complete.

This example writes each delta to stdout as it arrives.

const BASE_URL = "https://api.ai.wpengine.com";


async function chatStream(prompt, onDelta) {
  const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.AI_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "google/gemini-3.8-flash",
      stream: true,
      messages: [{ role: "user", content: prompt }],
    }),
  });


  if (!res.ok || !res.body) {
    throw new Error(`stream request failed: ${res.status}`);
  }


  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";


  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });


    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";


    for (const line of lines) {
      if (!line.startsWith("data: ")) continue;
      const data = line.slice(6).trim();
      if (data === "[DONE]") return;
      const chunk = JSON.parse(data);
      if (chunk.error) throw new Error(chunk.error.message);
      onDelta(chunk);
    }
  }


  throw new Error("stream ended before [DONE]");
}


await chatStream("Explain how a CDN works in about 100 words.", (chunk) => {
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
});

Save it as stream.mjs and run node stream.mjs.

5. Observe the output

Section titled “5. Observe the output”

Run the code from step 4 and watch stdout. Tokens print incrementally as the model produces them, not all at once at the end.

If nothing appears until the whole response completes, something in your stack is buffering the stream. Check for a proxy or middleware that buffers responses. Confirm that your code reads the body incrementally instead of awaiting the full body first: with res.body.getReader(), or with Utils::readLine() and Guzzle’s stream option.

Stream from WordPress

Section titled “Stream from WordPress”

The WordPress HTTP API doesn’t support SSE, and the WordPress AI Client doesn’t expose a streaming method. To stream tokens into a WordPress site, run code like the examples above in a service outside the WordPress request. Then push tokens to the browser over your own channel.

Last updated: