Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.edgee.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

The stream() method is used to make streaming chat completion requests to the Edgee AI Gateway. It returns an AsyncGenerator<StreamChunk> that yields response chunks as they arrive from the API.

​
Arguments

The stream() method accepts two arguments:
ParameterTypeDescription
model stringThe model identifier to use (e.g., "openai/gpt-5.2")
input string | InputObjectThe input for the completion. Can be a simple string or a structured InputObject

​
Input Types

​
String Input

When input is a string, it’s automatically converted to a user message: 
for await (const chunk of edgee.stream('gpt-5.2', 'Tell me a story')) {
  if (chunk.text) {
    process.stdout.write(chunk.text);
  }
  
  if (chunk.finishReason) {
    console.log(`\nFinished: ${chunk.finishReason}`);
  }
}
// Equivalent to: input: { messages: [{ role: 'user', content: 'Tell me a story' }] }

​
InputObject

When input is an InputObject, you have full control over the conversation:
PropertyTypeDescription
messages Message[]Array of conversation messages
toolsTool[]Array of function tools available to the model
tool_choiceToolChoiceControls which tool (if any) the model should call. See Tools documentation for details
tagsstring[]Optional tags to categorize and label the request for analytics and filtering. Can also be sent via the x-edgee-tags header (comma-separated)
compression_modelstringCompression model for this request: "agentic", "claude", "opencode", "cursor", or "customer". Each model is a bundle of compression strategies. Overrides API key settings when present.
compression_configurationobjectConfiguration for the compression model. Currently only available for agentic. Contains optional rate (0.0-1.0, default 0.8) and semantic_preservation_threshold (0-100).
For details about Message type, see the Send Method documentation. For details about Tool and ToolChoice types, see the Tools documentation. Example - Streaming with Messages: 
for await (const chunk of edgee.stream('gpt-5.2', {
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Write a poem about coding' }
  ]
})) {
  if (chunk.text) {
    process.stdout.write(chunk.text);
  }
}

​
Return Value

The stream() method returns an AsyncGenerator<StreamChunk>. Each chunk contains incremental updates to the response.

​
StreamChunk Object

Each chunk yielded by the generator has the following structure:
PropertyTypeDescription
choicesStreamChoice[]Array of streaming choices (typically one)
compressionCompression | nullToken compression metrics (if compression was applied)

​
StreamChoice Object

Each choice in the choices array contains:
PropertyTypeDescription
indexnumberThe index of this choice in the array
deltaStreamDeltaThe incremental update to the message
finish_reasonstring | null | undefinedReason why the generation stopped. Only present in the final chunk. Possible values: "stop", "length", "tool_calls", "content_filter", or null
Example - Handling Multiple Choices: 
for await (const chunk of edgee.stream('gpt-5.2', 'Give me creative ideas')) {
  chunk.choices.forEach((choice, index) => {
    if (choice.delta.content) {
      console.log(`Choice ${index}: ${choice.delta.content}`);
    }
  });
}

​
StreamDelta Object

The delta object contains incremental updates:
PropertyTypeDescription
rolestring | undefinedThe role of the message (typically "assistant"). Only present in the first chunk
contentstring | undefinedIncremental text content. Each chunk contains a portion of the full response
tool_callsToolCall[] | undefinedArray of tool calls (if any). See Tools documentation for details

​
Convenience Properties

The StreamChunk class provides convenience getters for easier access:
PropertyTypeDescription
textstring | nullShortcut to choices[0].delta.content - the incremental text content
rolestring | nullShortcut to choices[0].delta.role - the message role (first chunk only)
finishReasonstring | nullShortcut to choices[0].finish_reason - the finish reason (final chunk only)
Example - Using Convenience Properties: 
for await (const chunk of edgee.stream('gpt-5.2', 'Explain quantum computing')) {
  // Content chunks
  if (chunk.text) {
    process.stdout.write(chunk.text);
  }

  // First chunk contains the role
  if (chunk.role) {
    console.log(`Role: ${chunk.role}`);
  }

  // Last chunk contains finish reason
  if (chunk.finishReason) {
    console.log(`\nFinish reason: ${chunk.finishReason}`);
  }
}

​
Understanding Streaming Behavior

​
Chunk Structure

  1. First chunk: Contains role (typically "assistant") and may contain initial content
  2. Content chunks: Contain incremental content updates
  3. Final chunk: Contains finish_reason indicating why generation stopped
Example - Collecting Full Response: 
let fullText = '';

for await (const chunk of edgee.stream('gpt-5.2', 'Tell me a story')) {
  if (chunk.text) {
    fullText += chunk.text;
    process.stdout.write(chunk.text); // Also display as it streams
  }
}

console.log(`\n\nFull response (${fullText.length} characters):`);
console.log(fullText);

​
Finish Reasons

ValueDescription
"stop"Model generated a complete response and stopped naturally
"length"Response was cut off due to token limit
"tool_calls"Model requested tool/function calls
"content_filter"Content was filtered by safety systems
nullGeneration is still in progress (not the final chunk)

​
Empty Chunks

Some chunks may not contain content. This is normal and can happen when:
  • The chunk only contains metadata (role, finish_reason)
  • The chunk is part of tool call processing
  • Network buffering creates empty chunks
Always check for chunk.text before using it: 
for await (const chunk of edgee.stream('gpt-5.2', 'Hello')) {
  if (chunk.text) {  // ✅ Good: Check before using
    console.log(chunk.text);
  }
  // ❌ Bad: console.log(chunk.text) - may log null
}

​
Error Handling

The stream() method can throw errors: 
try {
  for await (const chunk of edgee.stream('gpt-5.2', 'Hello!')) {
    if (chunk.text) {
      process.stdout.write(chunk.text);
    }
  }
} catch (error) {
  if (error instanceof Error) {
    // API errors: "API error {status}: {message}"
    // Network errors: Standard fetch errors
    console.error('Stream failed:', error.message);
  }
}