Streaming Physics

Simulate realistic LLM streaming timing with configurable time-to-first-token (TTFT), tokens-per-second (TPS), and random jitter. Perfect for testing loading states, progress indicators, and streaming UX under realistic conditions.

StreamingProfile

The streamingProfile option can be set on any fixture to control the timing of streamed chunks.

Programmatic Usage

const mock = new LLMock();
await mock.start();

// Simulate GPT-4 streaming timing
mock.on(
  { userMessage: "hello" },
  { content: "Hello! How can I help you today?" },
  {
    streamingProfile: {
      ttft: 800,    // 800ms before first token
      tps: 50,      // 50 tokens/sec after that
      jitter: 0.2,  // +/-20% variance on each delay
    },
  },
);

JSON Fixture File

{
  "fixtures": [
    {
      "match": { "userMessage": "think carefully" },
      "response": { "content": "Let me think about this..." },
      "streamingProfile": {
        "ttft": 2000,
        "tps": 30,
        "jitter": 0.1
      }
    }
  ]
}

Interaction with latency

• When streamingProfile is set, it takes priority over the latency field.
• If streamingProfile is not set, the existing latency behavior applies (flat delay per chunk).
• If streamingProfile is set but has neither ttft nor tps, it falls back to latency.

Realistic Profiles

Here are some example profiles that approximate real-world LLM behavior:

// Fast model (GPT-4o-mini, Claude 3 Haiku)
{ ttft: 200, tps: 100, jitter: 0.15 }

// Standard model (GPT-4o, Claude 3.5 Sonnet)
{ ttft: 500, tps: 60, jitter: 0.2 }

// Reasoning model (o1, o3, Claude with extended thinking)
{ ttft: 5000, tps: 80, jitter: 0.1 }

// Slow/overloaded (rate-limited or cold start)
{ ttft: 3000, tps: 15, jitter: 0.4 }