OpenAI Releases GPT-Realtime-2.1 and GPT-Realtime-2.1-mini for Low-Latency Voice Agents API

OpenAI has released two new Realtime models to its API. They are named gpt-realtime-2.1 again gpt-realtime-2.1-mini. Both are aimed at low voice and multimodal experiences. The smaller model is a notable part of this release. It is a small real-time voice reasoning model. Shipping is the same as before gpt-realtime-mini. OpenAI also reduces p95 latency by at least 25% for all Realtime voice models. That reduction comes from improved caching.

What is GPT-Realtime-2.1-mini

gpt-realtime-2.1-mini is a small conceptual model of real-time voice interaction. It responds to audio and text input with a live connection. OpenAI ranks it as the fastest, most cost-effective option on the list.

The Realtime API processes and generates audio in a single model. This avoids combining separate speech-to-text and text-to-speech systems. That single-model design reduces latency and preserves diversity in speech.

Thinking is the key skill here. It means that the model can think inside before speaking. The mini tier also supports the use of tools, or calling a function, through the Realtime API. Together this allows a small model to plan a step, call your function, and then respond.

Big brother gpt-realtime-2.1. Updates GPT-Realtime-2 with improved alphanumeric recognition. It also improves quietness and handling of noise, and disturbance behavior. It supports speech with adjustable thinking effort, following instructions, and using a tool.

The fastest way to choose between them: use gpt-realtime-2.1 if you want real-time robust reasoning, tool use, instruction following, and voice agent behavior. Use it gpt-realtime-2.1-mini if you’re looking for a faster, more economical option.

Voice agents often stop during a tool call. The model opens a function call, then shuts down. Users think that the call is rejected and it disturbs. That creates incomplete results and a confused discussion environment.

Consultation with a verbal antecedent corrects this pattern. A model might say ‘I’ll check that order now’ before acting. It keeps talking while running the application. That behavior keeps multi-step voice operations consistent.

Thinking effort is adjustable at all levels. Developers can choose small, low, medium, high, or xhigh. Low is the default and keeps the latency low for easy turns. High effort increases latency and output token consumption. OpenAI advises starting from scratch for many production voice agents.

Latency and Cache Optimization

p95 latency is the 95th-percentile response time. It captures the slow tail that users experience. A cut of at least 25% of that tail helps the voice live. Improved caching drives this reduction in all Realtime voice models.

Temporarily saves costs, not just delays. Cached input tokens are charged at a steep discount. Because gpt-realtime-2.1-minicached audio inputs go down to $0.30 per 1M. New audio inputs cost $10.00 per 1M in comparison. Longer sessions are more beneficial, because the system requests the cache after opening the first opportunity.

Price and comparison

Priced in 1M tokens, divided into text, audio, and image. The mini maintains the previous small scale while adding reflection. The table below lists the published statistics.

The minimum issuance rate is $20.00 per 1 million. Full gpt-realtime-2.1 cost $64.00 same. That’s about a 3x gap in sound output. Teams can trade certain strengths for lower costs while maintaining flexibility.

Use Cases with examples

• Rating of customer support: Caller reports phone payment error. A small model causes trouble with little effort. It costs a lookup_account tool, then a check_invoice a tool. It narrates each step to keep the caller informed.
• Scheduling an appointment: The user requests to submit a reservation for next Tuesday. The model captures the exact date character by character. It verifies the information, then calls a reschedule work. Guaranteed values ​​prevent tool calls on guessed input.
• An in-app voice assistant: Mobile app streams microphone audio via WebRTC. A small model answers product questions in one or two short sentences. Low cost allows the feature to work at high volume.
• Field data capture: The technician asks the agent to write part of the number. Advanced alphanumeric recognition helps capture codes like ‘8-3-5-7-1’. The model reads the value back for validation before taking action.

Small Implementation

Browser clients connect via WebRTC. Your server creates a temporary client secret first. The browser then connects directly to the Realtime API. Server media pipes use WebSockets, and telephony uses SIP.

First, the server creates a temporary client secret. Save your custom API key on the server. Session optimization sets up a model, a low conceptual effort, and a single tool.

// Server: mint a short-lived client secret (API key stays server-side)
const r = await fetch(" {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    session: {
      type: "realtime",
      model: "gpt-realtime-2.1-mini",
      instructions: "You are a support agent. Reply in one or two short sentences.",
      reasoning: { effort: "low" },
      tools: [
        {
          type: "function",
          name: "lookup_account",
          description: "Look up a customer account by email.",
          parameters: {
            type: "object",
            properties: { email: { type: "string" } },
            required: ["email"]
          }
        }
      ],
      tool_choice: "auto"
    }
  })
});
const { value: EPHEMERAL_KEY } = await r.json(); // pass this to the browser

Next, the browser opens a connection to the WebRTC peer. Add a microphone track and data channel for events. It then sends its SDP offer to the call center.

// Browser: connect to the Realtime API over WebRTC
// EPHEMERAL_KEY comes from your server endpoint above.
const pc = new RTCPeerConnection();

const audioEl = document.createElement("audio");
audioEl.autoplay = true;
pc.ontrack = (e) => { audioEl.srcObject = e.streams[0]; };

const mic = await navigator.mediaDevices.getUserMedia({ audio: true });
pc.addTrack(mic.getTracks()[0]);

const events = pc.createDataChannel("oai-events");
events.addEventListener("message", (e) => console.log(JSON.parse(e.data)));

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

const sdp = await fetch(" {
  method: "POST",
  body: offer.sdp,
  headers: {
    Authorization: `Bearer ${EPHEMERAL_KEY}`,
    "Content-Type": "application/sdp"
  }
});
await pc.setRemoteDescription({ type: "answer", sdp: await sdp.text() });

Start with the effort of thinking down, and then lift it up only for difficult tasks. Add short instructions that separate strict rules from defaults. Run evals before and after any model migration.

Strengths and Weaknesses

Power:

• Thinking is now reaching the low-cost sub-category.
• The price is the same as before gpt-realtime-mini average.
• p95 latency reduced by at least 25% for all Realtime voice models.
• Tunable processing effort trades latency for the depth of each task.
• A single-model audio pipeline keeps conversations natural.

Weaknesses:

• The audio token price is difficult to convert into a cost per call.
• A higher thinking effort raises both latency and output tokens.
• Long sessions resend context and increase installation costs without pruning.
• A mini tier trades a specific skill against a full one gpt-realtime-2.1.

Interactive Descriptor

Check it out Technical details here. Also, feel free to follow us Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to Our newspaper. Wait! are you on telegram? now you can join us on telegram too.

Need to work with us on developing your GitHub Repo OR Hug Face Page OR Product Release OR Webinar etc.? Connect with us