OpenAI Async API Calls in Python: Fix the Await TypeError

Why does "object can't be used in await expression" happen?

The full message is usually "TypeError: object OpenAIObject can't be used in 'await' expression" (v0.x) or "object ChatCompletion can't be used in 'await' expression" (v1.x). It means you wrote `await` in front of something that is not awaitable.

A synchronous call blocks until the HTTP response arrives and then returns a finished object. By the time `await` sees it, there is nothing left to wait for, and Python rejects it. Putting `async def` around your function does not change what the inner call does.

Check your version first with `pip show openai` (or `python -c "import openai; print(openai.__version__)"`). The fix differs between 0.x and 1.x.

How do I fix it for each SDK version?

On openai 1.x, import the async client and await its methods. The call shape is identical to the sync client; only the class and the `await` differ.

On openai 0.x, the async variant is a separate method named `acreate`. That old interface (`openai.ChatCompletion`) was removed in 1.0, so prefer upgrading.

# openai >= 1.0
from openai import AsyncOpenAI
client = AsyncOpenAI()  # reads OPENAI_API_KEY

async def ask(prompt):
    resp = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content

# openai < 1.0 (legacy)
# resp = await openai.ChatCompletion.acreate(model="gpt-3.5-turbo", messages=[...])
# text = resp["choices"][0]["message"]["content"]

How do I run OpenAI async API calls in Python with asyncio.gather?

Create one coroutine per prompt, then pass them all to `asyncio.gather`. They run concurrently on one thread, because the time is spent waiting on the network, not computing.

The pipeline below reuses one client, caps concurrency with a semaphore, keeps results in input order, and records failures per item instead of losing the whole batch.

import asyncio
from openai import AsyncOpenAI

MODEL = "gpt-4o-mini"
MAX_CONCURRENCY = 10

async def generate_answer(client, sem, idx, prompt):
    async with sem:
        try:
            resp = await client.chat.completions.create(
                model=MODEL,
                messages=[{"role": "user", "content": prompt}],
                temperature=0.3,
                max_tokens=300,
            )
            return {"index": idx, "text": resp.choices[0].message.content, "error": None}
        except Exception as e:
            return {"index": idx, "text": None, "error": repr(e)}

async def main(prompts):
    sem = asyncio.Semaphore(MAX_CONCURRENCY)
    async with AsyncOpenAI(max_retries=5, timeout=60.0) as client:
        tasks = [generate_answer(client, sem, i, p) for i, p in enumerate(prompts)]
        return await asyncio.gather(*tasks)

if __name__ == "__main__":
    prompts = [f"Summarise slide {i} in one sentence." for i in range(1, 31)]
    results = asyncio.run(main(prompts))
    for r in results:
        print(r["index"], r["text"] or r["error"])

How do I avoid 429 rate limit errors with concurrent requests?

Limits are measured per organisation and model in requests per minute (RPM) and tokens per minute (TPM), and they rise with your usage tier. Firing 200 requests at once can exceed RPM, and long prompts exhaust TPM even at low request counts. Your actual numbers are shown on the limits page of your account, so read them there rather than guessing.

A semaphore limits how many requests are in flight, which is the practical control. As a rule of thumb, start at 5 to 10, and raise it while watching for 429s. Concurrency of N with average latency L seconds gives roughly 60*N/L requests per minute, so 10 concurrent calls at 3 seconds each is about 200 RPM.

The SDK already retries 408, 409, 429 and 5xx errors and connection failures with exponential backoff. The default is `max_retries=2` and the default timeout is 10 minutes. Raise retries on the client, or per call with `client.with_options(max_retries=5)`.

If you need custom logic, such as logging each retry, wrap the call with tenacity and set the client's max_retries to 0 so retries do not multiply.

from tenacity import retry, wait_random_exponential, stop_after_attempt, retry_if_exception_type
from openai import RateLimitError, APITimeoutError

@retry(
    retry=retry_if_exception_type((RateLimitError, APITimeoutError)),
    wait=wait_random_exponential(min=1, max=30),
    stop=stop_after_attempt(6),
)
async def call_with_backoff(client, **kwargs):
    return await client.chat.completions.create(**kwargs)

What are the most common mistakes with async OpenAI code?

Most failures come from a short list of habits. This table maps each to its fix.

What if I cannot convert my code to async?

If a large codebase uses the sync client or a library that blocks, run the sync calls in worker threads. The calls are I/O bound, so threads give real concurrency.

`asyncio.to_thread` fits inside async code. A ThreadPoolExecutor fits plain scripts. The sync client is thread-safe, so share one instance.

import asyncio
from concurrent.futures import ThreadPoolExecutor
from openai import OpenAI

client = OpenAI()

def ask(prompt):
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return r.choices[0].message.content

async def main(prompts):
    return await asyncio.gather(*(asyncio.to_thread(ask, p) for p in prompts))

# or, without asyncio:
with ThreadPoolExecutor(max_workers=10) as pool:
    results = list(pool.map(ask, ["hi", "hello"]))

When should I not use async calls, and what about slides, streaming and the Batch API?

Do not parallelise when prompt N needs the output of prompt N-1, such as a running summary or a consistent narrative across slides. Those must run sequentially, or you can pass a short shared outline to every independent call instead.

For non-urgent bulk work, the Batch API is 50% cheaper and completes within a 24 hour window. You upload a JSONL file of requests, poll the batch, and download results matched by `custom_id`. Use it for overnight document processing, and use async when a person is waiting.

For slides, extract text with python-pptx (`Presentation(path).slides`, then each shape's `text_frame.text` where `shape.has_text_frame`). Build `prompts` as a list in slide order, set `max_tokens` and a low temperature, and remember that cost scales with tokens: 100 parallel calls cost the same as 100 sequential ones, only faster.

For streaming, pass `stream=True` and iterate with `async for`. For structured output, the async client supports `client.chat.completions.parse` with a Pydantic model as `response_format`, or JSON mode. Close the client with `await client.close()` unless you use `async with`.

async def stream_answer(client, prompt):
    stream = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
    )
    async for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")