OpenAI Maximum Context Length Exceeded: Causes and Fixes

What does the "maximum context length exceeded" error mean?

The context window is the total number of tokens a model can handle in one request. It covers your system message, your document, any chat history, and the completion the model is about to write. If the sum goes over the limit, the API rejects the request with a message like "This model's maximum context length is N tokens. However, you requested M tokens (X in the messages, Y in the completion)."

That message is the diagnosis. X is your prompt, Y is your max_tokens (or the completion allowance), and X + Y must be at most N. People often fix X and forget that a large max_tokens eats the same budget.

Old tutorials quote a 2049 token limit. That belonged to early GPT-3 models. Current models have windows in the tens or hundreds of thousands of tokens, and some go beyond that, but the number differs per model and changes over time. Look up the limit for the exact model string you call rather than assuming one.

How do I count tokens with tiktoken before sending?

Tokenizers split text into subword pieces, so characters and words are only estimates. In English, 1 token is roughly 4 characters or 0.75 words. Non-English text, code, JSON and text with unusual symbols usually cost more tokens per character, sometimes twice as many.

tiktoken is OpenAI's open-source tokenizer. Use encoding_for_model so you get the right encoding for your model, and fall back to a general encoding if the model name is newer than your installed version.

import tiktoken

def get_encoder(model: str):
    try:
        return tiktoken.encoding_for_model(model)
    except KeyError:
        return tiktoken.get_encoding("o200k_base")  # fallback

def count_tokens(text: str, model: str = "gpt-4o") -> int:
    return len(get_encoder(model).encode(text))

print(count_tokens("Hello world"))

Which strategy should I use: truncate, map-reduce, refine or RAG?

Truncating is the worst default because it silently drops everything past the cut. It is only acceptable when the important content is at the start, such as an abstract or a news lead.

Long-context models change the calculus. If the whole document fits, one call avoids chunking bugs and merge errors, and the model sees cross-references between sections. The tradeoffs are cost (you pay for every input token) and latency, and models can miss details buried in the middle of very long inputs, so verify quality on your own documents.

When it does not fit, pick by goal.

How do I summarize text longer than the context limit?

Chunk, map, reduce. Split the text on paragraph or sentence boundaries into chunks of roughly 1,000 to 3,000 tokens, with 10 to 15% overlap so ideas that cross a boundary appear in both chunks. Summarize each chunk independently (the map step), then summarize the joined summaries (the reduce step). If the joined summaries still exceed your budget, repeat the reduce step recursively.

Budget explicitly: chunk_tokens + prompt overhead + max_tokens must stay below the window. The example below does that and retries on 429 rate limits.

import re, time, tiktoken
from concurrent.futures import ThreadPoolExecutor
from openai import OpenAI, RateLimitError

client = OpenAI()
MODEL = "gpt-4o-mini"          # cheap model for map step
FINAL_MODEL = "gpt-4o"         # stronger model for the final merge
enc = tiktoken.get_encoding("o200k_base")
tok = lambda s: len(enc.encode(s))

CHUNK, OVERLAP, OUT = 2000, 250, 400

def chunk_text(text):
    # split on paragraphs, then sentences for oversized paragraphs
    units = []
    for p in re.split(r"\n\s*\n", text):
        if tok(p) <= CHUNK:
            units.append(p)
        else:
            units += re.split(r"(?<=[.!?])\s+", p)
    chunks, cur, n = [], [], 0
    for u in units:
        t = tok(u)
        if n + t > CHUNK and cur:
            chunks.append("\n\n".join(cur))
            tail, m = [], 0           # carry overlap from the end
            for x in reversed(cur):
                if m + tok(x) > OVERLAP: break
                tail.insert(0, x); m += tok(x)
            cur, n = tail, m
        cur.append(u); n += t
    if cur: chunks.append("\n\n".join(cur))
    return chunks

def ask(model, system, text, max_tokens=OUT):
    for attempt in range(6):
        try:
            r = client.chat.completions.create(
                model=model, temperature=0.2, max_tokens=max_tokens,
                messages=[{"role": "system", "content": system},
                          {"role": "user", "content": text}])
            return r.choices[0].message.content
        except RateLimitError:
            time.sleep(2 ** attempt)   # exponential backoff
    raise RuntimeError("rate limited")

MAP_SYS = "Summarize this excerpt faithfully in under 150 words. Use only facts in the text."
REDUCE_SYS = "Merge these partial summaries into one summary. Do not add facts not present."

def summarize(text, style="a concise TL;DR of 5 bullet points"):
    parts = chunk_text(text)
    with ThreadPoolExecutor(4) as ex:
        sums = list(ex.map(lambda c: ask(MODEL, MAP_SYS, c), parts))
    # recurse while the combined summaries do not fit comfortably
    while tok("\n\n".join(sums)) > 6000:
        groups = chunk_text("\n\n".join(sums))
        sums = [ask(MODEL, REDUCE_SYS, g) for g in groups]
    final_sys = f"{REDUCE_SYS} Write the result as {style}."
    return ask(FINAL_MODEL, final_sys, "\n\n".join(sums), max_tokens=500)

How do I clean PDF text before chunking?

PDF extraction is where many summaries go wrong. Repeated headers, footers and page numbers inflate tokens and leak into summaries. Hyphenated line breaks ("inter-\nnational") create broken words. Multi-column layouts can interleave columns into nonsense, and scanned PDFs have no text layer at all and need OCR first.

Fix these before counting tokens, then use real structure (headings, pages, sections) as chunk boundaries instead of arbitrary offsets.

1. Extract text per page (for example with pypdf or pdfplumber) and OCR scanned pages first if the text layer is empty.
2. Remove lines that repeat on most pages (headers, footers) and standalone page numbers.
3. Rejoin hyphenated words split across lines and collapse single newlines inside paragraphs.
4. Detect headings and split there first, falling back to paragraphs, then sentences.
5. Spot check a few chunks by eye; garbled multi-column text is easier to catch now than in the summary.

import re

def clean(text):
    text = re.sub(r"(\w)-\n(\w)", r"\1\2", text)      # rejoin hyphenation
    text = re.sub(r"^\s*(Page )?\d+\s*$", "", text, flags=re.M)  # page numbers
    text = re.sub(r"(?<!\n)\n(?!\n)", " ", text)       # single newlines
    return re.sub(r"[ \t]+", " ", text).strip()

How do I control the style and avoid hallucinations in the merge step?

Put instructions in the system message and apply the final style (TL;DR, executive brief, "explain it to a second grader") only in the reduce step. If you apply it in the map step, you compress style and facts together and lose detail early.

Use low temperature (0 to 0.3) for faithful summaries and set a target length in words, not just "short". Most hallucination in map-reduce appears at the merge, where the model smooths over gaps. Tell it to use only the provided summaries, and for important documents keep names, numbers and dates in the map output so they survive.

Common mistakes to avoid: ignoring the completion budget, splitting mid-sentence, using no overlap, and trusting the final summary without spot checks against the source.

Is there a shortcut with LangChain, and what are the alternatives?

LangChain wraps this pattern. The chain types are "stuff" (one call), "map_reduce" and "refine". APIs move between versions, so treat the snippet as a starting point and adapt it to the version you have installed.

In the snippet, text is the cleaned document string from the earlier clean() step, loaded here from a file. Chains built with load_summarize_chain expect a dict with an input_documents key, not a bare list of documents.

If the user wants answers about the document rather than a full summary, use RAG: embed chunks, retrieve the few most relevant ones per question and answer from them. To shrink very large inputs cheaply before summarizing, extractive pre-filtering (TextRank or similar sentence ranking) keeps only the key sentences.

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains.summarize import load_summarize_chain
from langchain_openai import ChatOpenAI
from langchain_core.documents import Document

# the cleaned document string (e.g. the output of clean() above)
text = open("doc.txt", encoding="utf-8").read()

splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    chunk_size=2000, chunk_overlap=250)
docs = [Document(page_content=t) for t in splitter.split_text(text)]

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.2)
chain = load_summarize_chain(llm, chain_type="map_reduce")
print(chain.invoke({"input_documents": docs})["output_text"])