tqdm Progress Bar New Line Every Update: Causes and Fixes

Why do I get a tqdm progress bar new line on every update?

A tqdm progress bar new line on every update means the redraw trick has failed. tqdm does not clear the screen. It writes the bar text followed by a carriage return (\r), which moves the cursor to column 0, and the next update overwrites the same row. This only works if the terminal treats \r as a cursor move and nothing else has moved the cursor to a new row.

Four things break that: the rendered bar is wider than the terminal so it wraps, the output device ignores \r (a debug console, a log file, a CI capture), another print or log line lands between updates, or your code builds a fresh bar on each iteration so each one prints its own line.

Because each cause leaves a different signature, diagnose in order of likelihood rather than changing flags at random.

How do I diagnose it quickly?

Run this checklist top to bottom. It takes a minute and identifies the cause in most cases.

import sys
print('isatty:', sys.stdout.isatty(), sys.stderr.isatty())
import shutil
print('columns:', shutil.get_terminal_size().columns)

Cause 1: the bar is created more than once

The classic mistake is wrapping a tqdm inside a callback or inner loop. Each construction prints a fresh bar at 0 percent or at its first update, so you see a stack of lines and a tqdm progress bar new line per call.

The fix is one object, a known total, and update(1) per finished unit of work.

# Before: new bar every call
def cb(result):
    for _ in tqdm(range(1)):
        pass

# After: one bar, created once
from tqdm import tqdm
pbar = tqdm(total=200, dynamic_ncols=True)
def cb(result):
    pbar.update(1)
# ... run the work ...
pbar.close()

How do I show one progress bar for a backtrader optstrategy run?

cerebro.optstrategy expands the parameter ranges into a Cartesian product, so the total is the product of the range lengths. Create the bar before cerebro.run(), register cerebro.optcallback(cb), and close the bar in a finally block so a crash does not leave the cursor mid-line.

The callback runs in the main process each time a run finishes and receives that run's result. Because workers finish out of order and maxcpus controls how many run at once, treat the callback as a completion counter, not as an index into your grid. Do not assume the order matches submission.

Strategy code running in workers inherits the console, so any print() inside next() or notify_order() will break the bar. Silence those during optimization or route them through a flag.

import sys
import backtrader as bt
from tqdm import tqdm

class SmaCross(bt.Strategy):
    params = dict(fast=5, slow=20, verbose=False)
    def __init__(self):
        f = bt.ind.SMA(period=self.p.fast)
        s = bt.ind.SMA(period=self.p.slow)
        self.cross = bt.ind.CrossOver(f, s)
    def next(self):
        if self.p.verbose:
            print(self.data.datetime.date(0))  # off during optimization
        if self.cross > 0 and not self.position:
            self.buy()
        elif self.cross < 0 and self.position:
            self.close()

def main(data):
    fast = range(5, 15)
    slow = range(20, 40)
    total = len(fast) * len(slow)

    cerebro = bt.Cerebro(optreturn=True)
    cerebro.adddata(data)
    cerebro.optstrategy(SmaCross, fast=fast, slow=slow)

    pbar = tqdm(total=total, file=sys.stdout, dynamic_ncols=True,
                leave=True, position=0, desc='optimize')
    cerebro.optcallback(lambda result: pbar.update(1))
    try:
        results = cerebro.run(maxcpus=4)
    finally:
        pbar.close()
    return results

if __name__ == '__main__':
    main(bt.feeds.YahooFinanceCSVData(dataname='prices.csv'))

How do I stop prints and logs from breaking the bar?

Anything written to the console between two updates moves the cursor to a new row, so the next redraw lands below it. Use tqdm.write(), which clears the bar, prints the message, and redraws the bar beneath it.

For the logging module, wrap the run in logging_redirect_tqdm so handlers that write to the console go through tqdm.write. Handlers that write to files are left alone.

Passing file=sys.stdout matters when prints go to stdout and tqdm defaults to stderr: two streams can be flushed at different times and interleave. Putting both on one stream keeps ordering predictable. Setting PYTHONUNBUFFERED=1 (or python -u) has a similar effect by flushing stdout immediately.

import logging, sys
from tqdm import tqdm
from tqdm.contrib.logging import logging_redirect_tqdm

logging.basicConfig(level=logging.INFO)
log = logging.getLogger(__name__)

with logging_redirect_tqdm():
    for i in tqdm(range(100), file=sys.stdout, mininterval=0.2):
        if i % 25 == 0:
            tqdm.write(f'checkpoint {i}')
            log.info('logged safely at %d', i)

Why does it only break in the VSCode terminal?

Two VSCode-specific causes dominate. First, a narrow terminal panel: if the bar is wider than the column count, the line wraps and \r only returns to the start of the wrapped row, leaving a trail of lines. Use dynamic_ncols=True so tqdm re-reads the width each refresh, or set ncols=80 and widen the panel.

Second, launching under the Debug Console. The Debug Console is an output pane, not a terminal, so it does not interpret \r and isatty() is False. In a launch.json configuration, set "console": "integratedTerminal" instead of the default internalConsole for debugging, or start the script with Run Python File in Terminal.

Check also that the terminal is not running through a wrapper that captures output, such as piping to tee. Settings like python.terminal.executeInFileDir only change the working directory and do not affect redraws.

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: optimize",
      "type": "debugpy",
      "request": "launch",
      "program": "${file}",
      "console": "integratedTerminal",
      "env": { "PYTHONUNBUFFERED": "1" }
    }
  ]
}

How do I use tqdm with multiprocessing pools?

Keep the bar in the parent process and update it as results arrive. Workers should never create bars or print. imap_unordered yields results as they finish, which gives smooth progress even when task durations vary; plain imap yields in submission order and can stall behind a slow first task.

process_map from tqdm.contrib.concurrent does the same in one call using concurrent.futures. Use chunksize greater than 1 for many tiny tasks to cut inter-process overhead, but note that larger chunks make progress move in coarser steps.

If workers must report progress themselves, share a multiprocessing.Value or Queue and have the parent poll it. Multiple simultaneous bars need distinct position= values, and they behave best in a real TTY.

from multiprocessing import Pool
from tqdm import tqdm
from tqdm.contrib.concurrent import process_map

def run_one(params):
    fast, slow = params
    return fast, slow, fast * slow  # stand-in for a backtest

if __name__ == '__main__':
    grid = [(f, s) for f in range(5, 15) for s in range(20, 40)]

    with Pool(4) as pool:
        results = list(tqdm(pool.imap_unordered(run_one, grid, chunksize=4),
                            total=len(grid), dynamic_ncols=True))

    # one-liner alternative
    results2 = process_map(run_one, grid, max_workers=4, chunksize=4)