Skip to content

whisper_full_parallel: signed 32-bit overflow corrupts segment timestamps for long audio #4039

Description

@deftmousetail

Issue body

Summary

whisper_full_parallel() calculates parallel chunk offsets using 32-bit int multiplication before adding the result to an int64_t. For sufficiently long audio, the multiplication overflows and corrupts both the reported split positions and the segment-level timestamps merged from later processor states.

This is distinct from the token-level timestamp reset discussed in #2036 and #3726: the ordinary segment t0/t1 values and JSON transcription[].offsets become negative or wrap back near zero.

Reproduction

Create a long test file by repeating the repository's existing samples/jfk.wav fixture:

ffmpeg -stream_loop -1 -i samples/jfk.wav -t 3630 -ar 16000 -ac 1 long-jfk.wav

Run whisper-cli with more than one processor and JSON output:

./build/bin/whisper-cli \
  -m models/ggml-base.en.bin \
  -f long-jfk.wav \
  --processors 4 \
  --output-json \
  --output-file long-jfk-p4

The same problem can be exercised with --processors 2. --processors 1 does not use whisper_full_parallel() and produces valid monotonically increasing segment timestamps.

Observed result

In a real 3630.8-second input, the four-processor run printed:

whisper_full_parallel: the audio has been split into 4 chunks at the following times:
whisper_full_parallel: split 1 - 00:15:07.690
whisper_full_parallel: split 2 - 00:-14:-28.-960
whisper_full_parallel: split 3 - 00:00:38.730

The generated segment JSON was also corrupt, for example:

{
  "offsets": {
    "from": 1815370,
    "to": -859960
  }
}

Subsequent segment offsets remained negative until the wrapped timeline crossed zero. With two processors, the only split was likewise reported as 00:-14:-28.-960 and segment-level offsets were corrupted.

Expected result

The split positions for a roughly 60-minute input should be near 15, 30, and 45 minutes, and all merged segment timestamps should remain non-negative and monotonic through the end of the input.

Cause

On current master, n_samples_per_processor and i are int. These expressions overflow before the result is added to the int64_t offset_t:

result.t0 += 100 * ((i + 1) * n_samples_per_processor) / WHISPER_SAMPLE_RATE + offset_t;
result.t1 += 100 * ((i + 1) * n_samples_per_processor) / WHISPER_SAMPLE_RATE + offset_t;

The split-reporting expression has the same issue:

to_timestamp(100 * ((i + 1) * n_samples_per_processor) / WHISPER_SAMPLE_RATE + offset_t)

At 16 kHz, 100 * chunk_start_samples exceeds INT_MAX once a chunk boundary is later than approximately 22 minutes 22 seconds. Adding offset_t afterward does not prevent the earlier overflow.

Relevant current source:

  • src/whisper.cpp, result merge around lines 7280-7288
  • src/whisper.cpp, split reporting around lines 7321-7325

Suggested fix

Promote the multiplication before it occurs and calculate the offset once:

const int64_t chunk_offset_t =
    100LL * (i + 1) * n_samples_per_processor / WHISPER_SAMPLE_RATE + offset_t;

result.t0 += chunk_offset_t;
result.t1 += chunk_offset_t;

Use the same int64_t calculation for the displayed split positions. A regression test with a synthetic sample count corresponding to more than 45 minutes should be sufficient to exercise the overflow without running inference on an hour of audio.

Environment where observed

  • Windows 11 x64
  • whisper-cli from Lemonade SDK whisper.cpp-rocm v1.8.4
  • Vulkan backend on AMD Radeon RX 9070 XT
  • ggml-base.bin
  • 3630.8-second, mono 16 kHz decoded input
  • Reproduced with --processors 4 and --processors 2
  • Not reproduced with --processors 1

Although the observation used a Vulkan build, the overflowing arithmetic is in backend-independent whisper_full_parallel() code and remains present in upstream master.

Related issues

WHISPERCPP-BUG-EVIDENCE.txt
WHISPERCPP-BUG-REPORT.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions