Skip to content

fix(queue): stop the overflow warning reporting a count that is always 1 - #3

Open
arthi-arumugam-git wants to merge 2 commits into
armature-tech:mainfrom
arthi-arumugam-git:fix/overflow-warning-reports-a-count-that-is-always-one
Open

arthi-arumugam-git wants to merge 2 commits into
armature-tech:mainfrom
arthi-arumugam-git:fix/overflow-warning-reports-a-count-that-is-always-one

Conversation

@arthi-arumugam-git

@arthi-arumugam-git arthi-arumugam-git commented Aug 4, 2026 •

Copy link
Copy Markdown

The privacy queue's overflow warning fires once, on the first drop, and interpolates the running total at that moment:

self._dropped += 1
if not self._warned_dropped:
    self._warned_dropped = True
    self._warn(f"privacy queue overflow; dropped {self._dropped} oldest candidate(s)")

At that point self._dropped is always exactly 1. So an operator who loses fifty thousand candidates is told dropped 1 oldest candidate(s). The counter keeps incrementing and is never surfaced again.

For a telemetry SDK this is the one number you cannot afford to get wrong, because nothing downstream can tell that anything is missing. A wrong count reads exactly like a correct one.

Your Go SDK already words this correctly

q.warn("privacy queue overflow; dropped oldest candidate (further drops are counted but not warned)")

That names the situation without claiming a figure it cannot know yet. This PR adopts that wording verbatim, so the SDKs agree rather than three saying one thing and one saying another.

src/queue.ts and src/Delivery/PrivacyQueue.php carry the same interpolation. I have not touched them here. Happy to send them separately, or fold all three into this one, whichever you prefer to review.

Test

Adds tests/test_queue.py, which is the first queue test in this SDK. tests/queue.test.ts and queue_test.go exist in the TypeScript and Go SDKs; Python and PHP had none, which is the reason this survived.

The test enqueues fifty past capacity and asserts the warning fires once and does not claim dropped 1. On main:

assert 'dropped 1 ' not in '[mcp-analytics] privacy queue overflow; dropped 1 oldest candidate(s)'
FAILED tests/test_queue.py::test_overflow_warning_does_not_report_a_count_that_is_always_one

With the change, it passes.

One thing to flag about CI

This workflow skips pull requests from forks:

if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository

so nothing will run here automatically and the green tick you would normally look for will not appear. Locally, PYTHONPATH=src python -m pytest tests/test_queue.py passes with the change and fails without it. Worth approving the workflow if you would rather see it run in your own pipeline than take my word for it.

An open question rather than a change

The warning is one-shot by design, which is right for noise. But that leaves the real total unreachable: _dropped is counted and never read. Would you want it surfaced on flush(), or exposed as a property, so a long-running server can find out how much it actually lost? Happy to add that if it is useful, but it is a behaviour change rather than a fix so I have left it out.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

The warning fires once, on the first drop, and interpolates the running total at that
moment. The total is always exactly 1 there, so an operator who loses fifty thousand
candidates is told "dropped 1 oldest candidate(s)". The counter keeps incrementing and is
never surfaced again.

For a telemetry SDK the number of silently dropped events is the one figure you cannot
afford to get wrong, because nothing downstream can tell that anything is missing.

The Go SDK already words this correctly: "dropped oldest candidate (further drops are
counted but not warned)". That message names the situation without claiming a number it
cannot know yet. This adopts it verbatim, so the four SDKs agree rather than three saying
one thing and one saying another. The TypeScript and PHP queues carry the same
interpolation and are not touched here; happy to send those separately if you would like
them in one place or three.

Adds tests/test_queue.py, the first queue test in this SDK. TypeScript has
tests/queue.test.ts and Go has queue_test.go; Python and PHP had none, which is the reason
this survived. The test enqueues fifty past capacity and fails on main with
"dropped 1 oldest candidate(s)".

Note on CI: this workflow skips pull requests from forks
(github.event.pull_request.head.repo.full_name == github.repository), so nothing will run
here automatically. Locally, PYTHONPATH=src python -m pytest tests/test_queue.py passes
with the change and fails without it.
@arthi-arumugam-git

Copy link
Copy Markdown
Author

A note while this waits.

No CI has run on this branch, which looks like the fork approval gate rather than a problem with the change.

The behaviour: the queue overflow warning reports a count that is always the capacity rather than the number actually dropped, so the log line says the same number every time no matter how much was lost. It reads like a constant and tells you nothing about severity, which is the opposite of what an overflow warning is for.

Happy to adjust the wording of the log line if you would rather it said something different.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant