This is not a bug, although IMHO it is poor UI as implemented.
There are two reasons why your personal review queue and the indicator may never match. One is that the indicator is cached, since review activity is ongoing, especially on the larger stacks. The other was explained by SE staff in the m.Physics.SE thread The change to the taskbar is massively annoying last year:
[T]his number counts everything - including tasks you aren't personally eligible to review. This includes tasks you've already done, tasks you've skipped, and tasks that the system expects you might not be sufficiently impartial to review (e.g., things you've already flagged or posted). [Shog9]
In other words, it's supposed to be a status indicator, not a to-do list; it's just unfortunate that people like me think it looks like a "new messages" indicator as opposed to a thermometer. If you can think of a better design, you can always propose it at meta.stackexchange.com (MSE).
The review queue indicator began as a feature request two years ago, and was adopted to increase the number of participants in the review queues. Not long thereafter came the MSE feature request, Please dim the color of the review task counter when there are no actionable tasks, but the indicator has been argued to be too resource-intensive to personalize.