In the past few weeks I've ocassionally seen questions on the "close" queue that don't have any actual close votes. Has anybody else seen this? Is it a bug?
Yes, I've noticed that too. I think it's because we are getting more users being active on the site. See this question on the main SE meta site to answer why it's happening.
Any time a user without the vote-to-close privilege flags the question for a canned vote-to-close reason, it is automatically added to the close votes review queue as a way of reviewing the flag. The reason they used for the flag is the one presented at the top. I can only imagine that it being removed from the queue by too many "Leave Open" votes would automatically dismiss the flag as well, but I cannot confirm that. The flag is dismissed if the question gets closed, though.
You'll also notice that in cases where someone flagged as an exact duplicate, their flag is included in the count when selecting which question is the duplicate.