Here is a thing that surprised me: although nominating for the Hugo Awards is open to anybody who plunks down the money for it, only about a thousand people do.
A thousand people is a lot, but we hear about the Hugos as being the award from fandom at large - and I don't know how many people that is, but I know it's much larger than a thousand. So why is the voting population such a relatively low number?
I've been thinking about my own reasons for not voting till recently, and realized that - although the $50 for a supporting membership would have been out of my reach for a long time, even had I known then that this was all it took - the primary reason I didn't vote was that I wasn't reading books in the year they came out, so I didn't have enough data to form an opinion worth voting on.
It's actually a pain in the ass to read brand-new books, if you're a more rabid reader than your wallet can reasonably support. Hardcovers are expensive, and if you merely wait a year for the paperback, you can buy two or three books for the same price. New books don't always show up immediately in the library, and have shorter loan periods and longer wait lists than last year's works. New books are rarely to be found in used bookshops. It's just not efficient to be addicted to new releases; you get a much bigger bang for your buck by waiting.
So I think Hugo voters - well, nominators, anyway - aren't really fandom-at-large; they're fandom's-early-adopters, the folks who expend the energy/money to get the crack as soon as it's on the street. Not everybody is this kind of crazy, so it would explain the smaller numbers.