Isn't poll voting odd?

Help Support QuadraphonicQuad:

JonUrban

Forum Curmudgeon
Staff member
Admin
Moderator
Since 2002/2003
Joined
Mar 2, 2002
Messages
16,871
Location
Connecticut
You guys can find and read post after post of me describing what I wanted to do with the polls when they first started. The formula was basic and based in a user review as well as a surround review, like what you'd see in the old 'Stereo Review" magazines and Rolling Stone. It's a GUIDE. It is not gospel.

You will also read that I pleaded for voters to post the freaking comments on why they were voting as they did. People get all fired up because of the 30/30/30 thing. I used the remaining 10 percent as a catch-all. It doesn't have to mean that the package or graphics are great, it could mean that it's just awesome and WTF, here's an extra 10%.

Remember, this was just a small section of this forum that was to collect info on new releases so readers could find them all in one place. It grew way past that and evolved into something people argue about and get all worked up, claiming "I will never vote in a poll thread", etc. Who cares? It's a review.

When it was all me doing it on my own, I really got pissed when members would argue over the polls. It wasn't supposed to be something that got members worked up. It's just turned into that.

At this point, unless we started all over again, changing it is virtually impossible. And really, as time goes by, it's the only way we still hear from members who have left us, in body and in spirit. I enjoy reading an old poll file and seeing posts by people like Cai Campbell and Ed Bishop (EMB). Their thoughts live on through the poll files, and moving forward, the words of other members will live on past their actual life form.

Really, it's not worth getting all worked up about.
 

Sonik Wiz

2K Club - QQ Super Nova
QQ Supporter
Joined
May 30, 2005
Messages
2,518
Location
Kansas City
Remember, this was just a small section of this forum that was to collect info on new releases so readers could find them all in one place.
Well, it always stuck me as something Brian Moura (Quadlings) would have done in ancient times but with reader feedback.

Great great info decades old it has helped me to wisely purchase so many times. Or not. Rock on.
 

ssully

2K Club - QQ Super Nova
Since 2002/2003
Joined
Jul 2, 2003
Messages
3,053
Location
in your face
I agree that the poll voting system is a bit flawed, but by the same token it's also the largest assemblage of qualitative judgement about surround music available anywhere, and there's a lot of value in that. The programming team at facebook has a motto that goes something like "done is better than perfect" and I really think that applies here - yes, maybe you won't get a number grade for every release that perfectly reflects the title's quality on an absolute scale, but if you're thinking about buying something, chances are if you dip into the poll thread here you'll be able to get some kind of idea about whether it's worth pursuing or not.

I absolutely agree that posts in poll threads add just as much, if not more value than just simply voting.
I would say the threads related to each poll are the main value (though often overgrown with thickets of posts about purchasing and shipping issues -- which should be weeded out or banned). I'm happy to read (and add) subjective reviews and objective analysis on poll threads, but I rarely vote. The 1-10 scale is too ridiculous.

Switching to five-level scale would not be technically onerous. The archive of 1-10 poll data need not be a burden that prevents it from happening. The threads can stay, and the ranking data can be transformed. One could map them this way
1-2 = 1
3-4 = 2
5-6 = 3
7-8 = 4
9-10 = 5

The result of mapping existing data will still be a preposterous inflation of 4s and 5s, but going forward new release rankings might become more realistically and informatively distributed.

A discussion towards consensus of what 'average' means in terms of a surround release, could also be helpful to bring some sanity to rankings.
 
Last edited:

par4ken

1K Club - QQ Shooting Star
QQ Supporter
Joined
May 28, 2011
Messages
1,284
Location
NW Ontario
I would say the threads related to each poll are the main value (though often overgrown with thickets of posts about purchasing and shipping issues -- which should be weeded out or banned). I'm happy to read (and add) subjective reviews and objective analysis on poll threads, but I rarely vote. The 1-10 scale is too ridiculous.

Switching to five-level scale would not be technically onerous. The archive of 1-10 poll data need not be a burden that prevents it from happening. The threads can stay, and the ranking data can be transformed. One could map them this way
1-2 = 1
3-4 = 2
5-6 = 3
7-8 = 4
9-10 = 5

The result of mapping existing data will still be a preposterous inflation of 4s and 5s, but going forward new release rankings might become more realistically and informatively distributed.

A discussion towards consensus of what 'average' means in terms of a surround release, could also be helpful to bring some sanity to rankings.
Sorry but I don't see how going from a 10 star rating to a five star one is any kind of improvement. Ratings in the end really are meaningless the comments however are invaluable. Perhaps the ratings should of been simply for surround effect (disregarding all else) or had been broken into three categories for Sound Quality, Surround and Content. But as Jon points out it's too late to change that now.

Buying music solely based on a pole number is like buying equipment solely based on specs!
 

ssully

2K Club - QQ Super Nova
Since 2002/2003
Joined
Jul 2, 2003
Messages
3,053
Location
in your face
Gee, that's funny, since I buy equipment based on 'specs' -- and better yet actual measurements -- all the time. Without having heard it. For specs -- ie what the manufacturer claims --you have to know what they do and do not say. For much of audio, the hardware is a commodity item. I can buy based on features I want.

What I *do not* do is buy audio gear based on random users' (or audio magazines') sighted listening comments or rankings...they are not reliable and are often idiotic. So your analogy breaks down in both directions.

But if one is going to have a listener ranking and comment system: four and five star review systems are commonplace for restaurants , food, drink, movies, and....music. Nobody does '10 star' rankings. There is a reason for that. It is much easier to consistently decide between 4 or 5 levels, than 10, and the results make more intuitive sense. A 10 star system helps makes the rankings 'meaningless'. A narrower ranking range can foster a more meaningful outcome.

As for the difficulty I already showed how it can be done; It's called recoding. I could recode the existing table ranking data in minutes. The 'hardest' part would be making it into a web page. Converting the blank poll form from 10 to 5 is also trivial.
 

ar surround

2K Club - QQ Super Nova
QQ Supporter
Joined
Apr 3, 2010
Messages
2,627
Location
New Joisey
I’d say that the polls skew high because most of the stuff is actually quite good. Take Sgt Pepper for example. The surround mix may be less than what we had hoped for, but I’m not going back to the old stereo mix or even a MC upmix of the new stereo mix.
 

artwwweb

Senior Member
Joined
Dec 23, 2010
Messages
249
I started writing this in a poll thread when discussing a specific score, but then thought I shouldn't drift off-topic, so I looked for a more relevant thread...

I don't find the poll votes very useful because everybody votes high. However, I find the comments extremely useful. So I tend to comment and not worry too much about the votes.

It's understandable that people vote high:
1) Most people do only buy stuff they think they'll like, so of course the scores are going to be above average.
2) It helps people justify their taste/ financial outlay (and this is not a criticism - it's just what we all do to some extent).
3) New releases especially will be biased higher due to novelty - this is a known problem with all kinds of votes (e.g. films - I remember hearing years ago that in some 'best film ever' poll you weren't allowed to vote on a film less than 10 years old.)
4) Peer pressure - if most people vote high, we want to and/or feel we have to.

I think 1) is valid, but we ought to try to avoid the other reasons. It would be nice if somehow we could restart the voting (I know that would be unrealistic) so that a 6 or 7 are not embarrassing and even a 5 is reasonable effort. It's not just that it would separate the very good (7) from the truly excellent (10), it would also separate the 'not very good' (3) from the 'truly awful' (1)

Having browsed these and other comments, I think most people are roughly in agreement: The mix is the most important, then the sound, then the music. (NB: Of course the music is the most important on a personal basis, but on a surround forum, it's the mix that is the highlight. And crucially, one can usually audition the music on YouTube, etc. so it is less important to know others' opinions on the music when making a decision about buying.)

We are all going to have our own tastes and ways of assessing things, but maybe the most important thing for me is - it would be nice if more people made comments in the polls.
I hope my post, above, wasn't interpreted as a complaint about the work people put in, or even the voting system. If that was the case, I am sorry. I am really grateful for all the work that is done. I have found many great albums through the QQ polls.

I was saying that polls are skewed because of the voters (myself included), not the system. I wasn't even complaining about the voters, just musing on why it happens. By considering the possible reasons that we might bias our vote it would be possible to reduce some of that skewing, thereby giving a wider range of poll numbers and making the vote stats more revealing.

But I still think for me the most important thing is comments. Another point that has occurred to me is that people should not be afraid of making negative comments/ votes. I find QQ is really friendly 99.9% of the time so this is not too much of a concern, but people sometimes get very protective about an album they love so that might be putting voters/ commenters off.
 
Top