: Here's the part I found frightening:
: << An International Skating Union presentation at the U.S. championships
: revealed yesterday some new details apparently adopted by the six ISU committee
: members who are tweaking the system.
: Everyone had already heard that 14 judges would be lined up to mark events
: under the new system, but that a random system would select the marks of only
: But with a trimmed mean, the two highest and two lowest marks would be tossed
: out. That means that out of 14 original judges, only five marks would end up
: counting. >>
: From a statistical perspective, this seems questionable. Anyone with a math
: background wish to explain why (or how) this process would work?
- According to the stats text in
my office (and this is just off the top of my head, people with better
statistical backgrounds feel free to correct me), in a population of 14, a
sample of 5 gives you at least a .0004 level of confidence that your
sample is representative - in other words, for every 1000 times you draw a
sample of 5 from that population of 14, 996 of those 1000 draws will truly
represent the opinions of the entire 14. This is statistically significant
(i.e. reliable) and would be considered meaningful in most situations.
HOWEVER this os only so if there is a normal distribution among the
entire sample, i.e. that a certain percentage of the population falls
certain distance of the mean (the % vary slightly depending on the size
of the population, but the general "shape" of the distribution is the
same, like a snake that swallowed a basketball).
It's impossible to know whether
this is the case or not without seeing the entire set of 14 marks.
PLus...would you want to lose, say, Worlds if you happened to be the one
who got one of those 4 random draws that DIDN'T represent the opinions of
the entire panel?
- Tossing out the high
marks makes minimal difference to the mean.
- There is no guarantee that the mean or the marks the mean is
derived from truly represent the
performance being judged. In other words, if you have a panel full of
Balkovs, there will be a mean, but it might have nothing to do with the
quality of the performance. All the mean says is "here is a number that is
the average of all the marks awarded" - there is no compensation for the
fact that those marks might be flawed in some way.