My plan for the Ghostsix issue was superceded by the whole issue being resolved before my suggestion could be tried (if anyone wanted to) -- Ghostsix simultaneously quit and was kicked out; that's all over with now.
It looks like any dissension that arose between other people is being reconciled now; everybody's apologizing to everybody else and making friends again, so let's get back to the topic!
--------------------------------------------
I believe the principles set forth in Cliff's post above are absolutely vital for anyone who hopes to understand the scientific method to understand, and unfortunately they are far too little understood. Everyone who wants to do meaningful knife tests has to understand those principles, and so does everyone who wants to interpret the tests other people do and understand what the results mean. I'm going to flog the heck out of this horse because I really believe this is vital!
The vital principle is a scientist does not suppress data, no matter how clear he thinks it is that data doesn't mean anything. Why not? Because it's not for him to judge, all by himself with no peer review. A scientist is not afraid to publish data he believes to be invalid and explain why it's invalid and trust other scientists to understand that, or prove he's wrong about it. When everything is out in the open and everybody has a chance to assess the results of experiments and discuss what they mean, the truth emerges -- the scientific community is the free marketplace of ideas. (Ideally, anyway ... of course in the real world scientists are human beings etc. etc., but that's the way it's supposed to be.)
There are several reasons people are tempted to suppress data, but they're all bad. One is when you think you've made a mistake you naturally are not eager to tell the world about your screw-up. But, you may not have made such a screw-up as you think you have, and even if you have, somebody may find a use for that data that you never dreamed of -- somebody might later look at the information you found with your screwed-up experiment and discover something entirely different from what you were trying to discover -- and it might be much more important.
Remember how radioactivity was discovered?
Another reason to be tempted: You know durn well a certain person is going to find fault with everything you publish, and this data looks like a great opening for him to climb all over you, and you think the data really means nothing and isn't really relevant to the controversy.
That's okay. Post the data anyway, all of it. It might look bad for your position in the controversy and he might use it against you in the inevitable argument you know is coming as soon as you post, but if your position is correct the truth will emerge. (And btw, if you ever get caught suppressing controversial evidence....)
Or you can be tempted to bend over backwards, to suppress data that seems to favor your position but you don't think it's as meaningful as it looks at first glance and you want to be more than fair to the other side of the controversy. Don't do that either. For one thing, being more than fair to one side means being less than fair to the other side.... Regardless, you have no right to decide for other people which of the information they'll be allowed to know, not for any reason.
The extreme form of suppressing data is a popular way to do research for advertising purposes. For example, you want to prove your toothpaste is better than your competitor's -- in fact they're exactly the same. No problem! You just hire ten "independant testing labs" each to do a test with ten kids, five on your toothpaste and five on a competitor's, and make them all sign contracts swearing to report only to you and keep all their results secret. Because ten kids is not a statistically significant sample, some of those tests will show the kids on your toothpaste getting fewer cavities, some the opposite. Pick the test with the best results and publish it, file the others in the round file -- presto, you have proven your toothpaste is better!
Something like that
could be happening in the current controversy. Mad Dog dealers could be testing all the knives they have available and posting only the tests of the ones that turn out to be good. A gang of Mad Dog haters could be secretly getting their hands on knives and testing them and posting only the tests of the knives that turn out to have brittle edges.
People could be testing knives in secret and calling in impartial witnesses and doing another test only if that knife is defective (or not, whichever they want to prove) -- so the presence of impartial witnesses at tests is no safeguard against that.
Of course that could only skew the percentages; that kind of deception could only work if some MDKs are defective and some aren't; it couldn't accomplish anything if they all had brittle edges or if none of them did.
Please understand I am not suggesting for a moment that any such suppression of evidence is going on -- I don't believe it is, not for a moment. I'm only using it as an extreme example of what can happen when you start suppressing evidence.
Mike has talked about deleting the test of the first ATAK from his published results and replacing it with whatever results he finds testing Walt's. We all know what his motivation for that would be; he'd be trying to be fair -- so what? If a Mad Dog dealer were to suppress that evidence to help sell MDKs we would all be up in arms about it. If Mike does it for entirely different reasons, what difference would his motivation make? He'd still be suppressing evidence. The effect on the quest for truth would be exactly the same as if a Mad Dog dealer had suppressed that evidence.
I believe not one of the Mad Dog dealers would suppress that evidence, and Mike shouldn't do it either. The motivation for suppressing evidence makes no difference -- just like if you steal to give to charities, it's still stealing. To a scientist, suppressing data is a far worse crime than merely stealing money ... suppressing data hurts everybody in the world; it hurts the quest for truth -- mere money is nothing compared to that.
The whole question is moot in this instance, of course -- the data has been posted, people have downloaded and saved it, Mike can't suppress it now if he wants to. If he deletes it from his website somebody else will post it on the web somewhere; it's a moot question; I'm only arguing so much about it for the principle. We all need to understand the rules of science and why science has those rules if we're going to learn anything from knife tests.
-Cougar Allen :{)
P.S. I just realized I wrote an entire post about this controversy without saying a word about flying saucers or mind-control beams or
anything interesting at all! Sorry about that ... I'll try to make up for it with a later post....
-Wholly Brother Cougar, Speaker to the Cistern
