MDP,
I'm impressed. I would have thought most people's eyes would have glazed over a long time ago (I'm sure there are many who have). Anyway...into the void...
You only need a sample size of one when you are dealing with a probability of 100% for a specific event, ie. zero variance (variance determines sample size needed for your confidence level). You can use sample sizes of 2 without this, the t values for the 95% confidence intervals are in the book I referenced above.
Yes I checked. The d.f. goes down to 1. But did you look at the critical value? From 3 to 1 degrees of freedom, the critical value rises exponentially. Again, I have never seen a t test published with 1 degree of freedom. And with n=1 and degrees of freedom = n-1, you get 0 degrees of freedom and hence no critical value and hence no t test.
People make judgements like I have all the time in scientific work, you don't have to calculate the exact probabilities or even make any attempt to measure them if you can estimate that they are *much* lower than your criteria. You do these estimates based on your experience which you must bring into the work.
I partially agree with you. But conclusions base on data without statistical testing are usually couched in much more speculative terms. With statistical analysis, we can state mathematically what our confidence levels are. Without statistical analysis, we can speculate.
For my current field of work - Can. J. Phys., JQSRT, J. Mol. Spect, Phys. Rev., Phys. Rev. Lett., Physica., etc. . I expect this to change in a year or two as there are some other fields I would like to explore and other people I would enjoy working with.
In these papers are you the primary author? I'd be interested in reading some. If you would email me a reference or two that you think might be applicable this this discussion in terms of analysis, I would be interested in reading them. I spent three summers working at the Michigan State Cyclotron when I was a grad student where I was involved in building low pressure multiwire proportional counters for the 4 Pi group there. So I have some familiarity with particle detectors and a little physics. It's actually kinda funny because I remember having a chat about statistics with the head of the lab at that time and I was somewhat stunned when he asked me what a t test was. Evidently, particle physicists use mainly the Poisson distribution and one other test, I think the Chi Square, but can't remember.
There are cases where no specific statistic needs to be calculated. This does not mean that it is not statistical work but I think this is the defination you are using - and I am beginning to understand your perspective.
Again, I agree--somewhat. There are many scientific claims that can be made and supported by the data without statistical testing. And the history of science is littered with ideas that were once accepted but now rejected. We once thought phlogiston was produced when we burned a candle in a bell jar. Great idea and it seemed plausible because of its explanatory power. So many ideas are accepted simply because they have great predictive and explanatory power. But phlogiston died when we discovered oxygen.
But scientists still have to tread lightly here. String theory, for instance, has great explanatory power but where is the data?
During PhD examinations, the question I hear most frequently is: Why didn't you do the experiment?. I can understand why you didn't increase your sample size. You had confidence in your testing and your results. But recall that you extend this to all makers. Now all makers are branded as incompetent if one of their knives fail your test? For me, this goes beyond common sense and seems extreme. I just don't have a lot of confidence in this notion.
For me, "statistical work" is statistical testing. It's more than just presenting information and thinking speculatively about it. This is the foundation of modern science. Disproving hypotheses. In the past 50 years in the biological sciences (my main field), observational data have been marginalized and the emphasis is on experimentation and backed up by statistical testing (unfortunately, my opinion is that observaional data has been too marginalized but that's another story).
As the great philosopher of science Karl Popper would say, we don't prove things in science, we disprove things. That's why we accept things in terms of probability. We talk of things as being highly probable. And we used statistical analysis to back up our claims. Linus Pauling can speculate all he wants (or wanted) about vitamin c curing the common cold. All his experience and knowledge told him it was so. Some people "believed" him because he was a nobel laureate. But I'm one of those scientists (as are most of the scientists that I know personally and have read and studied in my field) that are pig headed and say: Show me the data and the statistical analysis. When scientists move into the handwaving arena, that's great because they can point the way to new ideas, but the data and grunt statistical analysis has to flow from there. Sure, we accept many ideas in science that have not been tested statistically. But we usually (and should) have less confidence in them. When we test our data statistically, we can state clearly and unambiguously what the confidence level is. That's the p value.
That should vary depending on what you want your Type I and Type II errors to be. Just because a lot of people ignore this and use 5% without consideration doesn't make it the best thing to do. Would you think that a 5% p value would be alright for a death sentance, obviously not. I read last year a very nice discussion in a medical journal where the author discussed this very issue
and debated at which level results should be evaluated at. I can't remember exactly at what level he was for, but I think it was 99+% based on the fact that branding noneffects being much less serious
than ignoring a harmful ones. I wish I have noted that article as it would have been useful as a refence when teaching introductory analysis as it clearly illustrates that many criteria are very
subjective.
Yes, this is a great issue. But I don't think we are talking about life and death issues unless you are talking about ruining some knifemaker's life [which could be the result of a type I error]. Then perhaps p < 0.01 or p < 0.001. Which would make your position far less tenable and guard against the type I error. I'd love to look up the critical value for p < 0.01 and degrees of freedom = 1. It surely must be astronomical.
Much of my research is small sample work and I usually work with n between 15 and 30 in an ANOVA design and p is often much less than 0.001 when results are significant (but I'm perfectly happy with p < 0.05 and in some instances, would argue that p < 0.1 should be conisdered. But I know my confidence level because I can calculate it. I could speculate about it and I might be right because I have experience in my field, but if I can do the required experiment with the proper n and don't and just rely on my "hunches" I would never get my work published and I would never get funded.
But I don't think publishing in a scientific journal and getting funded are at issue here. I've already stated that I find your results compelling. But I'm a bullheaded skeptic and I can't think of anything that I accept with the 100% confidence that you are espousing based on n=1 or n=2.
So I will again say that I think we will have to agree to disagree and let it go at that.
BTW, I grew up on a pig farm and I will attest that it is highly probable that all our pigs had 4 legs.
------------------
Hoodoo
The low, hoarse purr of the whirling stonethe light-pressd blade,
Diffusing, dropping, sideways-darting, in tiny showers of gold,
Sparkles from the wheel.
Walt Whitman
[This message has been edited by Hoodoo (edited 04-30-2000).]
