I was watching House the other day. Thirteen was beginning her clinical trial with a new medication, and Cartman discovers that she is taking the Placebo and not the real medicine.
One day I will do a comparation between the methods of CSI and House, but for now, let´s focus on this example
The Experimental Method
One of the great combat horses of the empirical science is the experimental method. It is a very simple thing, but with very good results: you focus on one or two variables, and you keep changing this variables (otherwise they would be names constants). E.g. you have a carbon resistor and you let a electric current go through it. You measure the ohms of the resistor. Then you go and, by instance, change the temperature a little bit, and measure again, and change it again, and again, etc, etc.
It is a very simple method and it is very elegant. In physics. Unfortunately it hardly works in anything else.
Seriously. I have a degree in chemistry. We love science. We love experimentation. We love the “experimental method”. But it really don´t work. Chemical combinations that says in the book that should work, often don´t. Even in chemistry, which is one of the “hardest” sciences, there is a huge degree of art and luck. No two chemical reactions are completely equal, although they often are very similar. In physics an experiment can always be repeated by other physics with the same amount of money. In chemistry we need the money and somebody who have enough experience to make the damn things to work as they should !
If this is how things work in chemistry, is not a huge mystery why, when people tried to use the “experimental method” to everything in life, results were less than impressive. Try to do experimentation in administration, sociology, economy, medicine, and why not, astrology, often shows a lack of knowledge of the limitations of the method, as well as the characteristics of the science in which the experimental method is being applied to. Usually, our “brilliant scientist” is trying to use methods from other sciences, like physics, in another field of knowledge, like education, without minimum regard for the particular conditions, theories and “state of the art” of this discipline.
This “importation” usually is done by a matter of status. It may be done by administrators, educators, astrologers, etc, who want to be seen as “real serious scientists”. So, they think that using the same method as physics, they will one day also get the Nobel prize. As I have a masters in business administration and another in education, well, I can say that I have seen it all !
Clinical trials and control groups
When using the experimental method in medicine, or other sciences, we can´t do the same as we did in our example with the resistor. We don´t have the power to really shift the variables back and forth. What we do instead is to use several subjects and apply the same condition to subjects with different, but known, traits, or to apply different conditions to subjects with the same known trait.
Clinical trials are usually of the second kind. We use a group of people with a known trait (usually the trait is having a disease) and we give them different conditions, by instance, placebo for one group and medicine for another. The placebo effect is one of those things that is so repeated that we think that we “understood” placebo… but the simple truth that the mind can heal if tricked to do so, was never fully investigated.
In the case of Thirteen and Cartman, as it is a work of fiction, there are obviously a lot of mistakes. In the episode, the nurse tells Cartman that Thirteen is in the Placebo group because her medicine has no smell. Well, to a study to be “double-blinded” means that no one knows what each pacient is taking ! You can´t have the nurses knowing what pacients are in the placebo group because their attitudes toward the patients can hugely affect the placebo effect ! Not only that, but there will be effects on the patients as well. I remember that the first tests with anti-HIV medication were greatly criticized: it was discovered that desperate pacients would exchange medication between each other, in the hope of being given at least 50% of the true medicine !
In reality is unethical to give placebo to patients with life threatening diseases, and I don´t think this kind of study is done anymore. What is done instead is to give to one group the new drug, and to the control group (not placebo group) a regular drug, used as the standard for the treatment of the disease. This makes it hard to prove the “pure effect” of the drug, but in the long run we are only interested in proving that the new drug is better than the standard treatment. No need to give water to patients to prove that !
Control groups and astrology
Astrologers who think that “one day astrology will be recognized as a science” usually do statistical research looking for “signatures”. But you see few to no researches using control groups.
The reason is obvious. Astrologers are usually so worried about finding the yes, that will hardly bother to look for the “no” as well.
I will first focus on case studies with a control group, then will say a little word about control groups in statistical research.
A good example would be http://www.astrology-reading.com.au/USA_Presidents_who_died_in_office.html. He predicted a few months ago, the somewhat unlikely prediction that Bush would be killed in october (who would care to kill Bush in october of his last term is beyond me).
Note that I am not claiming that his astrology is “bad” or in any way analyzing his astrological reasoning. What I ask you to see is his method, in which he looks for:
- transits of the outer planets to the angles of the chart of the US (the one he uses) for the times the US presidents died in office
- uses the solar return to the chart of the US for the times the US presidents died in office
- aparently uses other factors, like eclipses, also for the times the US presidents died in office
Well, you probably already know to where I am going… we see no indication that he ever bothered to create a control group ! There are 8 presidents who died in office, but almost 40 who didn´t!
Our goal is not to see what configurations were present in the death of the presidents, but to see which configurations were present when they died which weren´t present in the control group! Without this knowledge, we could as well use the moon in taurus as an important factor to know when a president will die !
I remember a “study” about suicide, in which the “researcher” concluded that uranus in scorpio was a significant factor to suicide. As his sample was composed by people in their middle twenties to thirties, I was not surprised by this brilliant conclusion ! If he bothered to have used a control group, he would have seen that several people who were born during the 7 years that Uranus stood in scorpio didn´t bother to kill themselves.
Statistical research and control groups.
There are two cases in statistical research.
One of them is the comprovation of hypothesis. You have a clear and concrete hypothesis that you want to see if it is rejected or not (a hypothesis can never be proved!). E.g. “do people who commit suicide have more probability of having an angular mars in their natal charts?”).
You take a very large group of people and then compare it to a control group. Then you say yes or no. This is serious research, and most often than not, very frustrating.
The second type of research is the “go fishing” approach. You thrown the line and wait to something to bite. Anything at all. No a priori hypothesis of any kind. This is messy and usually shows very poor research.
As the researcher will accept absolutely ANYTHING that has a significant difference between the test and control group, then he is bound to find something ! He will accept a mars angular, but also a pluto angular, a quiron sextile fortuna, ceres biquintile Cupid, absolutely anything can be found.
In this kind of research the researcher will always be happy because he made “several important discoveries”. Astrologers who use this method, don´t feel bad about yourselves. This is quickly becoming the standard in social sciences research for researchers who don´t care about their research but wants to publish something with not so much of an effort!
If you are using a “go fishing” approach, I would suggest a 20-80 approach. You take both your test group and your control group and divide them into a two parts. One part have the two 80% splits. You use them to find your “go fishing” rules. You can´t use any data of the 20% split.
AFTER you have generated a rule, you then go and try to use this rule in the 20% split (remember that the 20% split is formed by 20% of the test group and 20% of the control group). Then you can test your rule in this fresh sample, because it wasn´t used to create your rule.
But be prepared to cry when all your beautiful correlations suddenly starts to colapse and vanish !
Astrology, unlike medicine or sociology, has no presumption of causality. So, if relationship is such a difficult thing to prove through experimentation, even more so in astrology.
No matter how many events you have in your sample, you can never be totally sure that the event will repeat itself !
I have a coming article about the death of US presidents based on the inauguration charts. I can´t claim that it can work for foresight untill I have tested this method for, I don´t know, possibly centuries, as we don´t know when another death will happen.
So it is always sound advice to tell your readers that only because your research showed this and this, it doesn´t mean that will happen in the future ever again.
Or, in the case of the author of the article quoted, he removed the article in which he predicted Bush´s death. For some coincidence, it seems to just have vanished. Luckily, I was kind of expecting that, so I saved his article if anyone is interested.
Some years waaaay back, I wrongly predicted that Queen Elizabeth would die. Well, Queen is strong and alive. But my prediction is still open in the same forum for the world to see. There is no shame in getting it wrong, if you learn from your mistakes.