Physiological illiteracy impedes progress in medicine!

A recent paper in PNAS has caused a bit of a ruckus discussion. The authors (Tim W. Fawcett and Andrew D. Higginson from U-Bristol) have compared citation rates with the number of equations in the text of papers, and found that more equations per page is associated with a lower number of citations in non-theoretical papers, but that there was no effect in theoretical papers. This carries over to all citations as well since the number of non-theoretical papers is much larger than the number of theoretical papers. They conclude that a higher density of equations leads to lower numbers of citations, and that this indicates a worse dissemination of the results to the wider scientific community. They go as far as using the title: "Heavy use of equations impedes communication among biologists."

This engendered a barrage of more or less agitated rebuttals with titles such as: "Do not throw equations out with the theory bathwater," "Mathematical illiteracy impedes progress in biology," "A suggestion on improving mathematically heavy papers," and "No evidence that equations cause impeded communication among biologists." Anyway, an alternative view is that theoretical papers (with lots of equations) are mostly cited in other theoretical papers, although this sounds obvious and would never have been published in PNAS.

I would wager that the same is true in all fields: Physiologists certainly cite other physiologists more often than they cite clinical scientists (and almost never cite mathematical biologists). It is further easy to extrapolate to the conclusion that detailed physiological data in a paper is associated with fewer citations in clinical medical journals. The obvious interpretation is that overzealous use of physiology in articles impairs dissemination of medical science, and that physiologists should use more common words and talk more about patient-survival and quality-of-life in the same way that mathematical biologists should relegate their equations to the appendix. The idea that they might be comparing apples to oranges (clearly a biological concept) is not even discussed in the paper, which makes the discussion a bit lopsided. More as if they were writing an opinion-piece than an article on empirical science.

Advice for a Young Investigator

Santiago Ramón y Cajal was a 19th century neurophysiologist. I'm told he is well known, though he seems to have ignored the kidneys completely. Strange.

Any way, he wrote a book - "Reglas y consejos sobre investigacion cientifica" translated as "Advice for a Young Investigator" - to help budding scientists through their first years as researchers. While some of his advice is dated, such as the section on how to choose a wife, most is well worth the $14. What follows is a brief summary and some comments on the 1999 MIT Press pocket edition (with one of the worst quality bindings in modern history), translated by Swanson and Swanson (who have done a good job).

There is a Foreword by Larry W. Swanson with a short biography of Dr. Cajal and some suggested readings from his quite large production. Then there are three prefaces, for each of the second, third and fourth editions.

There are nine chapters, each addressing a different potential difficulty.

1. Introduction

The introduction concerns itself with which theory of science a scientist should build his understanding of nature upon. The answer is determinism. Cajal is very clear that a scientist should not spend his time delving too deeply into the various theories of knowledge and science. He writes:

"...by abandoning the ethereal realm of philosophical principles and abstract methods we can descend to the solid ground of experimental science..."

In this he comes quite close to my own thoughts on the matter (which may be why I like the book in the first place). Today, pure determinism is still usefully utilized in Chaos Theory, or the study of non-linear dynamics, but in general the focus today should be on statistical determinism. That is, the fact that we cannot know the full state of any system (at least any interesting system) induces variation and that this punches a quite large hole in classical determinism, but can be compensated for by studying larger populations and using statistics. Actually Cajal does not mention statistics at all, probably because it was just starting out as a field at the time.

2. Beginner's Traps

In the second chapter Cajal goes through some things that might unduly discourage the beginning scientist from his chosen path. Basically that well known feeling that those who went before were smarter, saw farther and thought deeper than you ever could. He makes the case that it only seems so after the fact, and that with hard work and time a scientist starting out today will appear equally daunting to those starting out in the future.

3. Intellectual Qualities

Cajal's main point about the mind of the investigator is that it does not have to be exceedingly brilliant as long as it is tenacious and a bit vain. He does mention originality, but his focus really is on hard work and the striving for glory. These are qualities he dejectedly notes are none too present in his contemporary Spaniards, while he holds the Germans in high regard. He writes:

"In Spain, where laziness is a religion rather than a vice, there is little appreciation for how the ... work of German [scientists] is accomplished - espeially when it would appear that the time required ... might involve decades!"

Rather poignantly, the endnotes have been expanded with the editions. In relation to a section written in the original 1893 edition Cajal later notes:

"This frank optimism is now greatly undermined by the hideous international war that began in 1914... It is sad to admit, but all nations become ferociously imperialistic as soon as possible... So much for the weak and unpatriotic!"

4. What Newcomers to Biological Research Should Know

In this chapter Cajal handles the opposing needs for the investigator to know a lot about disparate fields, especially the basic sciences, and at the same time specialize as much as possible to be able to produce original work.

Perhaps his most important point is the short section on "Mastery of Technique", as he writes:

"...the most important scientific conquests have been won by only a dozen men who have become known for their invention of improvement of a research method..."

5. Diseases of the Will

This is clearly the most entertaining section. The six most dangerous personalities - "who never produce any original work and almost never write anything." - are described in detail. I will only quote the his classification, the rest you should read in his original, quite wonderful prose.

"These illustrious failures may be classified in the following way: the dilettantes of contemplators; the erudite or bibliophiles; the instrument addicts; the megalomaniacs; the misfits; and the theory builders."

While these are quite fun, and you can always find some colleague that fits each personality, the more important lesson is that everyone gets trapped in these mires of the mind from time to time. The important thing is to realise that you are stuck in an unproductive mindset and be able to move on.

6. Social Factors Beneficial to Scientific Work

These are the ever current problems of funding, combining a profession with science and combining work and family. His basic stance is that you can always do science, but you should be ready to sacrifice having a life. Good advice.

7. Stages of Scientific Research

Here Cajal goes over the practicalities of the experimental method. Observation - Experimentation - Working hypotheses - and Proof. It is material gone over many times in other books, although Cajal has the benefit of being brief.

8. On Writing Scientific Papers

To start with Cajal references a Mr. Billings and states four rules: Have something to say; Say it; and Stop once it is said. The fourth rule wasn't as catchy so I'll just drop it. Then he writes a bit about credit and curtesy.

9. The Investigator as Teacher

The final chapter expounds on the importance of fostering future researchers to continue the work once you have gone. He concedes that it might be restful and rewarding to work in solitude but that "Posterity has always been generous with the founders of schools." He then goes on to describe the pains and pleasures of trying to combine research and teaching. First, how to find a suitable potential investigator; then how to guide him; and finally how to see him become a successful scientist leave science for profit and glory.

In conclusion, it is one of the few books that actually describes what it means to be a scientist, specifically a physiologist. It is certainly one of the very, very few that does so while being well written. I heartily recommend it to all my colleagues, and to any one else who might be interested in what science really is.

When is the evidence good enough?

Saturday Morning Breakfast Cereal is the name of a web-comic that is just about as good as xkcd, and they pose some very intimidating questions.

Anecdotal evidence is popular in medicine. Obviously it's not called that. It's called a "case report" or an "observational study". Basically you see a number of patients and then you pick one that supports some notion you have and put it on a pedestal. Another manner of doing it is to collect a number of these cases and call them a case-series. The next level is to collect all your cases and dig around until you find some common feature in all of them or in a sub-population. Finally you get to the level of looking at all people with a certain condition, usually nationality and some disease, and then you can call it "epidemiology". The problem with all of these designs is that if you look hard enough you can find something interesting-looking in any collection of data. This is especially true if you don't care what you are looking for.

The next level of scientific quality is to care what you are looking for from the beginning. This means you pick a group of people and say that, for example: "I believe those with higher blood pressure now will die sooner than those with lower", and then you wait. After some time, in the best case a predetermined time, you check how many have died and draw your conclusions. Such a conclusion might be that high blood pressure is a risk factor for death. This is not necessarily such a bad way of doing science. The problem is that then you start muddling through your data to look for a reason for the difference in death rates, and then you are suddenly producing anecdotal evidence again.

The highest level of evidence is not to use a found population with some difference, but to use a homogenous population and induce a change in some of them while comparing them to the rest. This is experimental science and it is the only way to show causal relationships. You still have to preconceive what you are looking for, otherwise you are still just muddling through the data looking for differences. This means that an experimental study is only valid as such for the question it was designed to answer. If you take the population in an experimental study and look at something else then you are back to doing an observational study, producing anecdotal evidence.

So, what is the problem? Anecdotal evidence often turns out to be correct. This is how much science comes about; you have an anecdote (say a case study, or some epidemiological finding, or a post-hoc analysis of an experimental study) that leads you to a hypothesis. Then you test the hypothesis with an experimental study to see if it holds up. Easy. This is how it is supposed to be. The problem is that anecdotal evidence is often taken for true directly (by the media, policy makers and the public at large), without further testing, and the later experimental evidence does not get the same attention even though the science is better.

For scientists this is not as big a problem as for the public. By following the field you learn what questions are being asked and which study was designed to answer which question. With clinical trials this is solved today by publication of the protocol and hypothesis before the study starts. Pre-clinical papers on the other hand are mostly experimental in nature, but they are often written so that it is not clear if the conclusion is based on the original question or if it is something the authors picked up on the way. This is because there is not enough space to tell the story as it happened, and that it frankly wouldn't be that good a read. The principle is that you take what you have and write the best story possible. That's how to get published. What it also means is that, unless you know the investigators and what their main focus is, you don't know if the study was a preconceived experimental study or a post-hoc study.

Getting know the investigators in your field means going to a lot of conferences and listening to a lot of talks. That's when it is a good thing to have some online comics to fall back to. Because falling asleep is embarrassing.