From Kim Stanley Robinson’s “Imagining Abrupt Climate Change : Terraforming Earth” (Amazon Shorts: 31 July 2005):
… paradigm shifts are exciting moments in science’s ongoing project of self-improvement, making itself more accurately mapped to reality as it is discovered and teased out; this process of continual recalibration and improvement is one of the most admirable parts of science, which among other things is a most powerful and utopian set of mental habits; an attitude toward reality that I have no hesitation in labeling a kind of worship or devotion. And in this ongoing communal act of devotion, paradigm shifts are very good at revealing how science is conducted, in part because each one represents a little (or big) crisis of understanding.
As Thomas Kuhn described the process in his seminal book The Structure of Scientific Revolutions, workers in the various branches of science build over time an interconnected construct of concepts and beliefs that allow them to interpret the data from their experiments, and fit them into a larger picture of the world that makes the best sense of the evidence at hand. What is hoped for is a picture that, if anyone else were to question it, and follow the train of reasoning and all the evidence used to support it, they too would agree with it. This is one of the ways science is interestingly utopian; it attempts to say things that everyone looking at the same evidence would agree to.
So, using this paradigm, always admitted to be a work in progress, scientists then conduct what Kuhn calls “normal science,” elucidating further aspects of reality by using the paradigm to structure their questions and their answers. Sometimes paradigms are useful for centuries; other times, for shorter periods. Then it often happens that scientists in the course of doing “normal science” begin to get evidence from the field that cannot be explained within the paradigm that has been established. At first such “anomalies” are regarded as suspect in themselves, precisely because they don’t fit the paradigm. They’re oddities, and something might be wrong with them as such. Thus they are ignored, or tossed aside, or viewed with suspicion, or in some other way bracketed off. Eventually, if enough of them pile up, and they seem similar in kind, or otherwise solid as observations, attempts might be made to explain them within the old paradigm, by tweaking or re-interpreting the paradigm itself, without actually throwing the paradigm out entirely.
For instance, when it was found that Newtonian laws of gravitation could not account for the speed of Mercury, which was moving a tiny bit faster than it ought to have been, even though Newton’s laws accounted for all the other planets extremely well, at first some astronomers suggested there might be another planet inside the orbit of Mercury, too close to the Sun for us to see. They even gave this potential planet a name, Vulcan; but they couldn’t see it, and calculations revealed that this hypothetical Vulcan still would not explain the discrepancy in Mercury’s motion. The discrepancy remained an anomaly, and was real enough and serious enough to cast the whole Newtonian paradigm into doubt among the small group of people who worried about it and wondered what could be causing it.
It was Einstein who then proposed that Mercury moved differently than predicted because spacetime itself curved around masses, and near the huge mass of the Sun the effect was large enough to be noticeable.
Whoah! This was a rather mind-bogglingly profound explanation for a little orbital discrepancy in Mercury; but Einstein also made a new prediction and suggested an experiment; if his explanation were correct, then light too would bend in the gravity well around the sun, and so the light of a star would appear from behind the sun a little bit before the astronomical tables said that it should. The proposed experiment presented some observational difficulties, but a few years later it was accomplished during a total eclipse of the sun, and the light of a certain star appeared before it ought to have by just the degree Einstein had predicted. And so Einstein’s concepts concerning spacetime began to be accepted and elaborated, eventually forming a big part of the paradigm known as the “standard model,” within which new kinds of “normal science” in physics and astronomy could be done. …