Physics is beset by serious problems, and it is so because of physics. By the first use of the word “physics” here I mean a set of scientific theories about the structure and properties of the material universe, and by the second I mean the human endeavour of enquiry which produces, examines and develops those theories. Call them physics-1 and physics-2 respectively.

Sprial Galaxy NGC 3370, about 100 million light years away, as seen by the Hubble TelescopeThe problem in physics-1 is that Einstein’s general relativity theory, which describes the nature of gravity, space and time, is not consistent with quantum mechanics, which describes the world at the subatomic level. The challenge is to render them consistent, by finding a unifying theory that combines an understanding of gravity with an understanding of the forces that bind elementary particles into atoms.

The problem with physics-2, the human fabric of institutes and university departments and their personnel devoted to research into physics-1, is – according to physicist Lee Smolin’s new book The Trouble With Physics (Penguin/Allen Lane) – that it is preventing itself from making progress in the tasks confronting physics-1. A particular way of thinking about that task has become so dominant that, despite its failure to make substantive progress, it is marginalising other ways of approaching the problem, and (worse) is making it difficult for original thinking (and thinkers) to get a foothold in physics-2.

Smolin has written a remarkable book. First, he gives a wonderfully clear account of the history of physics in the 20th century, with the aim of explaining the theory that has become so dominant in the last quarter-century, namely, “string theory”. Secondly, in ways that a layman (albeit with somewhat furrowed brow and protruding tongue) can follow, he explains string theory – or, more accurately, the vast landscape of string and superstring theories which between them are now the fashion and passion of most people working in physics. Thirdly, he explains what is wrong with them. Fourthly, he diagnoses the institutional pressures that force young physicists into flocking to string theory in order to get jobs in universities. Fifthly, he examines and criticises the worrying efforts to rewrite the nature of the scientific enterprise which some proponents of string theory undertake in the absence of empirical resources for testing the theory. Sixthly, he gives a frank and discomforting analysis of the “groupthink” that sociologists recognise in organisations more intent on protecting their vested interests than pursuing truth, and says that string physics exemplifies this dismaying trait.

And finally, but by no means least, he enters an eloquent plea on behalf of the mavericks, the loners, the original thinkers, the sceptics, the unusual and eccentric minds, who he believes are needed to free theoretical physics from the impasse it currently finds itself in. It is a plea to the institution of physics – to physics-2 – to make room for such people, because without them physics-1 is in danger of losing connection with the real world and the strict control of empirical testing.

Smolin is not only a distinguished and creative physicist in his own right, but has a rich understanding of the philosophy of science, and the courage, credentials and seniority to challenge the physics-2 community to reflect on the point with which he begins his book: that for the quarter-century in which string theory has been the dominating paradigm, no real progress has been made. What is known in physics is practically the same today, he says, as it was in the 1970s. And this sharply contrasts with the fact that every quarter-century beforehand, since the rise of physics in the seventeenth century, one or another substantial discovery has been made.

Smolin sees five major problems facing physics. The first is the need to combine general relativity and quantum theory to yield a unified theory of nature. The second is the need to make sense of quantum mechanics itself, which is full of unresolved puzzles and anomalies. The quantum world is a strange place, and its oddity is a hint that something more fundamental waits to be discovered. The third is the need to determine whether all the particles and forces of the standard model of subatomic physics can be understood in terms of a more inclusive theory that describes them as manifestations of a deeper reality. The fourth is to explain why the values of the free constants of nature – the numbers describing (for example) the masses of quarks and the strengths of the forces binding the atom – are as they are. And the fifth is to come up with an account of two profoundly puzzling phenomena that recent astronomical observations seem to reveal: the existences of dark matter and dark energy.

String theory, first proposed in the early 1980s, promises nothing less than to solve the first problem – the unification of relativity and quantum theory. It does so by postulating the existence of minuscule vibrating string-like strands and loops from whose vibrations the phenomena of gravity and the elementary particles alike arise. String theory succeeds in this remarkable endeavour by postulating nine spatial dimensions, six of them curled up so minutely as to be undetectable, together with various other assumptions: among the standard ones, that there is an unchanging background geometry, and that the cosmological constant – the degree of energy in the universe hypothesised by Einstein as counteracting the gravitational pull of the universe’s mass – is zero. The mathematics describing strings and their behaviour is beautiful, and the laws required to govern string behaviour are elegant and simple.

These facts, together with the power of the theory to achieve the grail of unification (in supersymmetric versions the theory unifies all the matter and force particles, the fermions and bosons), are immensely strong reasons to think it must be true.

But Smolin’s concerns about string theory are that there is no complete formulation of it, that no one has proposed its basic principles, or specified what its main equations should be. Worst of all, it makes no testable predictions because the number of possible interpretations of string theory is so large. Indeed string theorists talk of a “landscape” of many billions of possible solutions. To Smolin’s dismay this last fact has led some of string theory’s senior proponents to claim that experimental verification of theory is no longer necessary in science – the sheer beauty of the mathematics in which the theory is expressed, they appear to say, is enough to convince by itself. Others also appeal to the anthropic principle – the brute fact that the fundamental constants of physics and chemistry are fine-tuned in just such a way as to produce and sustain life – as a way out of the difficulty that otherwise no single version of the theory’s many possible versions presents itself as uniquely right.

It is the fact that string theory makes no testable predictions that gives Smolin his greatest concern, not only about the theory itself but about what this means for scientific culture. Although he acknowledges (as someone who accepts what the maverick philosopher of science Paul Feyerabend had to say about the matter) that there is no single all-embracing correct methodology that mechanically applies across all branches of science, nevertheless answerability to test and conformity to nature are broad parameters that anything properly describable as science must obey.

Insofar as anything might count as a test of string theory, Smolin says, it is the possibility that imminent empirical work might show that the speed of light has varied during the universe’s history. Anything that shows that general relativity might need adjustment would call string theory into question, for it assumes that general relativity is correct. So string theory might be undermined by these external considerations, even though by itself it makes no claims that are subject to experimental assessment.

But it also matters that physics should welcome and encourage a variety of other approaches to the five fundamental problems mentioned above. These include such theories as loop quantum gravity, “doubly special relativity” and modified Newtonian dynamics. All of these make testable predictions, and if wrong can be shown to be so, itself always an advance in science; and therefore, unlike string theory, they are “genuine scientific theories”. String theory’s critics, by contrast, see it as a form of metaphysics (in the pejorative sense of this term).

Physics PhDs have flocked to string theory, Smolin argues, because for the last quarter-century there has been little other chance of getting a post-doc or tenure-track appointment in university departments. He tells of a number of unusual, independent-minded physicists who were unable to get appointments because their work seemed heterodox, but whose contributions are now being recognised. And Smolin rightly points out that institutional factors in the academy play their nefarious part too, Add this sociological problem to the apparent betrayal of rigorous scientific method, and the dominance of string theory in physics-2 seems every bit as bad for physics-1 as Smolin says it is.

Naturally enough, Smolin’s views have prompted controversy, and have been widely criticised – mainly politely in newspapers and science magazines, too often rudely in the blogosphere where good manners are never much of a consideration.

It is impossible for laymen to evaluate the competing merits of detailed scientific theories, but to this reader at least it is very troubling that string theory seems immune to experimental test, and even more worrying that some of its votaries seem to think this does not matter. On these points Smolin makes a strong and disturbing case, and deserves applause for it. He equally deserves credit for a brilliantly lucid account of much difficult contemporary science, which it is every layman’s duty to know as much about as inexpertise allows. ■