Cover of South Africa's Brave New WorldThere are at least two possible stories one could tell about this book, and about the career of its author - a career of which the book is a kind of summation.

One tells of a former radical Africanist, Oxford disciple of the Marxist historian Thomas Hodgkin, and before that an idealistic anti-apartheid activist in his native South Africa. With age, material privilege and perhaps some personally embittering circumstances, he turns ever more to the right and begins denouncing, with every rhetorical trick at his disposal and sometimes with nasty racial overtones, the African National Congress (ANC) leaders he'd once so admired. It is a story of apostasy and bitterness, of a sour old man refusing to see any good in the great transformation the rest of the world so rightly celebrates.

The second story is the one RW (Bill) Johnson himself tells. He has remained consistent, always upholding liberal and democratic values. It's the ANC leadership - crooks, Stalinists, gangsters, murderers, incompetents, anti-white racists - who have betrayed everything he has always stood for, and for which they too once appeared or pretended to stand.

Which story is true? Of course, parts of both could be: that is, Johnson has indeed become embittered, but so much has indeed gone wrong in South Africa that he has good reason to be so. One South African reviewer says shrewdly that the book could well be seen as a long "I told you so", since Johnson has been a prophet of doom for so long. If this book is in the main a densely documented chronicle of follies, crimes and tragedies, then no honest account of the country's recent progress could avoid being so. Johnson's indictment is in great part utterly and accurately devastating, and he hits many of his targets bang on. Yet it is also quite startlingly one-sided, and marred by some grossly overstated claims, a fair bit of repetition and a rather undeveloped, only half-convincing overall framework of interpretation.

That framework is his model of "failed colonisation". The ANC's - and much of the world's - view is that the new South Africa is struggling to overcome the crushing inheritance of colonial rule, which is seen in entirely negative terms and argued to have persisted until the 1990s. (Even as an independent state, white-ruled South Africa in this perception remained subject to a form of "internal colonialism".) Johnson urges instead that we must recognise the real, enduring benefits that colonial rule brought. The key question is whether new states managed to build on the positive parts of the colonial legacy - as he says Brazil, for instance, has done - or, by failing or refusing to do so, regress toward the condition of a "failed state". Most of Africa has followed the latter course. Zimbabwe is the latest example, and among the most extreme. At worst, South Africa may be going down that path. In Johnson's eyes, former President Thabo Mbeki's support for Mugabe is a terrible portent of South Africa's possible future as well as a scandal in itself.

The "failed colonisation" image is Johnson's key analytical move, and it is this, if anything, that lifts the book above the category of polemical journalism. He supplements it with a picture of the collective psychology of the formerly colonised, on which he says he is following Frantz Fanon, though actually the portrait he paints of an irretrievably damaged, paranoid, vengeful psyche is much more like Octave Mannoni's than Fanon's. Johnson thinks that wounded collective psyche explains much about black South African behaviour and - in an extreme form - that of Mbeki. Mbeki is the prime villain of Johnson's story, though former ANC military supremo and later Defence Minister Joe Modise runs him close for sheer unrelieved malevolence. Johnson has some unfashionably harsh things to say about Mandela too: he was a do-nothing President, a misleadingly benign-looking figurehead for an already rotting regime, often little more than a puppet in the hands of Mbeki. Paranoid, manipulative, seething with suppressed rage at the "white" world, increasingly losing touch with reality and with his mental balance, Mbeki is portrayed here as one of the worst people ever to head a major country in modern times. It all came to a head, and was shockingly exposed to the world, with Mbeki's AIDS denialism. Johnson's account of the South African AIDS disaster is among his most powerful passages, if lacking the detail of Helen Epstein's or the sophistication of Didier Fassin's books on that theme.

His view of Jacob Zuma, conversely, is perhaps surprisingly positive. It's not that Johnson likes, trusts or admires the man, but simply that for him anyone who is not Mbeki, and has long been hated and conspired against by the latter, must be an improvement. Even so there is a strong element of damning with faint praise - it is not that he thinks Zuma innocent of corruption charges, simply that he was no more corrupt than anyone else in the ANC inner circle, and was singled out for blatantly political motives. Similarly, Zuma's sexual behaviour, including his polygamy, is neither better nor worse than that of many other ANC leaders, just more overt and maybe thus a bit more honest.

On wider fronts, the catalogue of calamity is unrelenting. Conspiracy theories abound, not least in relation to the murder of Communist Party leader Chris Hani. On the Truth and Reconciliation Commission, Johnson closely follows the damning judgements of Anthea Jeffery. More disconcertingly, he seems too often to assume - contrary to a great deal of evidence the other way - that apartheid-era courts were objective, but subsequent ones and the TRC itself biased. Thus when TRC findings clash with those of earlier legal processes, Johnson pretty consistently and almost it seems unquestioningly follows the latter's view. Every institution - police, army, prisons, civil service and, above all, the health and education systems - has gone down the drain. Black Economic Empowerment has been a total disaster - Johnson calls it a "Wild West world" of corruption, incompetence and alternate backscratching and backstabbing. The renewed racialisation of politics and language is the greatest tragedy of all, with black "Africanist" chauvinism abounding and whites (as well as Indians and Coloureds) systematically discriminated against. White flight has been the inevitable upshot, robbing the country of much of its most needed skills and human capital. Again Mbeki is the prime villain here, but again is suggested only to display in extreme, indeed mentally disturbed, forms a syndrome very widely present among black South Africans.

There is a repeated emphasis on how lacking in educated and talented people the ANC has always been. This obviously begs the question "compared to what?" It might also, to hostile readers, sometimes seem as though Johnson is pouring scorn on the mental capacities of black South Africans in general. That is obviously not his intention, but occasionally his choice of language is unwise. Too often he refers sweepingly to supposedly typical "African" modes of behaviour: always, of course, foolish and unattractive ones. He comments at one point: "The age of the big fat momma, the African earth mother, ended abruptly and the smarter malls teemed with slender young African women, hair straightened, and dressed to kill." This is at best in dubious taste. In another place he complains "how much writing about South Africa is still heavily ideological" - but seems to exempt himself from the stricture, and not to have much critical purchase on his own predispositions and biases.

The final pages show some signs of hasty writing but even so cannot escape the obvious, and indeed perhaps unavoidable, problem of making predictions that subsequent events have called into question - above all in relation to potential splits in the ANC (which have happened) and potential challenges to the party's electoral hegemony (which has now at least been dented). Yet Johnson's last chapter also seems at least partly to reverse positions and offer some grounds for optimism. Perhaps that relative (very relative) good cheer is not so surprising.

The standard "pessimistic" narrative about South Africa is that things began very well in the first post-apartheid years and under Mandela, and later got worse. Johnson's view is that actually they started dreadfully badly, in great part because of the political culture of exile, conspiracy, violence and neo-Stalinism which the ANC brought into government. Some things have subsequently improved. In the end, Johnson has hope in the strength of South Africa's democratic political culture and civil society. It is that which, he believes, may redeem the country from all the dreadful disasters he has chronicled. One might venture too that this final streak of qualified hopefulness is what redeems his book from being the unrelieved, sneering and snarling diatribe it otherwise comes so uncomfortably close to being.

As I write, Bill Johnson is gravely ill, having contracted necrotising fasciitis after cutting his foot while swimming. I wish him well, and hope desperately that he will soon recover enough to take a full part in the heated debates his book will undoubtedly prompt.

South Africa's Brave New World is published by Allen Lane