One evening towards the end of 1974, a large seminar room in Balliol College, Oxford, buzzed with excitement over what promised to be a fascinating public debate. The two protagonists – the English historian and activist EP Thompson and the exiled Polish philosopher Leszek Kolakowski – were still in their forties, but you could tell from their lined, expressive faces that they had already lived long and interesting lives. They were not academic functionaries so much as committed public intellectuals, reviled by the orthodox but revered by others as freethinking political prophets or even moral gurus.

Both Thompson and Kolakowski had come to fame in the preceding two decades for their valiant attempts to snatch hope for political progress from the jaws of betrayal, compromise and defeat, and they were regarded as the leading architects of a bundle of sentiments and doctrines known at the time as “socialist humanism”. But their backgrounds were very different, and it seemed they might now be travelling in different directions. The question for all of us that evening was whether they would manage to find any common ground.

Edward Thompson spoke first. He was a dissident British socialist who had left the Communist Party in 1956 and pledged himself to rescuing the best of the Marxist tradition from the grotesque distortions, as he saw it, of Stalinism and Soviet imperialism. He was the author of a paradigm-shifting book, The Making of the English Working Class (1963), and of masses of essays, articles, lectures and pamphlets in which he sought to energise the so-called “New Left” and equip it with fresh theoretical resources. His basic argument was that Marxism should be interpreted not as an economic theory of historical inevitability but as an affirmation of the creativity of ordinary working people, and their infinite capacity for devising new forms of poetry, piety and politics.

Thompson began with a tribute to the person he had long regarded as a comrade in the struggle for socialist humanism. Kolakowski had recently published a short book called Marxism and Beyond, which appeared in the same series of desirable paperbacks as Germaine Greer’s Female Eunuch and Shulamith Firestone’s Dialectic of Sex. For those who wanted to believe that Marxism might be a philosophy of personal liberation rather than a dogma of coercive collectivism, Marxism and Beyond had come like rain to parched soil. Kolakowski put forward a broad distinction between the mechanistic Marxism propounded by Engels, Lenin and Stalin – “Marxism of a positivist orientation”, as he called it – and a more appealing tradition of “pragmatic Marxism” that could be traced back to the neglected philosophical writings of the young Marx.

Positivist Marxism, according to Kolakowski, was a fraudulent imposture, pretending to discover “iron laws” of historical development with the dispassionate objectivity of the natural sciences: it was a theoretical nullity, with a murderous practical record. On the other hand the humanistic, pragmatic alternative – which focused on the idea of “alienation” or “the situation of man who has lost control of the world he himself called into being” – was, according to Kolakowski, still full of promise, and “philosophically worthy of continuation”.

The essays in Marxism and Beyond had been written in the ’50s, and Thompson recalled the passion with which he had followed the activities of Kolakowski and his fellow dissidents in Eastern Europe ever since that time. Their courage in the face of violence and oppression had been, he said, a constant inspiration to him and his friends in the New Left as they tried to launch an echoing movement of socialist humanism in Britain. And now that the unfortunate Kolakowski had lost his job as a philosophy professor in Poland – he was expelled from the Communist Party in 1966 and driven into exile in 1968 – Thompson wanted to salute him as an old comrade and embrace him like a long-lost friend.

Kolakowski looked listless, embarrassed and depressed. His former ardour and optimism had been replaced, it seemed, by bitterness and dry sarcasm, and he was not about to reciprocate Thompson’s warmth and generous elation. He recalled how one of his fellow philosophical dissidents in Poland had “been withdrawn from historical circulation, having been murdered by the missionaries of great historical justice,” and he made it clear that he was not remotely interested in raking over the coals of socialist humanism in the name of some new version of vengeful history. Experience had taught him the value of caution. He was not going to forget that as a professor in Warsaw University he had at least been treated with respect, and permitted to teach exactly what he chose.

When he fled to the West in 1969 and sought refuge in Berkeley, California, he was horrified by the spectacle of uncouth students, high on some abstract idea of revolution, who considered themselves entitled to choose how they should be taught. Indeed he was appalled by the entire “radical student movement” in the West: “what I saw and read I found pathetic and disgusting,” he said, and “what impressed me was mental degradation of a kind I had never seen before in any kind of leftist movement.” Now that he had escaped from the United States and gained admission to the cloistered decorum of All Souls College, Oxford, he had no intention of picking up a red flag and conspiring against the institutions that were offering him shelter and asylum. He had come to bury socialist humanism, not to praise it.

The rebuff was brutal but it should not have come as a surprise. Kolakowski had already appeared in the pages of the cold-war periodical Encounter, denouncing the spirit of “romantic nostalgia” and “irrational revolt” that, according to him, pervaded the Western left. “The kind of language that was used in the past to justify the most brutal oppression,” he said, “is now being repeated as though nothing had happened.”

Thompson had responded early in 1974 with a 100-page “Open Letter” in which he insisted on praising his old hero for his unswerving allegiance “not to the Communist Party as institution or as ideology, but to the Communist Movement in its humanist potential.” He went on to elaborate the idea of “humanist potential” in a copious and brilliant declaration about the connections between historical method and political hope. Drawing on the work of Alasdair MacIntyre and, indirectly, Jean-Paul Sartre, he presented Marxism as a historicised form of humanism: social reality, he suggested, was not a closed set of inert facts but an open-ended activity and a “value-making process”, constituted by human choices playing freely over a range of human alternatives.Marxism, according to Thompson, was not a quasi-scientific project of discovering objective “laws”, but a humanistic attempt to elicit the “logic” of free human action, and exhibit it against the practical background of unrealised human possibilities.

Kolakowski was massively unimpressed. He had in fact just written an angry response to the “Open Letter”, entitled “My Correct Views on Everything”, where he condemned his interlocutor as a dupe of the old-style communists. While Thompson and his New Left comrades fiddled with fine words about Marxism and human freedom, they seemed unaware that China and the Soviet Union were preparing a “great Apocalypse” – a war of total destruction “between two empires both claiming to be perfect embodiments of Marxism”. Meanwhile any impartial investigator could see that all the main tenets of Marxism, even in its well-meaning “humanistic” forms, had turned out to be “either false or meaningless”. In a better world he and Thompson might be able to make friends, and he hoped they might be “generous enough to forgive each other”. But for the time being they had nothing to say to each other, and, in the event, they never spoke again.

Back in All Souls, Kolakowski worked on an ambitious survey of the Main Currents of Marxism, which appeared in three large volumes in 1978. It received respectful reviews, but even its professed admirers found it hard to conceal the extraordinary tedium of reading page after page of second-hand information about hundreds of forgotten thinkers, especially as, if Kolakowski’s rough and ready summaries were anything to go by, they were all so pathologically deluded that no one in their right mind could expect to learn anything from them. But if the book was short of suspense, animation or narrative drive, it was at least relentlessly consistent. Marxist after Marxist was arraigned at the Kolakowskian assizes, and one by one they were condemned out of their own mouths for the sin of metaphysical pride.

Marxism was revealed, over and over again, as essentially a “historiosophy”, a Procrustean theory of history based on the a priori assumption that a time will come when humanity will break away from its age-old inheritance of scarcity, toil and woe, and enter into perfect possession of itself and its world. Marxist historiosophy was supposed to be atheistical, but it was really no more than a derivative of “humanism”, or the self-intoxicated belief in human omnipotence; and humanism in its turn was the bastard offspring of Christian theology. Marxism and humanism, far from offering a Godless alternative to theocracy, were really just trashy replicas of its cheapest and nastiest themes.

Main Currents of Marxism has had no appreciable impact on the work of later intellectual historians, and those of us who have worked our way through it from beginning to end are unlikely to want to open it again. It is nevertheless a touching document of a 20th-century life laid waste by the Marxist movement, and when Kolakowski died his obituarists had no hesitation in declaring it a masterpiece.

Masterpiece or not, Main Currents eventually bore fruit for Kolakowski. By the time of the collapse of the Soviet empire in 1989 he was too settled to consider moving from his Oxford home. But his work was reprinted to great acclaim in his native land, and he started making regular trips there, becoming a hero of the new political class, and achieving popular celebrity though a series of newspaper articles and broadcasts. Gradually, he made it clear that he had given up his old commitment to atheistic secularism. But he always refused to answer questions about his newfound sense of spirituality and the sacred. And unfortunately he never revisited his old habit of mocking socialism and humanism for their affinities with Christian theology; it is hard, after all, to see how that could still be a problem for someone who has renounced religious unbelief.