I’m walking through Traktornaya Ulitsa – Tractor Street – in St Petersburg, the first public housing scheme built in the Soviet Union. You might think you’d know what a council estate called Tractor Street in Leningrad looks like. A giant Lubyanka maybe, or a series of identical tower blocks. You’d be very wrong. This isn’t terrifying, totalitarian, inhumane – but pretty, elegant, humanist.

Designed in the mid-1920s by a team headed by Alexander Gegello, for the revolutionary workers of the Putilov factory, it’s a series of low-rise blocks of flats, designed around open courtyards, with plenty of trees. The first things you notice are the colours – pink and orange, providing accents in an already brightly painted city – and the half-arches which punctuate the buildings. It breaks with the local neoclassical architecture, but through subtle gestures – strange geometries, the lack of a divide between front and back – rather than by crude domination. Walking round Tractor Street over 80 years later, it’s battered but idyllic, with the original idealism palpable. But nearby, you can find the intimidating, shocking or toweringly banal housing of the ’50s, ’60s and ’70s. The story of what happened here, and how an early Marxist-humanist promise was eclipsed first by the brutal Stalin era, then the grimly utilitarian Brezhnev decades and finally the proudly philistine Yeltsin and Putin years, is written all over the buildings.

In the stereotype, the Soviet Union was one giant 1960s council estate. Blocks on blocks, leading interminably for miles out from the town centres, drawn up according to idealised, rationalistic plans that took neither emotion nor comfort into account.

If the Soviet Union’s confrontation with the environment, with its deployment of a rationalism taken to the point of destroying nature, is a humanism gone malign, then so is its architecture, at least in theory. In fact it was only after 15 years of experimentation that Party edicts created a recognisably “Stalinist” style. The USSR is most often associated with gigantic housing projects. These might be either the monumental terror of the high Stalin era, giant classical blocks with wide streets for the tanks to flow through easily; or perhaps the serried towers of Brezhnev’s reign, when engineers supposedly solved the problems of architecture via hundreds of thousands of prefabricated tower blocks, built within a couple of decades.

A trip round Tractor Street reminds that in the process something very valuable was lost. It forms a central part of Narvskaya Zastava in St Petersburg, an area designed between 1925 and 1932 with a similar wit and imagination. There are both new forms and new functions here. First, familiar building types – a school, a town hall – are radically re-imagined. There’s the School for the 10th anniversary of the Revolution, a building arranged in the shape of a hammer and sickle, just as churches used to be arranged in crosses, with an observatory on the top floor so that the children can see what there actually is in the skies; and the district Town Hall, a long, asymmetrical strip-windowed building by Noi Trotsky (no relation), culminating in a Constructivist tower with an illuminated hammer and sickle. But the other buildings in the area are completely new in their functions. One is a “Factory Kitchen”, a combined catering centre and social space designed to save local women from “kitchen slavery”, an avant-garde design of concrete and glass which is now besmirched by logos – a common feature of contemporary Russia, where every available space is covered in advertising. One wing of the kitchen now houses a McDonalds. In a slightly better state, over the road, is Gegello’s Gorky Palace of Culture, named after the great man himself.

During the 1920s, these “Palaces of Culture” and “Workers’ Clubs” were supposed to replace the old kinds of civic buildings – churches and palaces. The designer El Lissitzky wrote that these were “the power sources of the old order, whose power can be only be transcended by establishing new power sources belonging to our new order”. Initially, they literally converted churches into the new centres, with Noi Trotsky managing to turn one Orthodox church into a Constructivist workers’ club, but from the mid-20s on they got new buildings. The new clubs and “palaces” were combined concert halls, cinemas, educational centres and sports centres, which were intended to encourage audience participation rather than passive consumption. Now, they mainly continue as concert halls or mundane restaurants, but their promise can still be seen in the confidence and drama of the architecture.

But like the writers who began as idealistic young communists and found themselves writing paeans to slave labour, the architects of Narvskaya Zastava would soon be serving less humane functions. A few miles away from this working-class suburb, in the city centre, is the “Bolshoy Dom” or Big House, designed by Trotsky and Gegello for the NKVD, later the KGB, and still used by the FSB. Here, their modernist lines become rigid and imposing, and the use of glass is opaque rather than transparent in its effects. The building lords it over a stretch of central St Petersburg, looking more than anything else built in this era like a stereotypical Soviet building – domineering, “totalitarian” – and Constructivist architecture in the city ends here.

Moscow has much more Constructivist architecture, although here too its legacy is appallingly treated. Today, everything from pitched roofs to Azerbaijani themed restaurants have been added to incredible buildings like the Rusakov or Zuev Workers’ Clubs, where Constructivism’s raw, revolutionary futurist ferocity is far more apparent than in placid Tractor Street. The contrast between the serene Petersburg and the delirious Moscow is still striking – where the former is still in a stunning state of preservation, an almost complete neoclassical city with only a few crass recent edifices spoiling it, the capital is an exhilarating mess, a hypercapitalist chaos with an architectural vocabulary still deeply indebted to the Gothick kitsch of Stalinism, kept coherent seemingly only through its palatial metro system.

Moscow Planetarium is the saddest remnant of the Constructivist era. This was intended as a temple to popular science, offering visible proof of atheism and demonstrating the wonders of the cosmos. When it reopened a few years ago, an Orthodox priest was hired to bless the opening ceremony. It’s boarded up now, with the owners and the Mafia feuding over the site. An equally melancholic place to see what remains of the early revolutionary dreams is the Narkomfin building in central Moscow. This housing block was intended as an exemplar, at least in 1928, to the rest of the country. Sandwiched between the perimeter walls of the US Embassy and an especially nasty example of the grotesque Vegas architecture of the oligarchs, is a long block of flats, with a small park in front of it, a roof garden and a glass wing, connected by an overhead walkway. This was supposed to be a “communal house”, with housing on one side and collective facilities like a library and canteen in the wings. It’s now literally falling apart, visibly crumbling to pieces, despite a UNESCO campaign to save it. Yet its neglect began in the 1930s, when the idea of communal living was rejected as dangerously “leftist”.

What happened instead was a Stalinist architecture that borrowed from practically every historical period – neoclassical details, Gothic spires, Byzantine gilding, skyscraper technology, Victorian ornament – to create something both horrible and thrilling, the kitsch of a humourless dictatorship. Just next to the Narkomfin is one of the “Seven Sisters”, a series of skyscrapers commissioned by Stalin after the Second World War. If the era of workers’ clubs and collective housing had the confidence to create new forms, these elite skyscrapers created bizarre hybrids, amplified to freakish proportions. But in so doing, they borrowed liberally from ecclesiastical architecture – not as a way of replacing the church in a secular society, but in order to reproduce its irrationalist domination in a new form, through cathedrals of bureaucracy like the terrifying Ministry of Foreign Affairs.

Yet if this Stalinist style scraped the sky, its more ambiguous legacy is underground. The Moscow Metro’s “Palaces for the People” were designed with an eye to the crypts of cathedrals or the funerary spaces of the pyramids, to create light, beautiful spaces without any natural illumination. It’s still the most amazing thing in Moscow, and here perhaps a religious architecture was successfully secularised – but the mosaics and statues of heroic workers are totally a product of Stalin’s “socialist realism”, a didactic, literary architecture. Stalinist writers compared Soviet construction projects favourably to the Pyramids, as if to excuse the slave labour that built both; but the Moscow Metro at least serves a purpose other than glorification of the Pharaoh.

Today, after the suicide bomb attacks that occurred just a week before I visited, a ban on taking photos in these “palaces” is rigidly enforced, and the uniformed Militia men stalk these stunning vaults, keeping a close eye on anyone who might step out of line. Yet the Metro is still the most fascinating monument to an era that exemplifies the dark potential and the fantastical possibilities of humanism.