The people who protest the most about how New Labour has lost touch with its working-class roots are the very people the party has really lost touch with – the liberal middle classes. They complain precisely because Labour’s thinking is now closer to those poorer, white, working-class voters the intelligentsia claims to speak for.

A page from Julian Baggini's 'Spongers', showing David Threlfall as Frank Gallagher, lead 'sponger' in ShamelessThe thought strikes me in the wake of the wail of outrage provoked by the Labour housing minister Caroline Flint’s idea that social housing residents could be asked to seek work as a condition of their tenancy. “Social housing should be based around the principle of something for something,” she said.

Cue lots of tut-tutting. Isn’t this a classic example of scapegoating the poor for their plight, blaming the victims rather than the perpetrators for their crimes? How could a Labour government say this kind of thing?

To which I say: wake up and smell the Typhoo. While Guardian bloggers took turns to dismiss Flint’s “tirade against council tenants she regards as work-shy”, people in the “convenient cartoon image of a work-shy layabout, with tattoos and lager breath, watching daytime TV”, few took seriously the possibility that, however ill-conceived her plan, she might have a point.

One exception was the Mirror, a newspaper that is read by and speaks for the genuine working class, and so can say things that more genteel “socialists” would not contemplate. While disagreeing with Flint’s precise proposal, a Mirror editorial agreed with her diagnosis of the problem: “Council homes were for the working class. Not the unworking class.”

When I spent six months in the demographically representative S66 area of Rotherham, I heard many white, working class men say very similar things. These were often people who had worked in coal and steel, who supped their ale in the local pub and looked down on those who went to the nearby Working Men’s Club, a name which they now regarded as ironic. As they saw it, it was stuffed with “dole wallahs” – the willing unemployed who went for the cheap beer, paid for by honest, hard-working taxpayers like themselves.

The people who are most upfront and passionate about the scourge of “spongers” are ordinary working people like this. In echoing their concerns, New Labour is not becoming more like the Tories, it is becoming more like its traditional voters.

Take as another example Margaret Hodge’s suggestion last year that housing provision should favour families who have lived in the UK for many years over new economic migrants. Again, this produced howls of protest from the liberal chatterati, but these views were entirely in tune with mainstream British opinion.

This middle-class myopia is exemplified by how, while New Labour decries the feckless, the Guardianistas revel in Channel 4’s Shameless. Shameless allows its viewers the double indulgence of laughing at the scumbags of society while also loving them in an odd way. But if you had these people as neighbours, you’d find them neither amusing nor lovable. You’d probably try to get an ASBO put on them, another measure loathed by the left middle classes but embraced by those they purport to defend.

So why the total disconnection between middle-class squeamishness and popular brutality? One reason is that, for all the think-tank hand-wringing about the need for strong community values, the people who still live by them scare the pants off metropolitan lefties. For community implies membership, and this in turn implies that membership is conditional, and that some people are non-members. People who are newly arrived on our shores or prefer to cash a dole cheque rather than work are, on this view, not fully paid-up members, and so don’t deserve the same privileges as those who are. That’s real community for you – not an annual neighbourhood litter drive followed by a family picnic.

A second reason is a mistake identified by the philosopher TM Scanlon in his forthcoming book. Scanlon argues that we have assumed that to blame someone implies that we believe they were free, at a deep level, to have done otherwise than they did. It is not then surprising that educated liberals bend over backwards not to say anything that might blame the poor for their condition.

But, Scanlon says, this account of blame is wrong. Rather, blame is appropriate when someone’s behaviour contravenes the rules of “what we owe to each other”. To use an example which is not Scanlon’s own, and which he may not like, when someone willfully stops contributing to the public purse but keeps taking from it, they break the contract between the individual and her fellow citizens. It then may be appropriate to withdraw privileges – not as a punishment, but because the appropriate relations on which these benefits rest are no longer in place. Hence encouraging, or even forcing, people to take more responsibility does not imply it is their fault they didn’t take it in the first place.

This would ring true in the British heartlands. Whatever the “root causes” of unemployment or anti-social behaviour, the way most people see it, fairness usually demands that what you take depends at least in part in what you give.

I don’t think that Flint, Hodge or the mainstream opinions they represent are entirely right. But I do think that most of the outrage at Flint has missed more than one point. The uncomfortable truth left-liberals need to understand is that you do not stand with the ordinary working man or woman by defending illiberal measures against communitarian ones: you stand against them.