AT DOWNING Street upon the stair, I met a man who wasn't Blair. He wasn't Blair again today. Oh how I wish he'd go away." So read the mystery quatrain, allegedly penned by a disgruntled Cabinet minister, which circulated Westminster last week.

The verse paraphrases the American poet Hughes Mearns's well-known Antigonish: "As I was going up the stair, I met a man who wasn't there ..."

This isn't the first time that Brown has been hit by a rocket-propelled stanza, either. TS Eliot's Macavity: The Mystery Cat has been widely used by the prime minister's critics, and the poems relate to his tendency to absent himself when things go wrong ("Macavity's not there").

Obviously, this is just dodgy doggerel, but humour has a knack of revealing truth. The image of Gordon Brown now becoming fixed in the public mind, and reflected in these poems, is that of a politician who cannot face up to adversity. The truth is Brown does absent himself from difficulties. He went to ground during the Northern Rock debacle, and he has gone to ground again over the credit crunch.

What has the prime minister had to say of any significance about the crisis? Nothing that I can recall. Can it really be that the man who presided over the British economy with such apparent success for more than a decade, who is lauded as the best chancellor in 100 years, now has nothing to say about what is the greatest financial crisis since the second world war? Apparently, because he wasn't there again last week.

One suspects that Blair, like him or loathe him, would at least have been out there calming fears and dispelling rumours. Challenging irresponsible city traders; raising questions about the banks' behaviour and why we should bail them out. He would have had a soundbite, even if it was: "When the whorehouse burns down, it's not just the pimps who perish." OK, perhaps not that one.

It might not have done a great deal of good, but it would at least have given the voting public an impression that someone was thinking about it, that there was someone looking after our interests. But Brown's policy, as always, is to keep his face out of the papers and the bulletins during a crisis so he's not contaminated by bad news. It's what he does.

But the problem with presentational absenteeism is that it is a lot harder for a prime minister to hide than a chancellor. Before, Brown could bury himself in the Treasury whenever Blair was getting into a mess over party funding, foreign wars, NHS cock-ups. Not any more. He is on display at PM's Questions week on week, and he is looking ragged, exhausted, clapped out.

Now appearances aren't everything, of course. A few bags under they eyes are expected in a leader - a price worth paying for the privilege of being PM. But, increasingly, Brown looks like a loser and David Cameron is getting the better of him week by week - much as Blair used to ridicule the Tory MP John Major.

The truth is that this government is in the midst of a severe downturn. Brown is suffering his own credit crunch in the form of falling poll ratings. The latest YouGov survey has the Tories in the lead in any UK election by 16 points (43 to 27), and ICM last week gave Cameron's party a 13-point lead (42 to 29). These are very serious numbers and indicate that turbulence in political allegiances in Britain is as serious as turbulence in the markets. People are looking at Brown more closely than ever and are not liking what they see.

We are so used to thinking about the Tories as the party of no return that we are perhaps failing to notice that they are making a serious comeback. The Labour Party is demoralised and uncertain about its future. The atmosphere on the government benches is gloomy and negative, and there is turmoil in Brown's private office with the departure of his close aide of 10 years, Spencer Livermore.

He found life under Brown's new chief of staff - the public relations expert Stephen Carter - less than congenial. Brown is surrounding himself with people who made their names in advertising, David Muir, and investment banking, Jennifer Moses, rather than in Labour politics, which is why some are talking about a Tory takeover in Number 10. But this leadership requires more than a PR facelift. Labour has lost four million votes since 1997 and it will not be easy to get them back.

The problem is that people don't know what Labour stands for under Brown. The widely expected return to more traditional Labour values never happened. Indeed, Brown has been even more neo-liberal than his predecessor, pressing ahead with public sector reform, cutting inheritance tax, blocking attempts to curb the tax avoidance of "non-doms". Ministers such as John Hutton have been free to call for wealthy people to be "celebrated" just at the moment when the rich have plunged the world into economic crisis.

Brown has pushed ahead with terrorist detention; he has joined with France to lead a global revival of nuclear power. He could have used the fifth anniversary of the invasion of Iraq last week to try to draw a line under Britain's greatest foreign policy disaster in half a century. But he hasn't apologised for the war, or indicated a new strategy for getting out of it, other than retreat by stealth. He certainly hasn't tried to give us a vision of world affairs post-Iraq - a new philosophy of international relations following the collapse of American neo-imperialism.

It's just non-business as usual - wittering about a "national risk register"; maybe meeting the Dalai Lama, but only if it doesn't upset the Chinese; dithering over a free vote on the Embryology Bill. Say what you will about Blair, but he did at least make decisions and tried to give an account of the world in which he made them. They may have been the wrong decisions but - as he always said - at least he made them. The danger is that Brown appears to be a passive victim of events rather than the master of them. That is not a credible attitude for a prime minister, who must at all times be visible and proactive.

Brown was supposed to be the keeper of the soul of Labour; he has turned out to be just another desperate politician surrounding himself with "brand" managers to sell the political vision he doesn't have.

But wait. What's this? A new mystery poem has landed in my inbox. "As I was going to the polls, I met a man who lacked Ed Balls. I hope he comes again this way, so I can say he's had his day."