IT WAS the kind of early morning phone call that every chief executive dreads. Within just a few minutes, Jeffrey Kindler would find himself with little choice but to pull Pfizer's most promising product out of development after $1 billion worth of investment.

An independent safety board had shown that Torcetrapib, taken in combination with the cholesterol- lowering pill Lipitor, had caused a higher incidence of death and heart problems in clinical trial patients. In the study, 82 patients died while receiving the combination, compared with just 55 of those who received Lipitor alone.

Having served less than six months at the top of the world's largest pharmaceutical company, Kindler would be plunged into a full-blown crisis.

Torcetrapib, designed to raise levels of the "good" high-density lipoprotein cholesterol that clears fatty plaque out of the arteries, was meant to fill a gap left when the patent of Lipitor, Pfizer's biggest-selling drug with annual sales of $13 billion, expires at the end of the decade.

The news knocked $21bn off the company's market value. And, with little else in the drug pipeline with the same sales potential, Pfizer is now desperately in search of acquisitions that could shore up its fortunes.

In any other industry such a collossal loss would result in the exit of at least one top executive. In the pharmaceutical industry, however, such setbacks happen all too often.

AstraZeneca lost £4.1bn off its stock market value in one day in October after announcing that a promising stroke treatment had failed in phase three trials. That same week, GlaxoSmithKline said that it had disappointing data on its Redona treatment for diabetes and that it was dropping another drug for sepsis.

The reality is that only one in every 10,000 compounds will ever make it to the market. The rationale for such risky behaviour? If you get it right, the rewards are phenomenal. One analyst predicted that Torcetrapib could have been worth $20bn if it had worked.

Unfortunately for Big Pharma, market dynamics are changing and to such a degree that experts believe this economic model is simply no longer sustainable.

"I would suggest that every major pharmaceutical company is rethinking their strategy and starting to wonder if it would be more advantageous to invest in developing five drugs that have potential annual revenues of $500 million to $1bn rather than putting all their money into trying to develop the $5bn-a-year-drug," says Kenneth Kaitin, director of the Tufts Centre for the Study of Drug Development in Boston, Massachusetts.

"To put all your eggs in one basket is very risky and I don't think the stock market is going to reward companies that approach things in that fashion."

In the past, Big Pharma has made quite a good living from throwing huge piles of cash at developing drugs that target conditions that affect large numbers of people, such as cancer and heart disease. But the window of opportunity to make their money back is now shrinking. Companies are champing at the bit to produce generic versions of blockbusters as soon as the key parts of patents expire. At the same time, the cost of developing a blockbuster drug has escalated from $300m to $1.35bn in just the last 10 years.

The reasons for this are numerous. Pharmaceutical companies are focusing on diseases that are extremely complex - such as cancer, schizophrenia and manic depression - as well as ones that require the longer term use of drugs, such as rheumatoid arthritis. These indications require much larger and sometimes longer clinical trials. At the same time, Kaitin explains, regulators are becoming more demanding due to high-profile side-effect cases, and US health plans are asking for more proof of the economic and therapeutic advantages of one drug over another - again causing the need for more in-depth studies. Even after regulatory approval, drugs must be monitored in thousands of patients for unforeseen side effects.

Pfizer had 25,000 patients from North America, Europe and Australia enrolled in its phase three trials of Torcetrapib, many of which had been taking the drug for 18 to 20 months.

With such efforts being made, it seems unfathomable that a drug can get so far down the development process and then fail at the final human trials without any warning signs along the way.

Even Pfizer is at a loss to explain its recent blow. "By the time you get to phase three, you hope the product will make the finish line," says a Pfizer spokesman. "It was a complete surprise that we received these results. All you can do is go on the information you have. We have an independent safety monitoring board looking at the numbers on a monthly basis. At the end of October, we were told everything was fine. Then the review at the end of November showed this imbalance in the number of deaths on the combination drug versus the patients on Lipitor."

The problem that arises in the drug discovery process is that it is very difficult to predict how a drug will work in a human until very late in development.

Animal studies can only give some indication of safety. The initial phases of human trials are purposely kept small to minimise the damage if something goes wrong. But some drug side effects may only occur in one in 1000 people, and therefore only show up in the final large scale human trials.

Given the escalating costs of drug development, Big Pharma are beginning to realise that less has to be left up to serendipity.

Tom Shepherd, chief executive of drug discovery firm CXR Biosciences, says that pricey failures such as the one experienced by Pfizer have prompted the pharmaceutical industry to look at ways to predict human response earlier in the process. When scientists cracked the human genome, the industry initially focused its attention on using the information to find new ideas for drugs and treatments. But, in the past two years, researchers have also been looking at how this vital data could aid in establishing whether or not drugs are safe.

"The problem we're now beginning to understand more is that the variations in people make it difficult to have one standard way of administering a drug to the public," he explains. "We now know that some people are genetically predisposed to metabolise drugs faster or slower than the population as a whole and that this could be a reason why some people have side effects and others don't."

One way around this predicament would be to genotype patients before administering a drug. But these tests are costly and not widely available.

CXR Biosciences, based in Dundee, is working on another way to tackle the problem: it is developing tests using human genes that can detect whether individual compounds are safe for trialling. The company screens drug molecules on behalf of biotechnology and pharmaceutical companies using laboratory tests that contain human cells. But it is also creating mice that contain human enzymes to try to mimic how a person breaks down a drug.

CXR Biosciences is already working with Schering-Plough and has licensed one of its technologies to Pfizer. Their "Pfizer mouse" is missing a drug metabolism function in the liver. "It allows them to see what the drug itself does independent of any metabolism. If the drug itself is fine, then they know metabolism is affecting the process," Shepherd explains.

CXR made £2.8m in revenues from its work in 2005, a 60% increase on the year before. But it is still small potatoes, considering the vast budgets of global heavyweights such as AstraZeneca and GlaxoSmithKline.

"There's a terrific interest in this whole area," Shepherd insists. "The reason why we're not selling hundreds of millions of pounds worth of tests is because all these new technologies take a long time to prove.

"Once we have been working with companies for a while and they have put drugs through the models and have seen the benefits, then that is when we will see an increase in companies using it."

In the meantime, experts believe there are plenty of other ways to cut risk, cost and time out of a creaking system.

"You're looking at a development process that can take eight to 11 years. I know there are major pharmaceutical companies looking to halve that time. The fact they are trying to halve that time suggests that there is clearly a lot of waste in that process," says Harry Clark, chief executive of SI Associates, a Glasgow consultancy that advises five of the top 10 global pharmaceutical powerhouses on how to streamline their drug discovery process. "Examples of this are the fact that a lot of acts are repeated and repeated. Projects which in any other industry would be killed off also continue to attract investment well beyond the point that they should."

He continues: "Because of the historic emphasis on science and research in development, there has been less emphasis on business execution, things like the supply chain, delivering the proper quantities of materials to clinical trial sites on time or processing information in a more efficient manner. These things are not being done as well as they should be."

Kaitin adds that many pharmaceutical firms have failed to really capitalise on the cost-savings of outsourcing. While they may outsource clinical trials or research on a particular project, many pharmaceutical companies retain the ability to do everything in-house if required.

That could change in the future, according to Shepherd.

"The core competence that you need a big structure to do is market drugs globally. Pharmaceutical companies will still be large entities but will concentrate on global development and marketing and the rest will be left to the smaller biotechnology companies who are more creative," he predicts.

"What we might end up with is a lot of small companies doing early-stage research and then partnering or licensing the products out to Big Pharma to bring them to market."

One beneficiary of that strategy is Scottish specialty pharmaceutical firm ProStrakan. The Galashiels company already has an exclusive collaboration with Novartis to research, develop and commercialise antibodies for bone-related diseases.

ProStrakan provided the early-stage research and will continue to work on the drugs, but Novartis will shoulder the risk going forward by paying all the costs associated with its future stages of development and marketing. ProStrakan could receive more than $140m in milestone payments if all goes well, on top of any royalties from sales.

Wilson Totten, chief executive of ProStrakan, says that these deals allow smaller companies to follow a much more sustainable business model. He is using these milestone payments to buy the rights to two other drugs to sell in America that are closer to market and therefore lower risk. ProStrakan has been buying the rights to existing drugs that can be reformulated or improved and therefore require less extensive patient trials. Its share price jumped following news last week of positive phase three results for its patch version of a drug that treats chemotherapy-induced nausea.

However, Totten argues that large pharmaceutical companies have an important role to play in the healthcare system. "The people who should strive to find blockbusters are companies like Pfizer that have billions of dollars to spend."

Ultimately, the changing landscape does not spell the end of Big Pharma or blockbuster drugs. Without these giants and their deep pockets, breakthrough treatments would be impossible.

But it does mean that they will have to be far smarter about how they do business by either spreading the risk by creating drugs for therapeutic areas that are less competitive and less expensive, or slashing overheads by outsourcing or whipping their drug-discovery processes into shape.

As Clark puts it: "I don't think things will continue in the same way. Some of these companies may not survive with their current business model."