Is Surgery An Anachronism In An Evidence Based Age

Is surgery an anachronism in an

evidence based age?

Richard SmithEditor, BMJ

www.bmj.com/talks

What I want to talk about

• “The history of surgery: my contribution”

• Surgery: historical and literary reflections

• The state of evidence in surgery• Improving the evidence base

“The history of surgery: my contribution”

My role as emotional lightning conductor --hanging onto the second retractor

• “Married are you, Richard?”• “No.”• “Good idea. The only point I can see in being married is that it

saves you having to find somebody to go on holiday with.”• “Do you have horses?”• “No. I live in a flat.”• “Good chap. Super weather, isn’t it?”• And so it went until he cut something he shouldn’t have.

“Bloody hell. Will you pull on that retractor, Smith? You are a bloody fool. Don’t you know anything about surgery? For God’s sake, pull harder.”

My vasectomy• Twenty years later I had a similar conversation, only this time I was under

the knife. I was having a vasectomy. The surgeon, who did 20 vasectomies a day five days a week, had already cut my right vas when he asked me what I did.

• “I’m a sort of journalist.”• “What sort?”• “A medical journalist.”• “Oh. Where do you work?”• “At the BMJ.”• “What exactly do you do there?”• “I’m the editor.”• Suddenly the surgeon began to sweat. It had taken him two minutes to find

and cut one vas. He now took 20 minutes to find the second. “Oh God,” he said, “I suppose I’m going to read about this in next week’s journal.”

Surgery: historical and literary reflections

Hemicorporectomy

• Hemicorporectomy or translumbar amputation is probably the most mutilating operation ever to be described in surgical literature.

• Treatment for pelvic malignancy

• The procedure involves removal of the bony pelvis, both lower limbs, the external genitalia, the bladder, rectum and anus.

• First proposed in 1951 and performed in 1960• Two of the first three patients died within days of

oedema; one survived 19 years• A series of 10 cases reported from New York in 1982

Nemesis

• Nemesis is the goddess of divine justice and vengeance. Her anger is directed toward human transgression of the natural, right order of things and of the arrogance causing it. Nemesis pursues the insolent and the wicked with inflexible vengeance.

Sir William Arbuthnot Lane (1853-1946)

• Performed total colectomies for the treatment of “auto-intoxication,” something like chronic fatigue syndrome

• Operated on many of London’s fashionable set

• Operation had a 10% mortality• Dreadful side effects• The placebo effect guarantees that some will

be “cured”

Ironic quote from Arbuthnot Lane

•"If everyone believes a thing it is probably untrue!"

• Quoted by W. E. Tanner in Sir W. Arbuthnot Lane, "Genesis"

Monologue by Sir Patrick Ridgeon

• I tell you, Colly, chloroform has done a lot of mischief. It's enabled every fool to be a surgeon. I know your Cutler Walpoles and their like. They've found out that a man's body is full of bits and scraps of old organs he has no mortal use for. Thanks to chloroform, you can cut half a dozen of them out without leaving him any the worse, except for the illness and the guineas it costs him. I knew the Walpoles fifteen years ago. The father used to snip off the ends of people's uvulas for fifty guineas, and paint throats with caustic every day for a year at two guineas a time. His brother-in-law extirpated tonsils for two hundred guineas until he took up women's cases at double the fees. Cutler himself worked hard at anatomy to find something fresh to operate on; and at last he got hold of something he calls the nuciform sac, which he's made quite the fashion. People pay him five hundred guineas to cut it out. They might as well get their hair cut for all the difference it makes; but I suppose they feel important after it. You can't go out to dinner now without your neighbor bragging to you of some useless operation or other.

Cutler Walpole’s Nuciform Sac

• Cutler Walpole made his fortune from removing the nuciform sac from the fashionable in London

• “And have you had your nuciform sac removed, Mr Walpole.”

• “I’m one of the lucky and unusual people who was born without one.”

GBS on operations

• The large range of operations which consist of amputating limbs and extirpating organs admits of no direct verification of their necessity. There is a fashion in operations as there is in sleeves and skirts. ... There are men and women whom the operating table seems to fascinate: half-alive people who through vanity, or hypochondria, or a craving to be the constant objects of anxious attention or what not, lose such feeble sense as they ever had of the value of their own organs and limbs.

GBS on doctors sticking together

• The only evidence that can decide a case of malpractice is expert evidence: that is, the evidence of other doctors; and every doctor will allow a colleague to decimate a whole countryside sooner that violate the bond of professional etiquette by giving him away.

GBS on EBM

• It does happen exceptionally that a practising doctor makes a contribution to science; but it happens much oftener that he draws disastrous conclusions from his clinical experience because he has no conception of scientific method, and believes, like any rustic, that the handling of evidence and statistics needs no expertness.

John Rowan Wilson on surgeons

• Old Fred had always tended to discount the surgeons. He visualised them as a pack of blood and thunder, cut and thrust, bombastic dunderheads, too lacking in intelligence to constitute a danger to his own system. In his eyes they were the cavalry officers of medicine, dashing and romantic, useful to impress the simple hearted, but totally unimportant when there was any serious business to be done. He had outwitted the surgeons all his life and regarded them with scant respect.

The state of evidence in

surgery

Useless operations

• Ligation of the internal mammary artery for angina

• Extracranial/intracranial artery connection

• Radical mastectomy• Sympathectomy for peripheral

vascular disease• Spinal fusion

State of the evidence

• “Unless assessments of surgical procedures are seen to be unbiased, properly randomised, and with objective assessment of outcomes they will continue to lack credibility.”

• A G Johnson and J Michael DixonRemoving bias in surgical trialsBMJ, Mar 1997; 314: 916.

Large, randomised trials are needed

• Benefits are often small--so large trials are needed so as not to miss small but still important effects. Large trials are also better at detecting harms, which are often rare.

• Randomisation is needed to avoid the bias that is all pervasive

• The higher the quality of the trial the smaller the beneficial effect

The safety and efficacy of new interventional

procedures:published evidence and

clinical guidance

Tom Dent, Sally Wortley, Bruce Campbell

NICE review of evidence: methods

• 245 procedures• 84 are in the Safety and Efficacy Register of

New Interventional Procedures (SERNIP)• Authors randomly selected 25 procedures

considered “safe and efficacious enough for routine use” and 25 of uncertain safety and efficacy

• Two authors independently reviewed the evidence

NICE review of evidence: results

• 262 studies (an average of 5.2 per procedure)

• 178 (68%) were case series • 43 (16%) non-randomised

comparisons• 33 (13%) randomised trials • 8 (3%) systematic reviews

NICE review of evidence: results

• A fifth of studies supporting “safe and efficacious” procedures by randomised trials--in contrast to 3% of the studies supporting uncertain procedures

• Median number of patients in the studies supporting each procedure 583 (range 3 to 2873)

• Follow up poorly reported

NICE review of evidence: conclusions

• “Many interventional procedures enter clinical practice with limited published evaluation of their safety and efficacy”

• “The nature of the available studies means that the validity and durability of benefit often cannot be adequately assessed and important risks cannot be excluded”

State of the evidence

• Few systematic reviews• Few randomised trials• Those that exist are often of poor

quality--which is true of all trials• Very few placebo controlled trials• Too many case series, which are

ultimately uninterpretable

A snapshot of the evidence

• RCTs declined from 14% of research articles in the British Journal of Surgery in 1985 to 5% in 1992

• Treatments in general surgery are half as likely to be based on RCT evidence as treatments in internal medicine

• 75% of “surgical trials” are actually of medical treatments in surgical patients

• Only a third of surgical trials have adequate blinding• In a study of 10 international journals from 1988 to 1994,

Hall et al found that, of the few randomised controlled trials that were published, less than half included objective methods for assessing outcome

Article in two issues of the British Journal of Surgery

• Case series 29(51%)• Animal studies 7 (12%)• Non-systematic reviews 6 (11%)• RCTs 4 (7%)• Audit 3 (5%)• Four other designs 8 (14%)

Analysis of general practice studies from the BMJ

• Questionnaire survey 6 (27%)• RCT 5 (23%)• Database study 2 (9%)• Case control study 2 (9%)• Five other designs 7 (42%)

The evidence patients and doctors need on a new treatment

• A large, well conducted RCT comparing the new treatment with established evidence based treatments

• To be “evidence based” a treatment will at some time have had to be tested against placebo

• An RCT against placebo only if no evidence based treatment exists

A placebo controlled trial of athroscopy for osteoarthritis of

the knee• 185 patients randomise to arthroscopic débridement,

arthroscopic lavage, or placebo surgery • Patients in the placebo group received skin incisions and

underwent a simulated débridement without insertion of the arthroscope

• Blind assessment of outcome• Followed up for two years• Several outcome measures• 165 patients completed the trial• Moseley JB, O'Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A

controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002; 347: 81-88[

Moseley, J. B. et al. N Engl J Med 2002;347:81-88

Mean Values (and 95 Percent Confidence Intervals) on the Knee-Specific Pain Scale

Causes of lack of evidence and solutions

• Much of what I have to say comes from the following article

• Randomised trials in surgery: problems and possible solutions. – Peter McCulloch, Irving Taylor, Mitsuru

Sasako, Bryony Lovett, and Damian GriffinBMJ 2002 324: 1448-1451.


• History: Many operations developed before RCTs were expected

• Solution: Comprehensively review surgical evidence. Be clear what we know and don’t know.


• Commercial competition and personal prestige: many financial drivers in surgery

• Hard to test objectively “your operation”

• “Surgeons' eagerness to learn the operation [laparascopic cholecystectomy] seemed related more to commercial concerns than to concern for patients.”


• Solution: Move towards constant collection of data on outcomes of surgery and use statistical systems of continuous quality control to accumulate evidence of effectiveness and harms and decide when RCTs are necessary


• Surgeon’s equipoise: Surgeons are (attractively) decisive and used to acting in uncertain circumstances. They may thus be less comfortable with accepting the uncertainty and recognising that they have the “equipoise” necessary for a trial.

• Solution: include parallel, non-randomised, preference arms alongside RCTs.


• Lack of funding, infrastructure, and experience of data collection

• Solution: change to a culture of cooperation rather than competition. Form large groups to perform specific trials


• Lack of education in clinical epidemiology. Surgeons' knowledge of clinical epidemiology remains poor

• Solution: Include training in routine surgical training. (Clinical epidemiology is the basic science of clinical research.) More advanced training for some.


• The learning curve: Should RCTs be done on new treatments or once surgeons have learnt to do them? Obvious dangers in comparing a new with a familiar operation. Solution: statistical techniques can help. Determine whether the top of the curve has been reached before analysing data.


•The quality of the surgery: It must be similar for all interventions being tested

• Solution: quality control must be part of the trial. Video evidence might be collected.


• Development versus research: much innovation comes from small sequential changes not large leaps. When to do an RCT?

• Solution: Use statistical quality improvement techniques. Ask independent third parties to decide when an RCT is necessary


• Patient equipoise: This equipoise is really more important than the surgeon’s equipoise--and allocation of treatment by chance is especially hard when it’s a medical versus a surgical treatment

• Solution: Not easy. Decision analysis techniques may help.


• Blinding: clearly very difficult in surgical trials

• Solution: Blinded observers should be used routinely for assessing outcome

High quality databases can supplement or even provide an

alternative to RCTs• Must include individual data on all consecutive

cases

• Must use standard definitions of conditions and outcomes

• Must ensure data are complete and accurate

• Must include data on all known patient

characteristics that affect outcome

• Users must know how to risk adjust

NICE (National Institute of Clinical Excellence) on interventions

• All new interventions must be notified--by government decree; always doubt on what is a new procedure

• IPAC (Interventional Procedures Advisory Committee) prepares an overview (not a systematic review) of “the literature”

• IPAC also consults specialist advisers--people “held in high regard by their colleagues; they will not necessarily be enthusiasts, with the bias which can accompany such enthusiasm”

NICE on interventions

• IPAC rules– “Safe and efficacious”– Uncertainty– Unsafe

• Uncertain procedures can be undertaken “judiciously” but patients must be informed of the uncertainty; data must be gathered

• All subject to consultation• Guidance is published• Existing procedures have been reviewed

Comments

• This is progress• Or is it bureaucracy?• It looks like a compromise. Is this a

“practical, sensible way forward” or a “cop out”? Such a system wouldn’t be acceptable for drugs

• Can surgery ever be more evidence based?

Conclusions

• Surgery has been far from evidence based, which has led to some horrible excesses

• The state of evidence in surgery is poor• It’s much harder to do RCTs in surgery

than in medicine• But the problems can be overcome--and

should be

Conclusions

• High quality databases should be very helpful

• Various countries--including Australia and Britain--are developing ways of regulating new interventions

• These are very much at the beginning and have a long way to develop

Final thoughts

A good surgeon

•“Good surgeons know how to operate, better surgeons when to operate, and the best when not to operate.”

Is there anybody less evidence based than

surgeons?

YesPoliticiansManagers

Editors

Is Surgery An Anachronism In An Evidence Based Age

Health & Medicine

contribution surgery

history of surgery

state of evidence

expert evidence

nemesis nemesis

sir william arbuthnot

literary reflections

extirpating organs