Hospital Websites Evaluation

Systematic Review of Hospital WebsitesValid Medical Resource or Just Another Marketing Tool?Barry Little, M.D.

Systematic Review ofHospital WebsitesObjectivesMethodsResultsDiscussionSummary

ObjectivesDetermine whether regional hospital websites provide viable health & medical information Critique usability, content and quality of regional hospital websitesDiscuss the future of hospital websites

Methodology: Content &UsabilityIdentification of sites via HospitalWebUSAInitially #55 sites found in KentuckyDuplication of sites/website templates, consolidated into one siteOne hospital without a websiteTotal of #39 sites reviewed

Review was based on two main components: content and usability

Methodology: Content Evaluation Checklist (Y/N)Patient educational materialInteractiveLinks to educational material/referencesMaintenanceCertification

Methodology: Usability Score (0-5)5 highest, 0 lowestMinus 1 point for each of the following:Third party advertisingDistracting graphics/sound/animationPoor readability (fonts too small, illegible)Poor layout (too busy, color mismatches)Technical (slow to load, dead links, navigability)

Results: Educational MaterialNo distinction made between level or quality of material

54% (21) of sites did not offer any educational material within site

Websites that did contain material was often commercially created material (canned)

Chart1

2118

NO

YES

Educational Material

Sheet1

HospitalEducational MaterialInteractiveLinksMaintenanceCertificationUsability Score*

1. Norton Healthcare - Louisville, KentuckyYYYYN5*Usability Scale (1-5)

2. Appalachian Regional Healthcare - Lexington, KentuckyYNNYN4Advertisements (other than services offered)-1

3. ARH Regional Medical Center (Hazard) - Hazard, KentuckyYNYYN4Distracting graphics/animations/sound-1

4. Baptist Hospital East - Louisville, KentuckyYNYYN5Poor Readability (fonts too small, illegible)-1

5. Baptist Regional Medical Center - Corbin, KentuckySEE MAIN SITEPoor Layout (color mismatches, too busy)-1

6. Central Baptist Hosptital - Lexington, KentuckySEE MAIN SITETechnical (slow to load, dead links)-1

7. Cardinal Hill Rehabilitation Hospital - Lexington, KentuckyNNNYN5

8. Clark Regional Medical Center - Winchester, KentuckyNNNYN4

9. Ephraim McDowell Regional Medical Center - Danville, KentuckyYNYYN4

10. Fleming County Hospital - Flemingsburg, KentuckyNNYYN3

11. Frankfort Regional Medical Center - Frankfort, KentuckyYNYNN5

12. Greenview Hospital - Bowling Green, KentuckyYNNYN4

13. Harlan ARH Hospital - Harlan, KentuckySEE MAIN SITE

14. Highlands Regional Medical Center - Prestonsburg, KentuckyNYYYN5

15. Jackson Purchase Medical Center - Mayfield, KentuckyNNNYN4

16. Jennie Stuart Medical Center - Hopkinsville, KentuckyNNNNN3

17. Jewish Hospital HealthCare Services - Louisville, KentuckyNNYYN4

18. Kentucky River Medical Center - Jackson, KentuckyYNNYN3

19. King's Daughters' Medical Center - Ashland, KentuckyYNNYN4

20. Kosair Children's Hospital - Louisville, KentuckySEE MAIN SITE

21. Lake Cumberland Regional Hospital - Somerset, KentuckyYNYYN3

22. Lourdes Hospital - Paducah, KentuckyYNNYY3

23. Marymount Medical Center - London, KentuckyNNNNN3

24. McDowell ARH Hospital - McDowell, KentuckySEE MAIN SITE

25. Methodist Hospital - Henderson, KentuckyNNNYN4

26. Paul B. Hall Regional Medical Center - Paintsville, KentuckyNNYNN3

27. The Medical Center - Bowling Green, KentuckyNNNYN3

28. The Medical Center (Franklin) - Franklin, KentuckyNNNYN5

29. Middlesboro ARH Hospital - Middlesboro, KentuckySEE MAIN SITE

30. Morgan County ARH Hospital - West Liberty, KentuckySEE MAIN SITE

31. Monroe County Medical Center - Tompkinsville, KentuckyNNNYN5

33. Ohio County Hospital - Hartford, KentuckyNNYYN4Narcotic policy

34. Our Lady of the Way Hospital - Martin, KentuckyNNYYN3

35. Pikeville Methodist Hospital - Pikeville, KentuckyNNNYN3

36. Trover Regional Medical - Madisonville, KentuckyYNNYN4

37. Samaritan Hospital - Lexington, KentuckyYYYYN3links to advertisers

38. Shriners Hospital - Lexington, KentuckyNNNYN3

39. St. Claire Medical Center - Morehead, KentuckyNNNNN2

40. St. Joseph's Hospital - Lexington, KentuckyYYYYN3Specifically listed "Consumer Health Resources"

41. St. Elizabeth Medical Center North - Covington, KentuckyYYYYN3

42. St. Elizabeth Medical Center South - Edgewood, KentuckySEE MAIN SITE

43. St. Elizabeth Medical Center Grant County - Williamstown, KentuckySEE MAIN SITE

44. Suburban - Louisville, KentuckyYNNYN4

45. T.J. Samson Community Hospital - Glasgow, KentuckyYNYYN4

46. Tri-County Baptist Hospital - La Grange, KentuckySEE MAIN SITE

47. UofL Health Care - Louisville, KentuckyNNNYN4

48. University of Kentucky Medical Center - Lexington, KentuckyYYYYN5ONLINE APPOINTMENTS

49. Wayne County Hospital - Monticello, KentuckyUnable to locate website0

50. Western Baptist Hospital - Paducah, KentuckySEE MAIN SITE

51. Westlake Regional Hospital - Columbia, KentuckyNNNNN2

52. Whitesburg ARH Hospital - Whitesburg, KentuckySEE MAIN SITE

53. Williamson ARH Hospital - South Williamson, KentuckySEE MAIN SITE

54. Clark Regional - WinchesterNYNYN5

14

1. Norton Healthcare - Louisville, Kentucky

2. Appalachian Regional Healthcare - Lexington, Kentucky

3. ARH Regional Medical Center (Hazard) - Hazard, Kentucky

4. Baptist Hospital East - Louisville, Kentucky

5. Baptist Regional Medical Center - Corbin, Kentucky

6. Central Baptist Hosptital - Lexington, Kentucky

7. Cardinal Hill Rehabilitation Hospital - Lexington, Kentucky

8. Clark Regional Medical Center - Winchester, Kentucky

9. Ephraim McDowell Regional Medical Center - Danville, Kentucky

10. Fleming County Hospital - Flemingsburg, Kentucky

11. Frankfort Regional Medical Center - Frankfort, Kentucky

12. Greenview Hospital - Bowling Green, Kentucky

13. Harlan ARH Hospital - Harlan, Kentucky

14. Highlands Regional Medical Center - Prestonsburg, Kentucky

15. Jackson Purchase Medical Center - Mayfield, Kentucky

16. Jennie Stuart Medical Center - Hopkinsville, Kentucky

17. Jewish Hospital HealthCare Services - Louisville, Kentucky

18. Kentucky River Medical Center - Jackson, Kentucky

19. King's Daughters' Medical Center - Ashland, Kentucky

20. Kosair Children's Hospital - Louisville, Kentucky

21. Lake Cumberland Regional Hospital - Somerset, Kentucky

22. Lourdes Hospital - Paducah, Kentucky

23. Marymount Medical Center - London, Kentucky

24. McDowell ARH Hospital - McDowell, Kentucky

25. Methodist Hospital - Henderson, Kentucky

26. Paul B. Hall Regional Medical Center - Paintsville, Kentucky

27. The Medical Center - Bowling Green, Kentucky

28. The Medical Center (Franklin) - Franklin, Kentucky

29. Middlesboro ARH Hospital - Middlesboro, Kentucky

30. Morgan County ARH Hospital - West Liberty, Kentucky

31. Monroe County Medical Center - Tompkinsville, Kentucky

33. Ohio County Hospital - Hartford, Kentucky

34. Our Lady of the Way Hospital - Martin, Kentucky

35. Pikeville Methodist Hospital - Pikeville, Kentucky

36. Trover Regional Medical - Madisonville, Kentucky

37. Samaritan Hospital - Lexington, Kentucky

38. Shriners Hospital - Lexington, Kentucky

39. St. Claire Medical Center - Morehead, Kentucky

40. St. Joseph's Hospital - Lexington, Kentucky

41. St. Elizabeth Medical Center North - Covington, Kentucky

42. St. Elizabeth Medical Center South - Edgewood, Kentucky

43. St. Elizabeth Medical Center Grant County - Williamstown, Kentucky

44. Suburban - Louisville, Kentucky

45. T.J. Samson Community Hospital - Glasgow, Kentucky

46. Tri-County Baptist Hospital - La Grange, Kentucky

47. UofL Health Care - Louisville, Kentucky

48. University of Kentucky Medical Center - Lexington, Kentucky

49. Wayne County Hospital - Monticello, Kentucky

50. Western Baptist Hospital - Paducah, Kentucky

51. Westlake Regional Hospital - Columbia, Kentucky

52. Whitesburg ARH Hospital - Whitesburg, Kentucky

53. Williamson ARH Hospital - South Williamson, Kentucky

54. Clark Regional - Winchester

Sheet2


7. Cardinal Hill Rehabilitation Hospital - Lexington, KentuckyNNNYN5*Usability Scale (1-5)

8. Clark Regional Medical Center - Winchester, KentuckyNNNYN4Advertisements (other than services offered)-1

10. Fleming County Hospital - Flemingsburg, KentuckyNNYYN3Distracting graphics/animations/sound-1

14. Highlands Regional Medical Center - Prestonsburg, KentuckyNYYYN5ONLINE APPOINTMENTSPoor Readability (fonts too small, illegible)-1

15. Jackson Purchase Medical Center - Mayfield, KentuckyNNNYN4Poor Layout (color mismatches, too busy)-1

16. Jennie Stuart Medical Center - Hopkinsville, KentuckyNNNNN3Technical (slow to load, dead links)-1








33. Ohio County Hospital - Hartford, KentuckyNNYYN4








1. Norton Healthcare - Louisville, KentuckyYYYYN5

2. Appalachian Regional Healthcare - Lexington, KentuckyYNNYN4

3. ARH Regional Medical Center (Hazard) - Hazard, KentuckyYNYYN4

4. Baptist Hospital East - Louisville, KentuckyYNYYN5


11. Frankfort Regional Medical Center - Frankfort, KentuckyYNYNN5Narcotic policy




21. Lake Cumberland Regional Hospital - Somerset, KentuckyYNYYN3links to advertisers



37. Samaritan Hospital - Lexington, KentuckyYYYYN3

40. St. Joseph's Hospital - Lexington, KentuckyYYYYN3




48. University of Kentucky Medical Center - Lexington, KentuckyYYYYN5

1








































Sheet3

Educational MaterialEducational MaterialInteractiveLinksMaintenanceCertificationUsability Score*

NNO21NNYN5

NYES18NNYN4

NNYYN3

NYYYN5

NNNYN4

NNNNN3

NNYYN4

NNNNN3

NNNYN4

NNYNN3

NNNYN3

NNNYN5

NNNYN5

NNYYN4

NNYYN3

NNNYN3

NNNYN3

NNNNN2

NNNYN4

NNNNN2

NYNYN5

YYYYN5

YNNYN4

YNYYN4

YNYYN5

YNYYN4

YNYNN5

YNNYN4

YNNYN3

YNNYN4

YNYYN3

YNNYY3

YNNYN4

YYYYN3

YYYYN3

YYYYN3

YNNYN4

YNYYN4

YYYYN5

Sheet3

NO

YES


Results: Interactive

Criteria: real time interactivity - such as online scheduling, self registration, self diagnostic tools, bill payment

Vast majority of sites (82%) lacked any interactivity

Primary interactive items were financial in nature bill payment, bill review, etc.

Chart2

327

NO

YES

Interactive Objects

Sheet1























































14






















































Sheet2




10. Fleming County Hospital - Flemingsburg, KentuckyNNYYN3Distracting graphics/animations/sound-1

15. Jackson Purchase Medical Center - Mayfield, KentuckyNNNYN4ONLINE APPOINTMENTSPoor Readability (fonts too small, illegible)-1

16. Jennie Stuart Medical Center - Hopkinsville, KentuckyNNNNN3Poor Layout (color mismatches, too busy)-1

17. Jewish Hospital HealthCare Services - Louisville, KentuckyNNYYN4Technical (slow to load, dead links)-1







33. Ohio County Hospital - Hartford, KentuckyNNYYN4








3. ARH Regional Medical Center (Hazard) - Hazard, KentuckyYNYYN4

4. Baptist Hospital East - Louisville, KentuckyYNYYN5





19. King's Daughters' Medical Center - Ashland, KentuckyYNNYN4Narcotic policy




44. Suburban - Louisville, KentuckyYNNYN4links to advertisers









1








































Sheet3


NNO21NNO32NYN5

NYES18NYES7NYN4

NNYYN3

NYYYN5

NNNYN4

NNNNN3

NNYYN4

NNNNN3

NNNYN4

NNYNN3

NNNYN3

NNNYN5

NNNYN5

NNYYN4

NNYYN3

NNNYN3

NNNYN3

NNNNN2

NNNYN4

NNNNN2

NYNYN5

YYYYN5

YNNYN4

YNYYN4

YNYYN5

YNYYN4

YNYNN5

YNNYN4

YNNYN3

YNNYN4

YNYYN3

YNNYY3

YNNYN4

YYYYN3

YYYYN3

YYYYN3

YNNYN4

YNYYN4

YYYYN5

Sheet3

NO

YES


NO

YES

Interactive Objects

Results: External LinksPrimary criterion was presence of links to recognized medical authorities (CDC, WHO, Mayo Clinic, PubMed)

56% of sites did not offer external links to educational material, support groups, search engines, etc

38% of sites had neither online or external links to educational material

Chart5

2217

NO

YES

External Links

Sheet1























































14






















































Sheet2

HospitalEducational MaterialLinksInteractiveMaintenanceCertificationUsability Score*



15. Jackson Purchase Medical Center - Mayfield, KentuckyNNNYN4Distracting graphics/animations/sound-1

16. Jennie Stuart Medical Center - Hopkinsville, KentuckyNNNNN3ONLINE APPOINTMENTSPoor Readability (fonts too small, illegible)-1

23. Marymount Medical Center - London, KentuckyNNNNN3Poor Layout (color mismatches, too busy)-1

25. Methodist Hospital - Henderson, KentuckyNNNYN4Technical (slow to load, dead links)-1









54. Clark Regional - WinchesterNNYYN5

10. Fleming County Hospital - Flemingsburg, KentuckyNYNYN3

17. Jewish Hospital HealthCare Services - Louisville, KentuckyNYNYN4

26. Paul B. Hall Regional Medical Center - Paintsville, KentuckyNYNNN3

33. Ohio County Hospital - Hartford, KentuckyNYNYN4

34. Our Lady of the Way Hospital - Martin, KentuckyNYNYN3







36. Trover Regional Medical - Madisonville, KentuckyYNNYN4Narcotic policy


3. ARH Regional Medical Center (Hazard) - Hazard, KentuckyYYNYN4

4. Baptist Hospital East - Louisville, KentuckyYYNYN5

9. Ephraim McDowell Regional Medical Center - Danville, KentuckyYYNYN4links to advertisers

11. Frankfort Regional Medical Center - Frankfort, KentuckyYYNNN5

21. Lake Cumberland Regional Hospital - Somerset, KentuckyYYNYN3

45. T.J. Samson Community Hospital - Glasgow, KentuckyYYNYN4






1








































Sheet3


NNO21NNO32NNO22YNO6N5

NYES18NYES7NYES17YYES33N4

NNYYN3

NYYYN5

NNNYN4

NNNNN3

NNYYN4

NNNNN3

NNNYN4

NNYNN3

NNNYN3

NNNYN5

NNNYN5

NNYYN4

NNYYN3

NNNYN3

NNNYN3

NNNNN2

NNNYN4

NNNNN2

NYNYN5

YYYYN5

YNNYN4

YNYYN4

YNYYN5

YNYYN4

YNYNN5

YNNYN4

YNNYN3

YNNYN4

YNYYN3

YNNYY3

YNNYN4

YYYYN3

YYYYN3

YYYYN3

YNNYN4

YNYYN4

YYYYN5

Sheet3

NO

YES


NO

YES

Interactive Objects

NO

YES

Maintenance

NO

YES

External Links

Results: MaintenanceMajority of sites (85%) were well maintained

Disparity

Criteria: Primary - actual date of last update listed (uncommon)Secondary all links functional, dates for upcoming or recent events accurate, no glaring outdated information

Chart6

633

NO

YES

Maintenance

Sheet1























































14






















































Sheet2









































1








































Sheet3




NNYYN3

NYYYN5

NNNYN4

NNNNN3

NNYYN4

NNNNN3

NNNYN4

NNYNN3

NNNYN3

NNNYN5

NNNYN5

NNYYN4

NNYYN3

NNNYN3

NNNYN3

NNNNN2

NNNYN4

NNNNN2

NYNYN5

YYYYN5

YNNYN4

YNYYN4

YNYYN5

YNYYN4

YNYNN5

YNNYN4

YNNYN3

YNNYN4

YNYYN3

YNNYY3

YNNYN4

YYYYN3

YYYYN3

YYYYN3

YNNYN4

YNYYN4

YYYYN5

Sheet3

NO

YES


NO

YES

Interactive Objects

NO

YES

External Links

NO

YES

Maintenance

Results: Certification - vs Merit BadgesOnly one hospital (2.5%) listed any Internet certification (HON)

Several sites advertised JCAHO credentialing or other awards (US News Top 100, JD Powers, Solucient, etc)

Results: Usability ScoreMost sites were moderate-high quality in terms of usabilityCommon problems were poor layout/readability, lack of maintenance, distracting graphics

Chart1

0

2

14

14

9

# Websites

Score

# Website

Usability Scores

Sheet1























































14






















































Sheet2









































1








































Sheet3

Educational MaterialEducational MaterialInteractiveLinksMaintenanceCertificationUsability Score*Score#



NNYYN3143

NYYYN522

NNNYN401

NNNNN3

NNYYN410

NNNNN322

NNNYN4314

NNYNN3414

NNNYN359

NNNYN5

NNNYN5

NNYYN4

NNYYN3

NNNYN3

NNNYN3

NNNNN2

NNNYN4

NNNNN2

NYNYN5

YYYYN5

YNNYN4

YNYYN4

YNYYN5

YNYYN4

YNYNN5

YNNYN4

YNNYN3

YNNYN4

YNYYN3

YNNYY3

YNNYN4

YYYYN3

YYYYN3

YYYYN3

YNNYN4

YNYYN4

YYYYN5

Sheet3

NO

YES


NO

YES

Interactive Objects

NO

YES

External Links

NO

YES

Maintenance

# Websites

Score

# Website

Usability Scores

Results: Incidental FindingsOdditiesRecipesWeather reportsNotice of narcotic restrictionsNiceties Medication record (downloadable)Personalized health recordCarePage patients can create their own website, exchange email/post messages, keep family/friends updated on condition (site is certified by TRUSTe)Provider links to decision support

DiscussionWhat is the actual business model for hospital websites?Most function as static electronic brochuresAdvertising primary functionPatient education/medical information low priorityLittle direct e-commerce, delivery of online servicePoor interactivity; One way street on the information highwayHospital centric versus Patient/consumer centric

Discussion cont.What might the future hold for hospital websites?

OptimistInteractive portals for both patients and physiciansServices available online teleconsulting, counselingEducational & diagnostic toolsFoundation for National & Regional Health Information Organizations

Discussion Future of Hospital Websites cont.

PessimistClintonomics: Its all about the economy stupidHealthcare dollars for most hospitals are becoming increasingly scarce, operating margins smaller each yearRobust internet presence luxury item that has little return on the investment dollarHealthcare service unique: does not easily lend itself to other service models (hotels, restaurants, airlines, etc)

Discussion Future of Hospital Websites cont.

RealistInternet has been one of the largest forces for social, economic change in history; medicine is not immuneSmaller operating margins = unique ways to generate revenueDelivery of service via the internet will only increase, not decrease, in spite of the uncertainties as to howStay away from the leading (bleeding) edge of internet technology but when the time comes, know how to use it to your advantage.

SummaryHospital websites currently fail to meet consumer educational needs

Most sites have moderate-high usability but lack of interactivity & viable content, making them less relevant

Ultimate role of hospital websites unknown; successful evolution may ultimately hinge on ability to reinforce need to revisit site (commercially viable)

References:1) Gruca T, Wakefield, Hospital websites benchmarking americas best hospitals Journal of Business Research 5759(2002) 1-7.2) Smith D, A survey and evaluation of hospital websites, Masters research paper, Kent State University, submitted May, 1997.3) Wyatt J, Sullivan FR, eHealth and the future:promise or peril? British Medical Journal 2005; 331; 1391-1393.4) Fulda P, Kwasik H, Consumer health information provided by library and hospital webistes in the south central region, Journal Medical Library Association 92(3) July 2004, pp 372-375.5) Jain T, Barbieri R, Website quality assessment:mistaking apples for oranges, Fertility and Sterility, Volume 83, Issue 3, pp 545-547.6) Hagland M, What's in a Web site? Years into the Internet revolution, hospitals continue to struggle with their online presence. Healthc Inform. 2006 Jul;23(7):14. 7) Hospital Web Sites Disappoint Visitors http://www.forrester.com/Research/Document/Excerpt/0,7211,39846,00.html8) Zingmond DS, Lim YW, Ettner SL, Carlisle DM, Information superhighway or billboards by the roadside? An analysis of hospital websites, West J Med. 2001, December; 175(6): 385-3919) Hernandez-Borges AA, Macis-Cervi P, Gaspar-Guardado MA, DeArcaya ML, Ruiz-Rabaza A, Jimenez-Sosa A, Can examination of WWW usage statistics and other indirect quality indicators help to distinguish the relative quality of medical websites?J Med Internet Res 1999:1(1):e110) Boyer C, The health on the net code of conduct for medical and health websites, Computers in Biology and Medicina, Vol. 28, Issue 5:603-61011) Randeree E, Rao HR, E-health and assurance:curing hospital websites, Int. J Electronic Healthcare, 2004, Vol. 1, No. 1:33-4612) Goldwein JW, Benjamin I, Internet-based medical information: time to take charge, Ann Intern Med 1995 Jul 15;123(2):152-153

*******************

Hospital Websites Evaluation

Documents

siteone hospital

medical center mayfield

lourdes hospital paducah

methodist hospital henderson

harlan arh hospital

kentuckysee main site14

kentuckysee main site25

kentuckysee main site21