BENCHMARK REPORT. AgendaAgenda Beginner Level Basics Beginner Level Basics IntroductionIntroduction Generating reportsGenerating reports Interpreting.

BENCHMARK BENCHMARK REPORTREPORT

BENCHMARK BENCHMARK REPORTREPORT

AgendaAgendaAgendaAgenda

• Beginner Level BasicsBeginner Level Basics• IntroductionIntroduction• Generating reportsGenerating reports• Interpreting reportsInterpreting reports

• Beginner to Intermediate Level DetailsBeginner to Intermediate Level Details• Other report options Other report options • General tipsGeneral tips

• ResourcesResources

2REMINDER: the information generated by this report is only as accurate as the REMINDER: the information generated by this report is only as accurate as the data entered into the NCIR!data entered into the NCIR!

IntroductionIntroductionIntroductionIntroduction

3

Benchmark:Benchmark:Benchmark:Benchmark:

A A benchmarkbenchmark is a standard of excellence or is a standard of excellence or achievement that similar things are compared achievement that similar things are compared to or measured against (in order to evaluate to or measured against (in order to evaluate performance or level of quality).performance or level of quality).

4

NCIR Benchmark Report:NCIR Benchmark Report:NCIR Benchmark Report:NCIR Benchmark Report:

The The NCIR benchmark report NCIR benchmark report assesses the assesses the immunization records of a particular group of immunization records of a particular group of your clients and tells you your clients and tells you HOW MANYHOW MANY, , WHAT WHAT PERCENTPERCENT, and , and WHICHWHICH of these clients have of these clients have met and not met a particular “benchmark” for met and not met a particular “benchmark” for immunizations.immunizations.

5

Key concept:Key concept:Key concept:Key concept:•Select a group of Select a group of clients to assess. clients to assess.

•Select the criteria Select the criteria for meeting the for meeting the benchmark (BM).benchmark (BM).

•The report The report determines which of determines which of these clients have these clients have met the benchmark met the benchmark criteria and which criteria and which have not.have not.

•within any age or birth within any age or birth date rangedate range

(

clients who have not met benchmark

Up-to-date with: specific vaccine(s)

By: specific age (for childhood vaccines)

selected clients

clients who have met

benchmark

(

((

•Select a group of Select a group of clients to assess. clients to assess.

•Select the criteria Select the criteria for meeting the for meeting the benchmark.benchmark.

•Report determines Report determines which of these which of these clients have met the clients have met the benchmark criteria benchmark criteria and which have not.and which have not.

•within any age or birth within any age or birth date rangedate range

Key concept:Key concept:Key concept:Key concept:

select benchmark criteria

select clients

•To learn theTo learn the number and ercent number and ercent of clients up-of clients up-to-date (UTD) with a particular vaccine or group to-date (UTD) with a particular vaccine or group of vaccines: of vaccines:

• by a specific age (for childhood immunizations)by a specific age (for childhood immunizations)• by the date the report was runby the date the report was run

•For aFor a list of clients list of clients that indicates (for each that indicates (for each selected vaccine) whether the client was/had:selected vaccine) whether the client was/had:

• UTD by benchmark age; late UTD; due (and if UTD by benchmark age; late UTD; due (and if so, number of doses received); refusedso, number of doses received); refused

Why generate a report?Why generate a report?Why generate a report?Why generate a report?number and percent

list of clients

Why generate a report?Why generate a report?Why generate a report?Why generate a report?

number &

percent Info

list of clients

Generating ReportsGenerating ReportsGenerating ReportsGenerating ReportsBeginner LevelBeginner Level

10

To open: To open: To open: To open:

•Click “benchmark report” Click “benchmark report” (on the left side of the NCIR (on the left side of the NCIR screen, under the “reports” screen, under the “reports” section)section)

NoteNote: Benchmark reports : Benchmark reports must be generated from an must be generated from an “Administrator” account.“Administrator” account.

11

12

These are the These are the main sections main sections of concern for of concern for beginner level beginner level reports.reports.

Beginner ReportsBeginner ReportsBeginner ReportsBeginner Reports

6

3select clients

select benchmark criteria

Select Evaluation DateSelect Evaluation Date5

3

13

•The The 11stst/default option /default option generates results that can be generates results that can be confusing.* Until comfortable with report, use another confusing.* Until comfortable with report, use another option. option.

•Birth date range: Birth date range: If you have a specific birth date range, If you have a specific birth date range, enter dates or select using calendar icon.enter dates or select using calendar icon.

•Age range: Age range: If you don’t need a specific birth date range, the If you don’t need a specific birth date range, the age range option is simpler. Be sure to specify “days,” “weeks,” age range option is simpler. Be sure to specify “days,” “weeks,” or “years” in the dropdown menus.or “years” in the dropdown menus.

*For more info on this option, see the benchmark report manual.

Select: group of clients to assess Select: group of clients to assess (use only (use only oneone of the three options) of the three options)Select: group of clients to assess Select: group of clients to assess (use only (use only oneone of the three options) of the three options)

14

Select: evaluation dateSelect: evaluation dateSelect: evaluation dateSelect: evaluation date

•Type/select Type/select the current date the current date (the date that you are (the date that you are running the report). You must enter an evaluation date running the report). You must enter an evaluation date or the report will not run. or the report will not run.

• A date that is different from the report generated date can A date that is different from the report generated date can impact the results (designed for advanced level use).*impact the results (designed for advanced level use).*

*For more information on evaluation dates, see the benchmark report manual.

5

•The report will review the immunization status of each client to The report will review the immunization status of each client to see if it meets this BM criteria that you selected.see if it meets this BM criteria that you selected.

Select: benchmark criteriaSelect: benchmark criteriaSelect: benchmark criteriaSelect: benchmark criteria

Column: Vaccine

Rows: Age/Age Group

In cell: Number of doses

required to meet BM

6

16

•Childhood benchmarks (BM): Childhood benchmarks (BM): when you want to know if a group of when you want to know if a group of clients are UTD with vaccines routinely given in childhood. clients are UTD with vaccines routinely given in childhood.

•As childhood vaccine recommendations are based on age, the As childhood vaccine recommendations are based on age, the BM report includes BM report includes ageage and and immunizationimmunization criteria. (Aka: to meet criteria. (Aka: to meet a childhood benchmark, clients must be UTD by a particular age)a childhood benchmark, clients must be UTD by a particular age)

6 Select: benchmark criteriaSelect: benchmark criteriaSelect: benchmark criteriaSelect: benchmark criteria

Childhood benchmarks

17

•Adolescent/Adult Benchmark: Adolescent/Adult Benchmark: when you want to know if a group when you want to know if a group of clients (within any age or birth date range, but usually clients of clients (within any age or birth date range, but usually clients over age 6) are UTD with vaccines routinely given in adolescence over age 6) are UTD with vaccines routinely given in adolescence or adulthood. or adulthood.

•There is no age criteria for these vaccines. Clients meet the There is no age criteria for these vaccines. Clients meet the benchmark if they were UTD when the report was generated.benchmark if they were UTD when the report was generated.

6 Select: benchmark criteriaSelect: benchmark criteriaSelect: benchmark criteriaSelect: benchmark criteria

•Note: Note: Both males and females will be assessed for HPV.Both males and females will be assessed for HPV.

Adolescent/Adultbenchmarks

18

•You can select a full row: You can select a full row: Click on the first cell of the row Click on the first cell of the row that you want to select. The row will become highlighted. that you want to select. The row will become highlighted.

Select: benchmark criteria Select: benchmark criteria Select: benchmark criteria Select: benchmark criteria

In this example: all assessed clients who are UTD with 4DTaP, 3HepB, 3Hib, 1MMR, 3Polio, 4Pnuemo, & 1Var by 24 months of age will meet this benchmark.

6

19

•You can select a single cell: You can select a single cell: Just click on it.Just click on it.

Select: benchmark criteria Select: benchmark criteria Select: benchmark criteria Select: benchmark criteria

In this example: the report will determine which of the assessed clients were UTD with Hep B at 16 months.

•You can select multiple cells within the same row,You can select multiple cells within the same row, but but you cannot select cells in more than one row. you cannot select cells in more than one row.

6

In this example: clients UTD with 1Mening and 3 doses of HPV will meet BM

20

Select: benchmark criteria Select: benchmark criteria Select: benchmark criteria Select: benchmark criteria •To deselect: To deselect: Just click on the cell and it will no longer be Just click on the cell and it will no longer be highlighted. highlighted.

In this example: clients must have completed the 4:3:3:1:3:1 series by 24 months of age in order to meet the benchmark

6

To process:To process:To process:To process:

21

•Click generate (only one time, please) and then wait…Click generate (only one time, please) and then wait…

To view report: To view report: To view report: To view report:

22

•Report will take a few minutes to complete. Click Report will take a few minutes to complete. Click “refresh” occasionally to determine report status.“refresh” occasionally to determine report status.

•When status is “DONE,” click on blue benchmark link to When status is “DONE,” click on blue benchmark link to view report.view report.

Interpreting Report Interpreting Report ResultsResults

Interpreting Report Interpreting Report ResultsResults

For beginner level reportsFor beginner level reports

23

24

So, this is the So, this is the example example criteria that criteria that was selected, was selected, and now we and now we will look at the will look at the report that report that would be would be generated generated from this from this criteria.criteria.

Beginner ReportsBeginner ReportsBeginner ReportsBeginner Reports

6

3

Report run date: 1/9/13Report run date: 1/9/135

6

3Clients: age 24-36 months

Benchmark: UTD with

4:3:3:1:3:4:1 series by 24

months

•List of selected criteria for report (“filter conditions”)List of selected criteria for report (“filter conditions”)•Number & percent InformationNumber & percent Information•List of clientsList of clients

Key sections of reportKey sections of reportKey sections of reportKey sections of report

report

criteria number and

percent Info

list of clients

• Each selection you made for the report is listed here Each selection you made for the report is listed here • Tells you exactly what the report meansTells you exactly what the report means

Selected criteria for reportSelected criteria for reportSelected criteria for reportSelected criteria for report

“Report generated” Report generated” and “Evaluation” and “Evaluation” dates are the samedates are the same

Selected Clients: 24-

Selected Clients: 24-

36 months of age

36 months of age

Selected benchmark immunizations” Selected benchmark immunizations”

4:3:3:1:3:1 by 24 months4:3:3:1:3:1 by 24 months

Understanding Report Criteria Understanding Report Criteria Understanding Report Criteria Understanding Report Criteria

27

Clients:Clients: Benchmark:Benchmark:

Clients:Clients:Benchmark:Benchmark:

•The primary outcome of the report. The primary outcome of the report. •Found on the last two lines of the “filter conditions” section.Found on the last two lines of the “filter conditions” section.•The first number is the number of total clients that were The first number is the number of total clients that were assessed.assessed.

Number & percent infoNumber & percent infoNumber & percent infoNumber & percent info

number and

percent info

Understanding number and percent info:Understanding number and percent info:Understanding number and percent info:Understanding number and percent info:

•Line 1: UTD with BM vaccines by BM ageLine 1: UTD with BM vaccines by BM age•Line 2: UTD with BM vaccines as of date report generated (Line 2: UTD with BM vaccines as of date report generated ( includes includes clients on the first line plus those who were late UTD)clients on the first line plus those who were late UTD)

Misleading. Pretend this says: “Including late UTD”

4 of 10 clients (40%) 4 of 10 clients (40%)

UTD by 24 monthsUTD by 24 months

5 of 10 clients (50%) UTD by the report generated date 5 of 10 clients (50%) UTD by the report generated date (the 4 who met the benchmark and 1 late)(the 4 who met the benchmark and 1 late)

•Adolescent/Adult benchmarks have Adolescent/Adult benchmarks have no age criteriano age criteria•Clients Clients cannot be late UTDcannot be late UTD•The two number and percent lines are identicalThe two number and percent lines are identical

For Adolescent/Adult BM:For Adolescent/Adult BM:For Adolescent/Adult BM:For Adolescent/Adult BM:

5 of 12 clients UTD/ 5 of 12 clients UTD/

met all BM criteriamet all BM criteria

Clients cannot be late UTD. So, Clients cannot be late UTD. So, including late UTD clients, the same 5 including late UTD clients, the same 5

of 12 (42%) met all BM criteriaof 12 (42%) met all BM criteria

.

Client listClient listClient listClient list•Lists clients who did not meet all benchmark criteria Lists clients who did not meet all benchmark criteria ((Note: Note: you can choose which clients are listed- discussed you can choose which clients are listed- discussed on following slides).on following slides).

report

criteria

Number

and percent Info

List of

Clients

Did not meet benchmarkDid not meet benchmark•N C: N C: UTD (“completed”) UTD (“completed”) vaccines, but late UTDvaccines, but late UTD•N # : N # : Not UTD (and the # of Not UTD (and the # of doses received)doses received)•N : N : Not UTD (did not Not UTD (did not receive any doses of vaccine)receive any doses of vaccine)

Understanding the client listUnderstanding the client listUnderstanding the client listUnderstanding the client list

Met benchmarkMet benchmark• Y : Y : met all benchmark met all benchmark

criteria for that criteria for that vaccinevaccine

Column: Selected BM Vaccine Parenthesis: number of doses required to be UTD)

33

To view a specific record:To view a specific record:To view a specific record:To view a specific record:

• This list is an easy way to quickly review client names, This list is an easy way to quickly review client names, addresses, and phone numbers and see if any are obviously addresses, and phone numbers and see if any are obviously missing or wrong (e.g. out of state).missing or wrong (e.g. out of state).

• You can view a specific client record by clicking on the link You can view a specific client record by clicking on the link on the client’s last name.on the client’s last name.

34

Gender is coded

as “U” (unknown).Missing address

•You can view You can view client info but client info but you cannot make you cannot make changes.changes. •When you close When you close out of the screen out of the screen you’re still able you’re still able to view report.to view report.

Client comments

Reviewing client informationReviewing client informationReviewing client informationReviewing client information

35

Updating client informationUpdating client informationUpdating client informationUpdating client information

•Please correct or update any info that is incorrect or Please correct or update any info that is incorrect or missing (missing (NoteNote: Changes will not show up on a report : Changes will not show up on a report until the next day).until the next day).

• Steps: Steps: copy the Client ID and use it to search for the copy the Client ID and use it to search for the client through “manage clients.” Make edits. client through “manage clients.” Make edits.

Saving/exporting reportSaving/exporting reportSaving/exporting reportSaving/exporting report

•Be sure to save the report! Once you run the next Be sure to save the report! Once you run the next report, any previously run report will disappear!report, any previously run report will disappear!

•In order to save the report you must first export it as In order to save the report you must first export it as text or a spreadsheet or display it as a PDFtext or a spreadsheet or display it as a PDF

• Filter conditions can only be saved and viewed in PDF Filter conditions can only be saved and viewed in PDF format.format.

• You can use a spreadsheet if you want to sort or further You can use a spreadsheet if you want to sort or further filter the client information.filter the client information.

*See user manual for details on why you want to export information and what you can do with it.

To re-open a report: To re-open a report: To re-open a report: To re-open a report:

•Once you leave the BM Once you leave the BM screen you can still return to screen you can still return to the last report that was the last report that was generated. generated.

•Click on “check benchmark.” Click on “check benchmark.”

37

Other options when Other options when generating the generating the benchmarkbenchmark

Other options when Other options when generating the generating the benchmarkbenchmark

•Beginner to Intermediate LevelBeginner to Intermediate Level

38

6

3

39

Again, here is Again, here is the screen the screen where you where you can build the can build the report. report. 5

1

2

4

Screen used to build reportScreen used to build reportScreen used to build reportScreen used to build report

40

Sections that Sections that can be skipped can be skipped but may at but may at times be useful. times be useful.

1

2

4

Screen used to build reportScreen used to build reportScreen used to build reportScreen used to build report

1

41

Select: client populationSelect: client populationSelect: client populationSelect: client population

•Only relevant if your practice is associated in the NCIR Only relevant if your practice is associated in the NCIR with multiple sites/clinics and you want to assess clients with multiple sites/clinics and you want to assess clients from only one of those sites.from only one of those sites.

• If the above applies, select the If the above applies, select the 11stst option option and specify a and specify a site using the dropdown menu. site using the dropdown menu.

• Otherwise, leave the Otherwise, leave the 22ndnd/default/default option selected. option selected.

Dropdown MenuDropdown Menu

1

42

Select: client population (LHDs only)Select: client population (LHDs only)Select: client population (LHDs only)Select: client population (LHDs only)

•Local Health Dept. accounts display two extra client Local Health Dept. accounts display two extra client population options:population options:

•3rd option: 3rd option: Assesses only people who are listed in the NCIR Assesses only people who are listed in the NCIR as residents of your county (whether or not they are “active” as residents of your county (whether or not they are “active” clients). clients).

•4th option: 4th option: Assesses your “active” clients (regardless of their Assesses your “active” clients (regardless of their county of residence) AND people who are listed in the NCIR as county of residence) AND people who are listed in the NCIR as residents of your county,residents of your county,

2

43

Select: clients displayed on the client listSelect: clients displayed on the client listSelect: clients displayed on the client listSelect: clients displayed on the client list

•This section determines which clients are displayed in This section determines which clients are displayed in the client list. It has no other impact. the client list. It has no other impact.

•11stst/default option: /default option: Generates a list of clients who did NOT Generates a list of clients who did NOT meet the benchmark. meet the benchmark.

•22ndnd option: option: Generates a list of clients who DID meet the Generates a list of clients who DID meet the benchmark.benchmark.

•33rdrd option: option: Generates a list of all assessed clients (those who Generates a list of all assessed clients (those who met benchmark and those who did not).met benchmark and those who did not).

•You can run the report so that clients who refused a You can run the report so that clients who refused a vaccine are assessed as UTD with that vaccine.* vaccine are assessed as UTD with that vaccine.*

• 11stst/default option: /default option: only considers vaccinated clients as only considers vaccinated clients as UTD.UTD.

• 22ndnd option: option: considers both vaccinated clients and those considers both vaccinated clients and those who refused the vaccine as UTD. who refused the vaccine as UTD.

*Based on “client comments” recorded in the NCIR. For more info, see user manual.

4 Select: how to assess refusalsSelect: how to assess refusalsSelect: how to assess refusalsSelect: how to assess refusals

General Report TipsGeneral Report TipsGeneral Report TipsGeneral Report Tips

45

RefusalsRefusalsRefusalsRefusals

•One refusal of a vaccine is counted on the One refusal of a vaccine is counted on the benchmark report as one “dose” of that vaccine. benchmark report as one “dose” of that vaccine.

• Example: If a client has 2 refusals for HepB, then Example: If a client has 2 refusals for HepB, then the client would meet the benchmark for 1Hep but the client would meet the benchmark for 1Hep but would not meet a benchmark that was run for 3Hep. would not meet a benchmark that was run for 3Hep.

•You can now record multiple client refusals for the You can now record multiple client refusals for the same vaccine in “client comments,” as long as each same vaccine in “client comments,” as long as each entry has a different “applies-to” date. entry has a different “applies-to” date.

•You may want to review previous refusals and add You may want to review previous refusals and add “as-of” date.“as-of” date.

46

Refusal example Refusal example Refusal example Refusal example You want to know: You want to know: •Clients: age 13-19Clients: age 13-19•Benchmark: UTD with 3HPVBenchmark: UTD with 3HPV

47

48

If refusals do not meet benchmarkIf refusals do not meet benchmarkIf refusals do not meet benchmarkIf refusals do not meet benchmark

If refusals meet benchmarkIf refusals meet benchmarkIf refusals meet benchmarkIf refusals meet benchmark

50

Refusals on client list Refusals on client list Refusals on client list Refusals on client list

An “R” in the client list indicates that the client is considered UTD An “R” in the client list indicates that the client is considered UTD for all doses of the vaccine as a result of refusals. for all doses of the vaccine as a result of refusals.

NoteNote: “R”s only display on reports run for childhood BMs. For : “R”s only display on reports run for childhood BMs. For adolescent/adult BMs, UTD as a result of refusals is indicated the adolescent/adult BMs, UTD as a result of refusals is indicated the same as in UTD as a result of vaccination (“Y”).same as in UTD as a result of vaccination (“Y”).

•Childhood BMsChildhood BMs: You can assess clients : You can assess clients within any age or birth date range. within any age or birth date range.

•Adolescent/Adult BMsAdolescent/Adult BMs: Report will not run if : Report will not run if all selected clients are age 6 and under. You all selected clients are age 6 and under. You cancan assess clients within any age or birth date assess clients within any age or birth date range as long as part of the range includes range as long as part of the range includes clients who are over age 6 years.clients who are over age 6 years.

51

Client age and BM ageClient age and BM ageClient age and BM ageClient age and BM age

Flu shots for clients age 6 and underFlu shots for clients age 6 and underFlu shots for clients age 6 and underFlu shots for clients age 6 and under

You want to know: You want to know: •Clients: ages 24-36 monthsClients: ages 24-36 months•Benchmark: UTD with flu vaccineBenchmark: UTD with flu vaccine

Steps: Steps: •Select clients using an age or birth date range Select clients using an age or birth date range that includes people older than 6. For example: that includes people older than 6. For example: age 2 to 7 years. age 2 to 7 years. •You could export to Excel, arrange columns in You could export to Excel, arrange columns in order of date of birth, and quickly narrow the list order of date of birth, and quickly narrow the list down to only clients in your desired birth date down to only clients in your desired birth date range. range.

52

Flu shots for clients age six and underFlu shots for clients age six and underFlu shots for clients age six and underFlu shots for clients age six and under

53

Report sizeReport sizeReport sizeReport size

•It can be more manageable and useful to run It can be more manageable and useful to run reports for a narrow birth date or age range (to reports for a narrow birth date or age range (to reduce the number of clients included). reduce the number of clients included).

•Similarly it can be more manageable and Similarly it can be more manageable and useful to run reports on a regular, frequent useful to run reports on a regular, frequent basis in order to follow-up with clients basis in order to follow-up with clients throughout the year. throughout the year.

QUESTIONS?QUESTIONS?QUESTIONS?QUESTIONS?

55