c Consult author(s) regarding copyright matters · Professor of clinical epidemiology. Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Centre,

This may be the author’s version of a work that was submitted/acceptedfor publication in the following source:

Korevaar, Daniel, Hooft, Lotty, Askie, Lisa, Barbour, Ginny, Faure, Helene,Gatsonis, Constantine, Hunter, Kylie, Kressel, Herbert, Lippman, Han-nah, McInnes, Matthew, Moher, David, Rifai, Nader, Cohen, Jeremie, &Bossuyt, Patrick(2017)Facilitating prospective registration of diagnostic accuracy studies: ASTARD initiative.Clinical Chemistry, 63(6), pp. 1-11.

This file was downloaded from: https://eprints.qut.edu.au/108237/

c© Consult author(s) regarding copyright matters

This work is covered by copyright. Unless the document is being made available under aCreative Commons Licence, you must assume that re-use is limited to personal use andthat permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then referto the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe thatthis work infringes copyright please provide details by email to [email protected]

Notice: Please note that this document may not be the Version of Record(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) canbe identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.

https://doi.org/10.1373/clinchem.2017.272765

1

Facilitating Prospective Registration of Diagnostic Accuracy Studies:

A STARD Initiative

Running head: STARD for Registration

List of authors, academic degrees, positions, affiliations, and email addresses:

Daniël A. Korevaar, MD PhD, Physician researcher

Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical

Centre, University of Amsterdam, Amsterdam, the Netherlands.

[email protected]

Lotty Hooft, PhD, Co-director

Netherlands Trial Register and Cochrane Netherlands, Julius Center for Health Sciences and

Primary Care, University Medical Center Utrecht, University of Utrecht, Utrecht, the

Netherlands.

[email protected]

Lisa M. Askie, PhD MPH BN NICC RM RN, Director

Australian New Zealand Clinical Trials Registry (ANZCTR), Systematic Reviews and Health

Technology Assessment team, NHMRC Clinical Trials Centre, University of Sydney,

Sydney, Australia.

[email protected]

Virginia Barbour, MBBChir DPhil Professor

Office of Research Ethics and Integrity and Division of Technology, Information and

Learning Services, Queensland University of Technology (QUT) Brisbane, Australia.

[email protected]

Hélène Faure, Database Manager

2

ISRCTN registry, BioMed Central, London, UK.

[email protected]

Constantine A. Gatsonis, PhD, Professor of biostatistics

Center for Statistical Sciences, Brown University School of Public Health, Providence,

Rhode Island, USA.

[email protected]

Kylie E. Hunter, BA(Hons) MPH, Senior Project Officer

Australian New Zealand Clinical Trials Registry (ANZCTR), NHMRC Clinical Trials

Centre, University of Sydney, Sydney, Australia.

[email protected]

Herbert Y. Kressel, MD, Editor-in-Chief Radiology

Department of Radiology, Beth Israel Deaconess Medical Center, Miriam H. Stoneman

Professor of Radiology, Harvard Medical School, Boston, Massachusetts, USA; Radiology

Editorial Office, Boston, Massachusetts, USA.

[email protected]

Hannah Lippman, Database Editor

ISRCTN registry, BioMed Central, London, UK.

[email protected]

Matthew D. McInnes, MD, Associate Professor

Department of Radiology, University of Ottawa, Ottawa, Canada.

[email protected]

David Moher, PhD, Senior scientist

Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada; School of

Epidemiology, Public Health and Preventive Medicine, University of Ottawa, Ottawa,

3

Canada.

[email protected]

Nader Rifai, PhD, DABCC, FACB, Editor-in-Chief Clinical Chemistry

Department of Laboratory Medicine, Boston Children's Hospital, Harvard Medical School,

Boston, Massachusetts, USA; Clinical Chemistry Editorial Office, Boston, Massachusetts,

USA.

[email protected]

Jérémie F. Cohen, MD PhD, Postdoctoral research fellow

Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical

Centre, University of Amsterdam, Amsterdam, the Netherlands; INSERM UMR 1153 and

Department of Pediatrics, Necker Hospital, AP-HP, Paris Descartes University, Paris, France.

[email protected]

Patrick M. Bossuyt, PhD, Professor of clinical epidemiology

Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical

Centre, University of Amsterdam, Amsterdam, the Netherlands.

[email protected]

Corresponding author: Prof. Patrick M. Bossuyt

Department of Clinical Epidemiology, Biostatistics and Bioinformatics

Academic Medical Center - University of Amsterdam

PO Box 22700, 1100 DE Amsterdam, The Netherlands

Email: [email protected]

Phone: +31(20)566 3240 Fax: +31(20)691 2683

Article type: Special Report.

4

Word count (text only): 2,887.

Keywords: Trial registration; diagnostic accuracy; sensitivity and specificity; research waste.

Abbreviations: AUC = area under the receiver operating characteristic curve; ICMJE =

International Committee of Medical Journal Editors; ISRCTN = International Standard

Randomised Controlled Trial Number; WHO = World Health Organization; STARD =

Standards for Reporting Diagnostic Accuracy; TRDS = Trial Registration Data Set.

5

Abstract

Although the introduction of prospective trial registration policies has been successful in

reducing waste in research, diagnostic accuracy studies are rarely registered. We describe

why diagnostic accuracy studies should be registered, and where and how this can be done.

Advantages of registration include that the identification of unpublished studies, prevention

of selective outcome reporting, prevention of unnecessary duplication of research,

collaboration between researchers and linkage of study materials. In a survey among

representatives of 16 major trial registries, such as ClinicalTrials.gov, ANZCTR (Australian

New Zealand Clinical Trials Registry) and the UK based ISRCTN registry (International

Standard Randomised Controlled Trial Number), 13 responded of which 8 (62%) indicated

they always accept registration of diagnostic accuracy studies, and 5 (38%) do so in some

cases. However all but one of them (92%) indicated that their registry currently does not

provide specific guidance for registering diagnostic accuracy studies. A second survey among

the 85 members of the STARD Group (Standards for Reporting Diagnostic Accuracy)

resulted in the identification of 14 essential protocol items and was used to develop guidance

on how these items can be registered in existing major trial registries. We propose that

investigators responsible for diagnostic accuracy studies should register their study, before

recruiting patients, in one of the existing major trial registries that are willing to host such

studies. We also propose that governmental, research and academic institutions providing

funding for and journals publishing diagnostic accuracy studies require such registration.

6

Introduction

Over the past decade, there have been growing concerns about a wide range of sources of

avoidable waste in biomedical research.[1] Examples include studies addressing low priority

clinical questions, study results that are unreported, and published study reports that are not

as informative as they could be.[1 2]

One approach to reduce research waste has been the introduction of prospective trial

registration policies.[3] In 2005, the International Committee of Medical Journal Editors

(ICMJE) declared that its member journals would only consider reports of trials for

publication if they had been registered in a publicly accessible trial registry prospectively,

before enrolment of the first study participant.[4] Subsequently, similar requirements have

been implemented by several governmental organizations, funders and academic

institutions.[2] These policies led to a large increase in registered studies,[5] and about half of

recently published trials now report a registration number.[6]

Unfortunately, registration rates remain low for diagnostic accuracy studies,[7 8] despite the

fact that the compelling rationale for prospective registration applies to this type of study

design.[9-12] One reason may be that registration policies have so far mainly addressed

randomized trials of interventions, while diagnostic accuracy studies, which compare medical

tests against a clinical reference standard, are often considered to be observational. Another

reason may be the unavailability of specific guidance for registration of these studies.

In this article, we set out why we believe that there is a need to register diagnostic accuracy

studies, and where and how this can be done. We provide guidance for authors and readers,

based on two online surveys (Figure 1).

What are diagnostic accuracy studies?

7

Diagnostic tests are indispensable in clinical practice. Physical examination, imaging

techniques, laboratory tests and other forms of medical testing inform clinicians about the

likelihood that a tested patient has the suspected target disease or condition, and guide

subsequent decisions about further testing or treatment.

Most diagnostic tests are not perfectly accurate in classifying patients as having the target

condition or not, and some of those tested will a have false positive or false negative test

result.[13] Diagnostic accuracy is typically evaluated by comparing test results with those of

a reference standard in a series of patients suspected of having a target condition. The

diagnostic accuracy of the test under investigation is then expressed in measures such as

sensitivity and specificity, positive and negative predictive values, likelihood ratios and area

under the receiver operating characteristic curve (AUC).[13]

Every year, thousands of these diagnostic accuracy studies are being undertaken. Some are

based on retrospective review of chart data, many others prospectively collect data in patients

who have specifically consented to these data being collected. Sometimes, evaluations of

diagnostic accuracy are incorporated into larger overarching projects that have also other,

non-diagnostic study objectives.[7]

Why register diagnostic accuracy studies?

There are numerous reasons why prospective registration of diagnostic accuracy studies is

beneficial for those involved in conducting research as well as those who rely on the research

findings.

Research transparency as an ethical obligation. Full disclosure of all study materials,

including the protocol, is increasingly considered to be an ethical obligation, not only toward

individuals who participated in the study on the understanding that their data would

8

contribute to scientific knowledge, but also toward future patients and public funders of

research, including tax payers.[14] The Declaration of Helsinki states: “Every research study

involving human subjects must be registered in a publicly accessible database before

recruitment of the first subject”.[15]

Identify unpublished studies. As described for trials of interventions, many initiated

diagnostic accuracy studies are never published, while others take years to get published.[8

16-19] This may lead to reporting bias when summarizing the available evidence in

systematic reviews, meta-analyses and clinical practice guidelines.[19] Such bias could result

in invalid implementation of medical tests in clinical practice and suboptimal clinical

decision making, potentially threatening patient safety and efficient use of healthcare funds.

Registration facilitates the identification of unpublished studies, thereby providing an

opportunity to prevent this major source of research waste.[20] If a registered record is

available for the study, principal investigators can be contacted to obtain study results, and to

identify reasons for not publishing them. By helping the inclusion of unpublished studies in

systematic reviews, registration could also increase precision of summary estimates in meta-

analyses and power for assessing sources of heterogeneity. An analysis of 117 systematic

reviews assessing medical interventions published in 2012-2013 identified that 35%

contained a screen of trial registries for unpublished trials.[21]

Identify and prevent selective reporting within studies. Selective reporting of pre-defined

primary outcomes appears to be common among published diagnostic accuracy studies.[16]

Such selective publication may provide a distorted view of diagnostic accuracy, leading to

erroneous impressions of the usefulness of the test in clinical practice. If studies are

prospectively registered with pre-specified outcomes, peer reviewers and journal editors will

have the opportunity to identify and prevent selective reporting in study reports submitted for

publication. In a survey among peer reviewers, it was shown that one-third examined

9

corresponding trial registries and reported discrepancies with the peer-reviewed manuscript to

journal editors.[22] Unreported outcomes can be identified, and corresponding results can be

obtained by contacting the principal investigators.

Facilitate the peer review process. The registered record provides a more comprehensive

knowledge of the methods, allows for a comparison between what was originally proposed

and what was actually done, and facilitates the identification of potential reporting biases as

registered and reported outcomes can easily be compared.

Prevent unnecessary duplication of research and identify research gaps. Unpublished or

ongoing diagnostic accuracy studies may be unnecessarily duplicated if other researchers are

not aware of their existence.[10] Before initiating a new study, researchers and funders could

search trial registries to identify ongoing or completed studies with similar objectives. This

may help them in appreciating the research gaps that further studies could address. Such

practices may lead to a more efficient allocation of research funds and efforts. In addition,

unnecessary exposure of study participants to potentially harmful tests or other interventions

can be prevented.

Obtain additional study information. Reports of diagnostic accuracy studies are often

incomplete in their description of essential study design features, making it difficult for

readers to assess the risk of bias and the applicability of the study findings.[23 24] In such

cases, corresponding completed registered records may be useful as an additional source of

information on study methods, as has been shown for trials of therapeutic interventions.[25]

Facilitate collaboration between researchers. By facilitating the identification of studies that

are about to start or ongoing, registration has the potential to improve collaborations between

researchers that are operational in similar fields of research.

10

Facilitate study participant recruitment. Patients are among the main users of trial

registries.[26] Those willing to participate in scientific research, and clinicians looking for

relevant research projects for their patients to participate in, will have the opportunity to

identify and participate in eligible studies if these are registered.

Improve protocol and study quality. Diagnostic accuracy studies are particularly sensitive to

bias, which may jeopardize the trustworthiness of study results.[27] Registration can improve

protocol development, as it may encourage researchers to carefully consider a standardized

set of critical protocol elements that are likely to affect the results of their studies, thereby

improving the overall methodological quality of the study.

Facilitate linkage of study material. Increasingly, individual studies lead to multiple

reports.[28] Researchers can publish the trial protocol, report results in conference abstracts,

in applications with regulatory bodies, in patient information leaflets, on websites, and in one

or more full publications in biomedical journals. These reports often focus on different

research questions and outcomes, sometimes based on a subset of the total study group.

Linkage of reports that originate from the same study can be difficult.[28] This is challenging

for researchers who want to identify all the relevant evidence for a systematic review, and

may come across publications with potentially overlapping participants. Linkage will be

highly facilitated if all diagnostic accuracy studies have a unique registration number that is

included in any report generated from the study.

In contrast to randomized trials, as performed for pharmaceuticals, diagnostic accuracy

studies can also be retrospective in nature, relying on chart records or previously collected

data. Some of the benefits of registration may not fully apply to such retrospective studies,

such as the potential for collaboration, but many other advantages still apply, if the authors

register their study before they extract the data and start performing their analyses.

11

Where to register diagnostic accuracy studies?

No central trial register exists that focuses specifically on diagnostic accuracy studies, and we

do not see the need to start a separate register. We believe that registration of diagnostic

accuracy studies within existing major trial registries is a better option than developing a

registry specifically designed for such studies, because (1) these registries are widely known

and used in the scientific and clinical community, (2) some already contain numerous

diagnostic accuracy studies, (3) a limited number of large, well-designed registries has

practical advantages over many small ones that focus on specific types of research, and (4)

diagnostic accuracy studies are often part of broader studies that also contain non-diagnostic

outcomes.

The major existing trial registries are ClinicalTrials.gov, a service of the US National

Institutes of Health, and 15 Primary registries in the World Health Organization (WHO)

Registry Network (see http://www.who.int/ictrp/network/primary/en/). These are considered

as ‘Primary Registries’ because they “meet specific criteria for content, quality and validity,

accessibility, unique identification, technical capacity and administration”. At present,

ClinicalTrials.gov and the 15 Primary Registries in the WHO Registry Network primarily

focus on the registration of trials of therapeutic interventions, although some welcome studies

with other designs as well.

We performed an online survey among representatives of these 16 registries to assess their

policy towards registration of diagnostic accuracy studies (Figure 1). The survey and results

are provided in Supplemental Data 1 and 2. Thirteen registry representatives (81%)

responded; of these 8 (62%) indicated they always accept registration of diagnostic accuracy

12

studies, and 5 (38%) do so in some cases. A list of major trial registries where diagnostic

accuracy studies can be registered is provided in Table 1.

How to register diagnostic accuracy studies?

In our survey of trial registries, 12 of 13 (92%) respondents indicated that their registry

currently does not provide specific guidance for registering diagnostic accuracy studies, with

the UK based ISRCTN registry (International Standard Randomised Controlled Trial

Number) being the exception.[29] Lack of explicit instructions for registering diagnostic

accuracy studies may be an important reason for researchers to believe that it is difficult or

impossible to register these studies in existing trial registries.

Yet registering diagnostic accuracy studies can be as straightforward as registering a

randomized trial of a new drug. The WHO, which has encouraged trial registration for years,

has proposed a ‘Trial Registration Data Set’, containing 20 items that cover essential study

protocol information that must appear in a registered record for a trial to be considered fully

registered (see http://www.who.int/ictrp/network/trds/en/). Each Primary Registry in the

WHO Registry Network mentioned in Table 1 requires registered records to at least contain

these 20 items. The format of ClinicalTrials.gov is slightly different, although there is major

overlap between items that need to be registered here as well.

Since many items on WHO’s Trial Registration Data Set apply to any type of research

involving humans, there should be no barriers to registration of diagnostic accuracy studies.

Items unique to diagnostic accuracy studies can be incorporated in the Trial Registration Data

Set. To identify such items, we performed a second online survey, among the members of the

STARD (Standards for Reporting Diagnostic Accuracy) group (Figure 1). This group consists

of 85 experts in the field of diagnostic research (Box).[23 24 30]

13

The survey and results are provided in Supplemental Data 3 and 4. In brief, we built a list of

20 items potentially relevant when register a diagnostic accuracy study that are not currently

covered by WHO’s Trial Registration Data Set. This list was established based on the

STARD 2015 reporting guideline.[23 24 30] The items were proposed to survey respondents,

and they could indicate whether they felt that they should or should not be included in the

registered record of a diagnostic accuracy study. Items that were tagged as relevant by two

thirds or more of the respondents were included in our final list. Respondents also had the

opportunity to suggest items not included on the list. The inclusion of these items was

decided based on discussion by the authors of this manuscript, consisting of five trial registry

representatives (LMA, HF, KEH, HL, LH), three current or former editors-in-chief of

journals that publish diagnostic accuracy studies (VB, DM, NR), and five persons with broad

experience in performing diagnostic accuracy studies and systematic reviews thereof (DAK,

JFC, CAG, MDM, PMB).

Seventy-one STARD group members (84%) responded. Based on the survey results, we

recommend that 14 items that are currently not (explicitly) covered by WHO’s Trial

Registration Data Set are incorporated for diagnostic accuracy studies.

Table 2 provides the WHO’s Trial Registration Data Set, and a proposal of where and how

the 14 newly identified items can be mentioned when registering diagnostic accuracy studies,

and examples from existing registered records. This applies to 7 of the 20 items within

WHO’s Trial Registration Data Set; the current definition of the 13 other items equally

applies to diagnostic accuracy studies and does not need any modification.

The Trial Registration Data Set also contains several required or optional data fields related

to the ‘study type’ (item 15), where only fixed options can be selected, for example, whether

the study is ‘interventional’ or ‘observational’, or whether the ‘method of allocation’ is

‘randomized’or ‘non-randomized’. To improve uniformity across registered records, we

14

provide suggestions on how to address these required data fields for diagnostic accuracy

studies in Supplemental Data 5.

How complete are registered records of diagnostic accuracy studies currently?

With the list of 14 recommended items identified in this project, we evaluated the content of

recently registered diagnostic accuracy studies. Using the search term diagnostic accuracy,

we searched WHO’s International Clinical Trials Registry Platform Search Portal on

November 1st, 2016. This engine searches through trial registration data provided by the

Primary Registries and ClinicalTrials.gov. The 30 most recently registered records of

diagnostic accuracy studies were selected. For each record, one author (DAK) assessed if the

newly identified recommended items listed in Table 2 were reported, and in which data field;

difficult cases were discussed with a second author (PMB).

A summary of results is provided in Table 3, with results per registered record in

Supplemental Data 6. Some recommended items were almost always registered, for example:

identification of the record as a diagnostic accuracy study (item 1; 90%), the target condition

(item 2; 97%), and the index test whose diagnostic accuracy is under evaluation (item 3;

100%). Other recommended items were much less frequently registered, for example:

information available to the performers or readers of the index test (item 5; 47%),

information available to the assessors of the reference standard (item 6; 40%), the setting in

which patients will be recruited (item 8; 50%), how data will be collected (item 10; 47%),

and how participants will be sampled (item 11; 37%). We found that 6 records (20%)

contained at least 11 of the 14 recommended items, illustrating that it should be possible to

register all the information proposed in Table 2 in the existing format of trial registers.

15

Who do we want to reach, and who can use this guidance?

With this statement, we aim to reach several key stakeholders:

Researchers. We call on researchers to prospectively register their diagnostic accuracy

studies in one of the major trial registries. We suggest that the guidance proposed will help to

ensure that registered records are fully informative and made uniform across registries. Trial

registries. By hosting registered records of diagnostic accuracy studies, trial registries play a

crucial role. We call on existing major trial registries to ensure that registered records of

diagnostic accuracy studies are sufficiently informative. Registries could incorporate the

proposed guidance into their data field definitions, by adding statements such as: “For

diagnostic accuracy studies, please (also) report [...]”.

Journals. A substantial increase in the number of clinical trials registrations was observed

after the ICMJE implemented its clinical trial registration policy.[5] Here we call on journals

and publishers to actively encourage authors in registering their diagnostic accuracy studies.

This could be done by addressing registration in journals’ instructions to authors, adding a

statement such as: “We recommend researchers to prospectively register their diagnostic

accuracy studies in a publicly accessible trial registry, and to report the corresponding

registration number in the final study report”, or by making the reporting of a trial registration

number – if available – a mandatory field in the submission process and in the reporting of

the abstract.

Funders. Parties funding or employing researchers could strongly recommend or require

registration of diagnostic accuracy studies as a necessary condition for obtaining full funding

or ethical approval. The Wellcome Trust, an “independent global charitable foundation

dedicated to improving health through science, research and engagement with society”, for

example, requires all funded clinical trials to be registered.

16

Institutional review boards and ethics committees. At some centers, clinical trials must have

a registration number before they can get Institutional Review Board approval.[2] Ethics

committees outside of universities can also enforce this. Similar policies could apply to

diagnostic accuracy studies.

Conclusions

Prospective registration provides major opportunities to prevent research waste. The

arguments and guidance provided here can be used to facilitate prospective registration of

diagnostic accuracy studies, and improve registration rates. We propose that investigators

responsible for diagnostic accuracy studies register their study, before recruiting patients, in

one of the 13 existing major trial registries that are willing to host such studies. We also

propose that existing major trial registries consider incorporating the porposed items, as these

are unique to diagnostic accuracy studies, and that bodies funding and journals publishing

diagnostic accuracy studies require full, informative and prospective registration of such

studies.

17

Box: paper’s provenance

STARD is a reporting guideline originally published in 2003 by a large group of researchers

and journal editors with the aim of improving the reporting of diagnostic accuracy studies.

An updated version was recently published: STARD 2015. One of the 30 items on the

updated checklist is “Registration number and name of registry”, which is now recommended

to report in any study report of a diagnostic accuracy study.

The STARD Project Team (DAK, JFC, LH, PMB) decided that STARD should actively take

the lead in promoting registration of diagnostic accuracy studies. This formed the basis of

this article. In this process, the STARD Project Team collaborated closely with the other

authors of this article, who are representatives from trial registries (LMA, HF, KEH, HL,

LH), (former) editors-in-chief of journals that publish diagnostic accuracy studies (VB,

HYK, DM, NR), and researchers/clinicians in the field of diagnostic accuracy (CAG, MDM).

18

Acknowledgements

We thank the following collaborators for participating in the first survey, which aimed to

identify the policy of major trial registries towards registration of diagnostic accuracy studies:

Luiza Rosangela da Silva (Brazilian Clinical Trials Registry), Taixiang Wu (Chinese Clinical

Trial Registry), Gladys Jiménez Rivero (Cuban Public Registry of Clinical Trials), Noemie

Manent (EU Clinical Trials Register), Susanne Jena (German Clinical Trials Register),

Masoud Solaymani-Dodaran (Iranian Registry of Clinical Trials), Elizabeth Pienaar (Pan

African Clinical Trial Registry), Udaya Ranawaka (Sri Lanka Clinical Trials Registry),

Wasee Tulvatana (Thai Clinical Trials Registry) and Tony Tse (ClinicalTrials.gov). We thank

the members of the STARD 2015 group for participating in the second survey, which aimed

to identify important protocol items that need to be registered for diagnostic accuracy studies.

19

References

1. Macleod MR, Michie S, Roberts I, et al. Biomedical research: increasing value, reducing

waste. Lancet 2014;383(9912):101-04 doi: 10.1016/S0140-6736(13)62329-

6[published Online First: Epub Date]|.

2. Moher D, Glasziou P, Chalmers I, et al. Increasing value and reducing waste in biomedical

research: who's listening? Lancet 2016;387(10027):1573-86 doi: 10.1016/S0140-

6736(15)00307-4[published Online First: Epub Date]|.

3. Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical

trial enterprise. JAMA 2012;307(17):1861-64 doi: 307/17/1861

[pii];10.1001/jama.2012.4230 [doi][published Online First: Epub Date]|.

4. De Angelis CD, Drazen JM, Frizelle FA, et al. Is this clinical trial fully registered?--A

statement from the International Committee of Medical Journal Editors. New England

journal of medicine 2005;352(23):2436-38 doi: NEJMe058127

[pii];10.1056/NEJMe058127 [doi][published Online First: Epub Date]|.

5. Viergever RF, Li K. Trends in global clinical trial registration: an analysis of numbers of

registered clinical trials in different parts of the world from 2004 to 2013. BMJ open

2015;5(9):e008932 doi: bmjopen-2015-008932 [pii];10.1136/bmjopen-2015-008932

[doi][published Online First: Epub Date]|.

6. van de Wetering FT, Scholten RJ, Haring T, et al. Trial registration numbers are

underreported in biomedical publications. PLoS one 2012;7(11):e49599 doi:

10.1371/journal.pone.0049599 [doi];PONE-D-12-18367 [pii][published Online First:

Epub Date]|.

7. Korevaar DA, Bossuyt PM, Hooft L. Infrequent and incomplete registration of test

accuracy studies: analysis of recent study reports. BMJ open 2014;4(1):e004596 doi:

10.1136/bmjopen-2013-004596[published Online First: Epub Date]|.

20

8. Korevaar DA, Cohen JF, Spijker R, et al. Reported estimates of diagnostic accuracy in

ophthalmology conference abstracts were not associated with full-text publication.

Journal of clinical epidemiology 2016;79:96-103 doi:

10.1016/j.jclinepi.2016.06.002[published Online First: Epub Date]|.

9. Altman DG. The time has come to register diagnostic and prognostic research. Clinical

chemistry 2014;60(4):580-82 doi: clinchem.2013.220335

[pii];10.1373/clinchem.2013.220335 [doi][published Online First: Epub Date]|.

10. Rifai N, Altman DG, Bossuyt PM. Reporting bias in diagnostic and prognostic studies:

time for action. Clinical chemistry 2008;54(7):1101-03 doi: 54/7/1101

[pii];10.1373/clinchem.2008.108993 [doi][published Online First: Epub Date]|.

11. Rifai N, Bossuyt PM, Ioannidis JP, et al. Registering diagnostic and prognostic trials of

tests: is it the right thing to do? Clinical chemistry 2014;60(9):1146-52 doi:

clinchem.2014.226100 [pii];10.1373/clinchem.2014.226100 [doi][published Online

First: Epub Date]|.

12. Hooft L, Assendelft WJ, Hoeksema HL, et al. [A national prospective trial register for

randomised controlled trials: ethical and practical necessity]. Nederlands tijdschrift

voor geneeskunde 2004;148(38):1866-69

13. Linnet K, Bossuyt PM, Moons KG, et al. Quantifying the accuracy of a diagnostic test or

marker. Clinical chemistry 2012;58(9):1292-301 doi: clinchem.2012.182543

[pii];10.1373/clinchem.2012.182543 [doi][published Online First: Epub Date]|.

14. Hooft L, Bossuyt PM. Prospective registration of marker evaluation studies: time to act.

Clinical chemistry 2011;57(12):1684-86 doi: clinchem.2011.176230

[pii];10.1373/clinchem.2011.176230 [doi][published Online First: Epub Date]|.

21

15. World Medical A. World Medical Association Declaration of Helsinki: ethical principles

for medical research involving human subjects. JAMA 2013;310(20):2191-94 doi:

10.1001/jama.2013.281053[published Online First: Epub Date]|.

16. Korevaar DA, Ochodo EA, Bossuyt PM, et al. Publication and reporting of test accuracy

studies registered in ClinicalTrials.gov. Clinical chemistry 2014;60(4):651-59 doi:

clinchem.2013.218149 [pii];10.1373/clinchem.2013.218149 [doi][published Online

First: Epub Date]|.

17. Brazzelli M, Lewis SC, Deeks JJ, et al. No evidence of bias in the process of publication

of diagnostic accuracy studies in stroke submitted as abstracts. Journal of clinical

epidemiology 2009;62(4):425-30 doi: S0895-4356(08)00228-X

[pii];10.1016/j.jclinepi.2008.06.018 [doi][published Online First: Epub Date]|.

18. Wilson C, Kerr D, Noel-Storr A, et al. Associations with publication and assessing

publication bias in dementia diagnostic test accuracy studies. International journal of

geriatric psychiatry 2015;30(12):1250-56 doi: 10.1002/gps.4283 [doi][published

Online First: Epub Date]|.

19. Korevaar DA, van Es N, Zwinderman AH, et al. Time to publication among completed

diagnostic accuracy studies: associated with reported accuracy estimates. BMC

medical research methodology 2016;16(1):68 doi: 10.1186/s12874-016-0177-

4[published Online First: Epub Date]|.

20. van Enst WA, Scholten RJ, Hooft L. Identification of additional trials in prospective trial

registers for Cochrane systematic reviews. PLoS one 2012;7(8):e42812 doi:

10.1371/journal.pone.0042812 [doi];PONE-D-12-10497 [pii][published Online First:

Epub Date]|.

22

21. Jones CW, Keil LG, Weaver MA, et al. Clinical trials registries are under-utilized in the

conduct of systematic reviews: a cross-sectional analysis. Syst Rev 2014;3:126 doi:

10.1186/2046-4053-3-126[published Online First: Epub Date]|.

22. Mathieu S, Chan AW, Ravaud P. Use of trial register information during the peer review

process. PLoS one 2013;8(4):e59910 doi: 10.1371/journal.pone.0059910 [doi];PONE-

D-12-38579 [pii][published Online First: Epub Date]|.

23. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: An Updated List of Essential

Items for Reporting Diagnostic Accuracy Studies. Clinical chemistry

2015;61(12):1446-52 doi: 10.1373/clinchem.2015.246280[published Online First:

Epub Date]|.

24. Cohen JF, Korevaar DA, Altman DG, et al. STARD 2015 guidelines for reporting

diagnostic accuracy studies: explanation and elaboration. BMJ open

2016;6(11):e012799 doi: 10.1136/bmjopen-2016-012799[published Online First:

Epub Date]|.

25. Scherer RW, Huynh L, Ervin AM, et al. ClinicalTrials.gov registration can supplement

information in abstracts for systematic reviews: a comparison study. BMC medical

research methodology 2013;13:79 doi: 1471-2288-13-79 [pii];10.1186/1471-2288-13-

79 [doi][published Online First: Epub Date]|.

26. Hunter K, Vu T, Sausa R, et al. Australian New Zealand Clinical Trials Registry

(ANZCTR) user survey results from 2009 and 2015. Cochrane Colloquium. Vienna,

2015.

27. Whiting PF, Rutjes AW, Westwood ME, et al. A systematic review classifies sources of

bias and variation in diagnostic test accuracy studies. Journal of clinical epidemiology

2013;66(10):1093-104 doi: S0895-4356(13)00200-X

[pii];10.1016/j.jclinepi.2013.05.014 [doi][published Online First: Epub Date]|.

23

28. Altman DG, Furberg CD, Grimshaw JM, et al. Linked publications from a single trial: a

thread of evidence. Trials 2014;15:369 doi: 10.1186/1745-6215-15-369[published

Online First: Epub Date]|.

29. Lippman H. Diagnostic accuracy: 60% of the time, it works every time. BioMed Central

blog network 2016.

30. Korevaar DA, Cohen JF, Reitsma JB, et al. Updating standards for reporting diagnostic

accuracy: the development of STARD 2015. Research Integrity and Peer Review

2016;1(1):7

24

Table 1. Major trial registries where diagnostic accuracy studies can be registered.

Registry Website Condition for registration of diagnostic accuracy

studies

Registries that always accept registration of diagnostic accuracy studies:

Australian New Zealand Clinical Trials

Registry www.anzctr.org.au/ -

Brazilian Clinical Trials Registry www.ensaiosclinicos.gov.br/ -

Chinese Clinical Trial Registry www.chictr.org/en/ -

German Clinical Trials Register drks-neu.uniklinik-

freiburg.de/drks_web/ -

ISRCTN registry www.isrctn.com/

“The ISRCTN registry is a primary clinical trial

registry recognized by WHO and ICMJE that accepts

all clinical research studies (whether proposed, ongoing

or completed), providing content validation and

curation and the unique identification number

necessary for publication.”

Netherlands National Trial Register www.trialregister.nl/ -

Thai Clinical Trials Registry www.clinicaltrials.in.th/ -

ClinicalTrials.gov www.clinicaltrials.gov

“ClinicalTrials.gov accepts registration of all studies

that ‘are in conformance with applicable human

subjects or ethics review regulations (or equivalent)

and applicable regulations of the national (or regional)

health authority (or equivalent).’”

Registries that in some cases accept registration of diagnostic accuracy studies:

Cuban Public Registry of Clinical Trials registroclinico.sld.cu/

“The Cuban Public Registry of Clinical Trials does not

have a specific data set for this type of trials, but if the

trial fulfills the fields it is possible to register it.”

EU Clinical Trials Register www.clinicaltrialsregister.eu/

“If the study is also a clinical trial. This means that

there is an investigational medical product being

administered to the subjects taking part in the trial.”

Iranian Registry of Clinical Trials www.irct.ir/ “If they comply with the WHO’s definition of clinical

trials.”

Pan African Clinical Trial Registry www.pactr.org/ “Currently Pan African Clinical Trial Registry only

accepts RCTs and CCTs.”

Sri Lanka Clinical Trials Registry trials.slctr.lk/ “Where there is a clear intervention being tested.”

Registries that did not respond in our survey:

Clinical Research Information Service,

Korea cris.nih.go.kr/ -

Clinical Trials Registry – India ctri.nic.in/ -

Japan Primary Registries Network rctportal.niph.go.jp/ -

Table legend: WHO=World Health Organization; ICMJE=International Committee of

Medical Journal Editors; RCT=randomized clinical trial; CCT=controlled clinical trial.

25

Table 2. WHO’s Trial Registration Data Set (TRDS), modified for diagnostic accuracy

studies.

Item on TRDSa Modifications for diagnostic accuracy studies Examples from existing registered records of

diagnostic accuracy studies

1. Primary Registry and Trial

Identifying Number As defined for trials in WHO’s TRDS.

2. Date of Registration in Primary

Registry As defined for trials in WHO’s TRDS.

3. Secondary Identifying

Numbers As defined for trials in WHO’s TRDS.

4. Source(s) of Monetary or

Material Support As defined for trials in WHO’s TRDS.

5. Primary Sponsor As defined for trials in WHO’s TRDS.

6. Secondary Sponsor(s) As defined for trials in WHO’s TRDS.

7. Contact for Public Queries As defined for trials in WHO’s TRDS.

8. Contact for Scientific Queries As defined for trials in WHO’s TRDS.

9. Public Title As defined for trials in WHO’s TRDS.

10. Scientific Title

Item 1:

Include ‘diagnostic accuracy’ or one or more

accuracy measures in the title

(e.g. sensitivity, specificity, predictive value,

likelihood ratio, AUC).

“The diagnostic accuracy of exhaled breath

fingerprinting by eNose in diagnosing asthma

and atopy.”

(NTR1398)

11. Countries of Recruitment As defined for trials in WHO’s TRDS.

12. Health Condition(s) or

Problem(s) Studied

Item 2:

List the target condition that the diagnostic tests

should detect, or the target event to predict.

“Health condition(s) or problem(s) studied:

endometriosis”

(ACTRN12616001106426)

13. Intervention(s)

Item 3:

Index test: Provide an informative description of

the index test under evaluation. Include (if

applicable) the generic name and/or commercial

name.

“Device: VitalScan Magnetocardiograph. A

passive, non-contact, mobile medical device

that measures, displays, stores, and retrieves

magnetic fluctuations caused by heart activity at

a patient's bedside.”

(NCT02921438)

Item 4:

Reference standard: Provide an informative

description of the reference standard. Include (if

applicable) the generic name and/or commercial

name.

“Reference standard: Automated blood culture

technology, in place as standard NHS care in

microbiology laboratories at participating sites,

and performed prospectively as part of usual

clinical care.”

(ISRCTN97997760)

Item 5:

Index test: Describe the information available to the

performers or readers of the index test

(e.g. clinical information, other test results, results

of the reference standard).

“All patients will undergo both of these

diagnostic tests, though the order will be

randomised. Interpreting physicians of each

study will be blinded to the results of the other.”

(ACTRN12616001016426)

Item 6:

Reference standard: Describe the information

available to the assessors of the reference standard

“A second sonographer, blinded to all clinical

information and the primary sonographer's

ultrasound interpretation, will review the de-

identified ultrasound images remotely.”

26

(e.g. clinical information, other test results, results

of the index test).

(NCT02190981)

Item 7:

Index test: Describe the intended use and clinical

role relative to other tests in the existing clinical

pathway

“Current recommendations to rule out a heart

attack are based upon the patient having a

normal ECG (heart-tracing) and the absence of

a protein in the blood called troponin, which is

released when the heart muscle is injured.

Troponin is currently measured between 6 and

12 hours from Emergency Department

presentation. The majority of tests are normal

but this wait leads to excessive patient anxiety,

unnecessary prolongation of hospital stay and

ED overcrowding. The development of a

diagnostic pathway that allows more rapid

assessment of patients with chest pain and

prevents unnecessary hospital admission will

have a profound benefit on healthcare services.

This study aims to test the safety of a novel

diagnostic pathway for the rapid assessment of

patients with chest pain that allows the rapid

rule-out of heart attack. This pathway is based

upon the very recent development of a new

blood test that detects heart injury called the

high-sensitivity Troponin (hs-Tn) assay.”

(ISRCTN21109279)

14. Key Inclusion and Exclusion

Criteria

Item 8:

Setting: Describe the setting in which patients will

be recruited

(e.g. primary care facilities, university hospital’s

emergency department).

“Patients with clinical suspected pulmonary

embolism; Simplified Well's score >4

(pulmonary emblism) or D-dimer value

≥500ng/ml; Patients that undergo MCTPA

(Multidetector Computed Tomography

Angiography) in the Emergency Department for

suspected pulmonary embolism.”

(NCT01635257)

Item 9:

Participant recruitment: Describe on what basis

potentially eligible participants will be identified

(e.g. symptoms, results from previous tests, from

patient registries).

15. Study Typeb

Item 10:

Data collection: Describe how data will be

collected

(e.g. prospectively, retrospectively).

“Retrospective analysis of a quality

measurement project examining the quality of

vital parameter measurement in consecutive

patients ≥75 years presenting to the emergency

department.” (NCT01639430)

Item 11:

Participant sampling: Describe how participants

will be sampled

(e.g. a consecutive series, a random series, a

convenience sample, other).

16. Date of First Enrollment As defined for trials in WHO’s TRDS.

17. Target Sample Size As defined for trials in WHO’s TRDS.

18. Recruitment Status As defined for trials in WHO’s TRDS.

19. Primary Outcome(s) Item 12:

Study’s primary outcome measures:

“Sensitivity and specificity of the Thessaly test,

in determining the presence of meniscal tears,

27

*Name of the outcome: Include the index test(s)

whose diagnostic accuracy is evaluated, and the

target condition.

*Metric or method of measurement: Include (if

applicable) which estimates of diagnostic accuracy

will be calculated for the primary outcome

(e.g. sensitivity and specificity, or positive and

negative predictive values).

*Timepoint(s) of primary interest: ;

For prognostic and predictive tests, define the

follow-up period of interest

(e.g. events within 12 months after the index test

was performed);

For diagnostic tests, this can be at the time of the

index test or reference standard.

when employed by General Practitioners.”

(ISRCTN43527822)

“Accuracy (sensitivity and specificity) of

ultrasound scan as a tool for early detection of

superficial endometriosis.”

(ACTRN12616001106426)

“Predictive accuracy of copeptin, C reactive

Protein (CRP) and procalcitonin in terms of

developing organ failure, necrosis and/or

superinfection and mortality [Time Frame:

Duration of hospitalization]”

(NCT01293318)

20. Key Secondary Outcomes

Item 13:

Describe any pre-defined analyses of variability in

diagnostic accuracy

(e.g. across clinical subgroups).

“Sensitivity/specificity measures for the urine

test will be determined in subgroups of patients

who have a kidney transplant or a combined

pancreas/kidney transplant.”

(NCT01315067)

Item 14:

-Describe any other outcomes of secondary interest.

“Any adverse events reported by the patient in

the week following the procedure.”

(NCT02498041)

“Inter-rater agreement: The study will quantify

the rate of inter-rater agreement between the

clinician performing the point-of-care

ultrasound and a blinded reviewer of the

ultrasound images”.

(NCT02190981)

Table legend:

aWHO’s Trial Registration Data Set is available at http://www.who.int/ictrp/network/trds/en/.

WHO=World Health Organization.

bWHO’s Trial Registration Data Set also contains several required or optional data fields

related to the ‘study type’ (item 15), where only fixed options can be selected (e.g., whether

the study is ‘interventional’ or ‘observational’). Suggestions on how to address these for

diagnostic accuracy studies are provided in Supplemental Data 5.

28

Table 3. Completeness of registered records of diagnostic accuracy studies (n=30).

Item Description Information registered

1 Include ‘diagnostic accuracy’ or one or more accuracy measures in the title 27 (90%)

2 List the target condition that the diagnostic tests should detect, or the target event to predict 29 (97%)

3 Index test: Provide an informative description of the index test under evaluation. 30 (100%)

4 Reference standard: Provide an informative description of the reference standard. 19 (63%)

5 Index test: Describe the information available to the performers or readers of the index test 14 (47%)

6 Reference standard: Describe the information available to the assessors of the reference standard 12 (40%)

7 Index test: Describe the intended use and clinical role relative to other tests in the existing clinical pathway 18 (60%)

8 Setting: Describe the setting in which patients will be recruited 15 (50%)

9 Participant recruitment: Describe on what basis potentially eligible participants will be identified 22 (73%)

10 Data collection: Describe how data will be collected 14 (47%)

11 Participant sampling: Describe how participants will be sampled 11 (37%)

12 Study’s primary outcome measures 20 (67%)

13 Describe any pre-defined analyses of variability in diagnostic accuracy 2 (7%)

14 Describe any other outcomes of secondary interest 19 (63%)

29

Figure 1. Two online surveys formed the basis for this project.