© University of Reading 2007 Dr Liam J. McGuffin RCUK Academic Fellow [email protected] 20 April 2014 McGuffin Group.

April 10, 2023 © University of Reading 2007www.reading.ac.uk/bioinf

Dr Liam J. McGuffinRCUK Academic [email protected]

McGuffin Group Methods for Quality Assessment

Three methods for different categories:

• ModFOLD v 1.1 – Server, QMODE1

• ModFOLDclust – Server, QMODE2

• ModFOLD v 2.0 – Human, QMODE1

(now a server, QMODE2)

2To put your footer here go to View > Header and Footer

ModFOLD v 1.1 (Server)• Combines 6 QA scores using a Neural Network (4

scores in CASP7)

• Considers models individually

• Trained using TM-scores and fold recognition models

• Outputs a single score for each model (QMODE1)

TM-score

SS (new)

SS-weighted (new)

ModSSEA

MODCHECK

ProQ-MX

ProQ-LG

3To put your footer here go to View > Header and Footer

ModFOLDclust (Server)• Simple clustering method - unsupervised• Compares all sever models against one another• Outputs overall score plus per-residue accuracy

(QMODE2)

2

0

i

i

dd

1

1S

2. Per-residue accuracy -Mean S-score rearranged to give distance in Angstroms

Aa

iar S1N

1S

Si = S-score for residue idi = distance between aligned residues according to TM-score superpositiond0 = distance threshold (3.9) Sr = predicted residue accuracy for the modelN = number of models A = set of alignmentsSia = Si score for a residue in a structural alignment (a)

1

S

1dd

r0r

1. Overall/global model quality -Mean TM-score between models(Similar to 3D-Jury)

Mm

mT1N

1S

S = quality score for modelN-1 = number of pairwise structural alignments carried out for model M = set of alignmentsTm = TM-score for alignment of models

4To put your footer here go to View > Header and Footer

ModFOLD v 2.0 (Manual)• Combines ModFOLD scores, ModFOLDclust score and

initial server ranking using a NN• Considers models individually (sort of)• Compares each model against 30 nFOLD3 server

models to get a ModFOLDclust score (server version)• Per-residue accuracy from ModFOLDclust method

(server version)

TM-score

SS (new)

SS-weighted (new)

ModSSEA

MODCHECK

ProQ-MX

ProQ-LG

Server rank (new)

ModFOLDclust (new)

5To put your footer here go to View > Header and Footer

Observed quality (GDT-TS)

Pre

dic

ted q

ualit

yModFOLDclust – all TS1 models

Pre

dic

ted q

ualit

y

Observed quality (GDT-TS)

ModFOLD 2.0 - all TS1 models

ModFOLDclust – T0498

Observed quality (GDT-TS)

Pre

dic

ted q

ualit

y

ModFOLDclust – T0499

Observed quality (GDT-TS)

Pre

dic

ted q

ualit

y

6To put your footer here go to View > Header and Footer

Correlation of output with GDT-TS

Method Kendall (Tau)

Spearman (Rho)

Pearson (R)

ModFOLDclust

0.76 0.91 0.92

ModFOLD 2.0 0.74 0.90 0.91

ModFOLD 1.1 0.52 0.71 0.71

Wilcoxon signed rank sum tests

(H0 = GDTx ≤ GDTy, H1 = GDTx > GDTy)

ModFOLDclust

Zhang-Server

ModFOLD 2.0

pro-sp3-TASSER

ModFOLDclust 1.000 0.181 0.147 0.000

Zhang-Server 0.820 1.000 0.162 0.000

ModFOLD 2.0 0.854 0.839 1.000 0.000

pro-sp3-TASSER 1.000 1.000 1.000 1.000

Results continued…

• ModFOLD 1.1:• Increase in average per-target correlation since CASP7?• Decrease in global correlation? But diff. data sets.

• ModFOLD 2.0:• Fewer outliers but no significant difference from

ModFOLDclust • Benchmarking on CASP7 set showed an increase in Kendall’s

Tau (not significant, training artefact?)• ModFOLDclust:

• Most simple & effective method, but CPU intensive• Still room for improvement, doesn’t consistently recognise

best model• Marginally better than Zhang-Server in terms of cumulative

GDT-TS, but difference is not significant

Conclusions

7To put your footer here go to View > Header and Footer

The ModFOLD server

Method Relative speed

Upload options

Output mode

ModFOLD 1.1

Fast Single and multiple

QMODE1

ModFOLDclust

Slow Multiple only

QMODE2

ModFOLD 2.0

Medium

Single and multiple

QMODE2

http://www.reading.ac.uk/bioinf/ModFOLD/

[email protected]

References:• McGuffin, L. J. (2008) The ModFOLD Server for the Quality

Assessment of Protein Structural Models. Bioinformatics, 24, 586-7.

• McGuffin, L. J. (2007) Benchmarking consensus model quality assessment for protein fold recognition. BMC Bioinformatics, 8, 345.