© CSIR 2009 Measuring library leadership and competence at the CSIR Martie van Deventer Portfolio Manager: CSIR Information Services CICD Winter Seminar,

© CSIR 2009

Measuring library leadership and competence at the CSIR

Martie van Deventer

Portfolio Manager: CSIR Information Services

CICD Winter Seminar, Pretoria: NLSA, 29 June 2009

© CSIR 2009

Roadmap

• What do we regard as competence?

• How do we measure @ CSIR

• Indicators of technical competence

• Leadership competence

© CSIR 2009

Competence

• > knowledge about a subject• > skill & experience in the field• > behaviour & attitude• Combination ~ phases of

competence (beginner – seasoned professional)

• Brain surgeon fainting @ scene of accident … competent?

© CSIR 2009

Library schools … do they provide knowledge or competence?

• Information and knowledge management• Information seeking, searching and retrieval• Knowledge organization and representation• Management• User studies• Information literacy• ICTs• Ethical, legal and economic aspects of information• The information / knowledge society and globalisation• … and other academic courses

Source: Bothma & Britz, 2007

© CSIR 2009

Library schools … cont• Within (L)IS programmes

– Focus on (Information and) Knowledge Management– Computer trouble shooting skills– Multimedia and web development– Media and Publishing Studies– ICTs– Records management– Archival studies

• As new programmes– Multimedia– Publishing studies– Archival studies

Source: Bothma & Britz, 2007

© CSIR 2009

CICD-type progammes … do they provide knowledge or competence?• Strive for continuing education &

professional development• Promote lifelong learning• Intend to act as facilitator between

employer and employee• All these are essential … but … where &

when does experience & skill enter the equation?

© CSIR 2009

Where can we go to gain the necessary competence to …

• Make our clients happy?• Make us happy?

– Turn us into true leaders?– Comfortable in a virtual world?– Embed our services?– Support open access: Install institutional repositories, publish

online, using Web 2.0 or other appropriate tools - not as toys but with the explicit goal to bring about fame?

– Simultaneously be IP astute?– Smart at negotiating contracts?– Curate digital content?– Collaborate in international teams?

© CSIR 2009

Measuring competence is difficult• Often need to use indicators of competence• CSIR measures portfolio/unit performance

against indicators– Life isn’t fair – the CSIR cannot compensate for past

wrongs!– Perceptions rule – the CSIR cannot fix perceptions

about the library profession!

• We, in CSIRIS, appoint against competence– Interview for knowledge & experience– Practical evaluation to test skill & attitude

© CSIR 2009

CSIR measuring performance

• Weighted scorecard system (Key Performance Indicators (KPIs))– Set targets approved by Executive

• Elements measured– Operational service delivery– Financial sustainability– Customer satisfaction– HR & staff transformation– Corporate governance & citizenship

• Progress measured quarterly (KPIs)• Annual evaluation (KPIs) • Performance measured six monthly (Key

Result Areas (KRAs)) … competence?– Measured against negotiated set targets

© CSIR 2009

Evaluation dashboard 2009

Executives happy!

Customer satisfaction (20%)

> 80%

Financial sustainability (10%)

+ Balance + R100k external

HR & staff transformation (10%)

> 50% black professionalsStaff development

Corporate governance & citizenship (10%

BBEEE purchasingSaving on electricitySocial responsibility

Operational service delivery (50%)

TOdB reliable publication equivalent statistics> Repository contentRecords Management> Moving to eLibraryeJournal usage increasingCuration strategyTraining excellence

© CSIR 2009

Executives are happy when …• We save them money• We make the organization famous!

• We save their time, and

• We make them famous!

Researchers are happy when …

Measuring impact

Make me famous!

Save me time!

Save me more time!

Source: Botha, Erasmus & Van Deventer, 2009

© CSIR 2009

Indicators of financial competency

© CSIR 2009

Ensuring that information products & resources provide value for money

Product List Price Offer to CSIR Final negotiated price Savings Rand

£6886 (R110705) $13,496.56 (R134960) R 24,255R 193,374.00 R 174,211.00 R 19,163.00 R 19,173

$12600 $10000 $2600 R 26,000

Business Insights Free. Value approx R42000 R 42,000

$18480 R 192,892.00 R 1,886

Wiley $10746 $1161.72 R 11,617

R 14,434.00

R 20,000.00$5322 (R40028) $10260 R 72,524

€32000 2008/9 €12000 €20000 R 229,820.00Total R 461,709

Blackwell (Refusal to pay extra because of a publisher change to US$ invoicing)Sabinet SACatRefWorks

Technology Section: approx £2800 ?

Free in addition to current subs if we continue (cannot give discount)

EbscoHost

R194778.50 9 (R10.54/$). Asked for $ invoice when R wasapprox R9.60$11907.72. Picked up incorrect calculations

WorldCat and Articles First.

Cancelled – WorldCat available free on Internet. Enough article databases

SA ePublicationsNot cost effective. SAJS now free

SPIE Digital Lbrary (Aug) 2008/9: $15582 (R112552)Beilstein (Aug)

© CSIR 2009

Value for money

2006-2009 Researchers Downloads Downloads per researcher

Oct-06 815 60 562 74Mar-07 849 92 598 109Mar-08 788 141 428 179Mar-09 722 140 752 195

Equals: 195 x $25 = $4 860 (R46k) value per researcher or R11.30 vs $25/ R35 per download

Technical competencies required

• Negotiation – we deal with sharp businessmen who are selling their product

• Statistical analysis - to check if investments where correctly made

• ROI investigations • Report writing• Marketing and promotional activities to encourage usage• Technical installations of products• Managing contracts that cost the organisation in excess

of R1m each• Better searches on these products than what the

average information literate person can do?

© CSIR 2009

Indicators of client satisfaction

© CSIR 2009

User education & training to enable our clients

0 1 2 3 4 5

A ve ra ge

B 1 : D ea ling w ith E nqu iriesB 2 : B ook ing p roc es s

B 3 : P re -even t c om m un ic a t ion (s u ffic ien t / c lea r / t im e ly )C 1 : C om prehens ivenes s

C 2 : C ha lleng ingC 3 : U s e fu lnes s to y ou r job

C 4 : S u ffic ien t lea rn ing by do ing ex e rc is esC 5 : The even t c on ten t w as p res en ted a t a leve l w h ic h

C 6 : D iffic u lt c onc ep ts w ere ex p la inedC 7 : The ideas fi t ted in to a m ean ing fu l p ic tu re by the

D 1 : W ork book /handou ts qua l i tyD 2 : E x e rc is es and c as e s tud ies

D 3 : V ideos and trans parenc iesD 4 : S o ftw are and s y s tem s

D 5 : The m ate ria ls c on tribu ted to m y unders tand ingD 6 : The m ate ria ls w ere w e ll o rgan is ed

E 1 : K now ledge o f s ub jec tE 2 : P reparednes s

E 3 : P res en ta t ion s k i l lsE 4 : C la ri ty o f ins truc t ions / ex p lana t ionsE 5 : F eedbac k on ex e rc is es / ques t ions

E 6 : C rea t ions o f g roup rapport (" vibe " )E 7 : E nc ouragem en t o f g roup pa rt ic ipa t ion

E 8 : The fac i l i ta to r as k ed ques t ions tha t m ade m e th inkE 9 : The fac i l i ta to r s t im u la ted m y in te res t in the c on ten t

F 1 : C om fo rtF 2 : L igh t ing

F 3 : R e fres hm ents and m ea lsF 4 : A ud io vis ua l equ ipm en t

G 1 : O ve ra l l ra t ing o f the lea rn ing even t

Qu

es

tion

s

Overall evaluation Access to Peer Review ed Scientific Publications

01/04/08 - 31/03/2009

Technical expertise

Tracking customer satisfactionInformation specialists client satisfaction, Apr 08-Mar 09

86

87

88

89

90

91

92

93

94

95

A pr M ay Jun Ju l A ug S ep O ct N ov D ec/Jan F eb M ar

% S er ies1

86%

95%

© CSIR 2009

Technical competencies behind the scene• Ability to do significantly higher quality literature

searches, faster than they can - knowing both the product and the search strategies, having both skill and experience

• Develop training that incorporates knowledge about learning styles and end user behaviour

• Ability to utilise a variety of technologies to assist with skills transfer

• Provide guidance and develop tips and tricks to assist with information overload

• Provide guidance and lead others to work faster and smarter

© CSIR 2009

Indicators of operational excellence

© CSIR 2009

Bringing fame to the institution & our researchers

Records added 2007/8

.pdf down- loads 2007/8

Records added 2008/9

.pdf down- loads 2008/9

Total 1,349 113,327 1,075 340,616

I tem s in the repos ito ry : 2,424T o ta l num ber o f .pd f f i les dow n loaded : 453,943

Showing usageResearch Space Financial Year comparison

0

5 ,000

10 ,000

15 ,000

20 ,000

25 ,000

30 ,000

35 ,000

40 ,000

Apri lM

ay

JuneJu ly

A ugust

S ep tem

ber

O ctobe r

N ovembe r

D ecembe r

Janua ry

Feb rua ry

Marc

h

Month/Year

Nu

mb

er o

f it

ems

2007 /8 T o ta l no o f P D F s v iew ed

2008 /9 T o ta l no o f P D F s v iew ed

© CSIR 2009

Technical competencies required …

• Indexing of internationally acceptable standard• Open source coding to address unique needs• Statistical analysis of usage• Curation of repository content• Repackaging of content• Due diligence in terms of legal requirements• Training so that others could effectively use and

utilise the facility• Enhancements to the open source products we

are using

© CSIR 2009

Leadership KRAs

• Collaboration (internal & external)

• Partnerships (internal & external)

• Managing service enablers

• Staff development & satisfaction

• Guiding the organization

• New developments in eResearch enablement (curation, collaboration, information infrastructure)

• Records management

• Open access

NeDICC: attempting to collaborate with the USGS on establishing a WDCBHH in SA

CSIR/UP partnership: looking at the components of VREs

Source: Van Deventer & Pienaar, 2009

© CSIR 2009

• I am not convinced that we, as a profession, really do understand the new competencies for high quality, service … relevant to our stakeholders … or that we have a place to go for help when we need practical assistance!

• I do not know how to measure if new recruits are rapidly acquiring essential competencies.

• I need an independent qualification authority to assist when it comes to identifying leadership competence.

• I know it is time to increase the pace … and we are doing the best we can!

• If you can help … speak to me!!

Finally

© CSIR 2009

References• Botha, E., Erasmus, R. & Van Deventer, M. 2009. Evaluating the impact of a Special Library and Information

Service. Journal of Librarianship and Information Science, Vol. 41, No. 2, 108-123 (2009). Available: http://lis.sagepub.com/cgi/reprint/41/2/108 (Accessed 6 June 2009)

• Bothma, T & Britz, J. 2007. Collaboration amongst (L)IS Schools in South Africa. Presented at the World Library and Information Congress (WLIC): 73rd IFLA General Conference and Council, 22 August 2007. Education and Training Section, University of Kwazulu-Natal Library. URL: http://www.ifla.org/IV/ifla73/index.htm (Accessed 6 June 2009)

• Frame, M. 2009. World Data Center for Biodiversity & Human Health: Infrastructure Requirements of a National Informatics System (edited version)

• Satgoor, U. 2005. CICD & Library schools: partnership for LIS CEPD. RETIG Indaba. 28 February. Available: http://www.infs.ukzn.ac.za/cicd.doc (Accessed 6 June 2009)

• Van Deventer, M. 2009. Knowledge Retention. Presentation to: Pretoria Chapter of the Knowledge Practitioner’s Group. Pretoria.

• Van Deventer, M & Pienaar, H. 2009. Virtual research environments: learning gained from a situation and needs analysis for malaria researchers. 2nd African eScholarship and digital curation conference. Pretoria: 12-13 May. Available: http://www.library.up.ac.za/digi/docs/mvdeventer_paper.pdf (Accessed 6 June 2009)