CRISP evaluation using the STEFi approach

CRISP Evaluation

16th of March 2017, CoU Meeting, Brussels

Dr Nathalie Hirschmann (Center for Technology and Society – Technische Universität Berlin)

EVALUATION

CRISP Methodology

Source: Deliverable 5.2. February 2016

Information provider

Audit and Inspection

review & decision

Auditor (third party)

EVALUATION CERTIFICATION

SurveillanceAttestationS-T-E-FiAssessment

Configuration selection and determination

R3R2R1

2

Systematisation of criteria

Video surveillance

3

performance

accuracy

robustness

fair distribution

accountability

withdrawal mechanisms

end user safety

system interference

risk

user error protection

environment

awarenesstransparent use

feeling of uneasinessphysiological invasiveness

reliability

maintenance resources

usability

ethical codes

good practice/safety codes

documentation protective measures to ensure efficiencyusability

user manual

customisation

interoperability

lifecycle costs

energy efficiency

personal data

prohibition of discrimination

due process

Systematisation of criteria

Video surveillance

4

performance

accuracyrobustness

SECURITYfair distribution

accountability withdrawal mechanismsend user safety

system interferencerisk

user error protection

environment

awarenesstransparent use TRUST

feeling of uneasiness

physiological invasiveness

reliability

maintenance resources

usability

ethical codesgood practice/safety codes

documentationprotective measures to ensure efficiency

usability

user manual

customisationEFFICIENCYinteroperability

lifecycle costsenergy efficiency personal data

prohibition of discrimination

due process

FREEDOMINFRINGEMENT

Video surveillance

5

Police authority

Private securit

yPolitical actor

…

SECURITYCitizen

Employee

Political actor

…

TRUST

Municipality

Political actor

…Infrastructure operator

EFFICIENCYData protection expert

NGO‘s

Political actor

…Lawyer

sINFRINGEMENT

FREEDOM

S-T-E-Fi experts

‘Information provider’

S-T-E-FI ASSESSMENT

CONFIGURATION          

R1 R2

CERTIFICATION

10

Evaluation process

Video surveillance

‘Information provider’

S-T-E-FI -ASSESSMENT

CONFIGURATION          

R2

1. Specification of the security area

Client informs about:

2. Detailed technical specifications of the system

3. Basic information on at least three application scenarios

11

Evaluation process – configuration

Video surveillance

1. Reliable answering of all assessment questions by the project leader and ‘appointed experts’

2. Identifying conflicts between and within S-T-E-Fi dimensions

3. Conflict resolution

R 1

‘Information provider’

S-T-E-FI -ASSESSMENT

12

Evaluation process – link between configuration and assessment

Video surveillance

R 1

13

Evaluation process – assessment

Video surveillance

‘Information provider’

S-T-E-FI -ASSESSMENT

R1

1. Reliable answering of all assessment questions by the project leader and ‘appointed experts’2. Identifying conflicts between and within S-T-E-Fi dimensions

3. Conflict resolution

R 2

14

Assessment: report R2

R 2

Summary of how the process was conducted

Range of stakeholders involved

Information gathered during configuration

Selected assessment questions

Results of the assessment questions

Type and number of identified conflicts

Conflict resolution

Potentially remaining, unsolved conflicts

‘Information provider’

S-T-E-FI -ASSESSMENT

CONFIGURATION          

R1 R2

1. Specification of the security area

Client informs about:

2. Detailed technical specifications of the system

3. Basic information on at least three application scenarios

CERTIFICATION

15

From evaluation to certification

Video surveillance

1. Reliable answering of all assessment questions by the project leader and ‘appointed experts’2. Identifying conflicts between and within

S-T-E-Fi dimensions3. Conflict resolution

18

Assessment criteria – example

“Yes” or “No” Is the personnel trained

to increase public awareness on the

possible impacts of the system?

What kind of training does the personnel

receive and how often?

QUESTIONS:

T R U S TAwareness, Training

e.g. “Our personnel is trained on the system in the field every year. This is recorded in the personnel files.”

P R O O F

19

Conflict identification – example

RESPONSE OPTIONS NO YES

NO C- (0) C+ (1)

YES C- (0) C- (0)

Criterion A: ‘Observability’ (T)‘Are people constantly observed by the system?’

Criterion B: ‘Transparency’

(T)‘Is the system clear on what it offers?’

C – (0): no conflict; C+ (1): conflict; needs to be addressed.

24

Identification of conflicts

25

Conflict resolution

1. Ensuring compliance with applicable legal requirements

2. Implementation of technical changes to the security system and/or implementation of changes to the applied operating procedures

3. Negotiating a solution by following a practical and legitimate decision-making process

participation & systematisation: enabling/encouraging early stakeholder participation; assessment criteria are systematised according to four dimensions

systemic view: transferring assessment criteria into a matrix structure; identifying conflicts between/within dimensions and making them visible

conflict resolution: indicating identified conflicts which need to be solved to involved parties in order to move on with certification

S

Fi

T

E

27

CRISP’s approach

28

For further questions, please contact:

Dr. Leon HempelZentrum Technik und Gesellschaft – TU BerlinHardenbergstr. 16-1810623 Berlin – [email protected]