From modelling guidelines to model portfolio management

From modelling guidelines to model portfolio management

• THORSTEN ARNOLD, GREY BRUCE CENTRE FOR AGROECOLOGY

• JOSEPH H.A. GUILLAUME, AUSTRALIAN NATIONAL UNIVERSITY

• TUOMAS LAHTINEN, SYSTEMS ANALYSIS LABORATORY, AALTO UNIVERSITY

• R. WILLEM VERVOORT, SYDNEY INSTITUTE OF AGRICULTURE, THE UNIVERSITY OF SYDNEY

EMS Paper, Jan 2020

What is a Data Management Strategies (DMS), and why do organizations have one?

DMS: “a defined process for handling the data within an organization.”

The DMS ensures that data … is stored, categorized and standardized using defined & known frameworks

… can be accessed and processed in a manner required by the organization

… data use is assured, monitored, and safe.

What is aModel Management Strategy (MMS),

and why would an organization have one?

….

Lessons & experiences on Model Management

Operations Research

Earth Systems modelling agencies

Hydro-meteorological real-time forecasting and other research initiatives

Private engineering firms

Academia-driven data and model hubs

National-scale authoritative models

Ontario’s drinking water protection program

Ontario’s Oak Ridges Moraine Groundwater Program (ORMGP)

http://data.geus.dk/geusmap

What is a model application?

Monitoring data

Assumptions and scientific theories

Software code

Input dataSoftware

procedures

Output data Interpretable results

Software procedures

Knowledge & skills, documentation

MMS must meet needs ofhigh-level managers/boards

Generality

Simplicity

Practicality

Objectives of a MMS(1) Recognize broad model functions

1. Regulatory requirements or core organizational tasks

2. Organize and share existing information

3. Prioritize the collection of new information

4. Communication support

Objectives of a MMS

(2) Respect operational workflows

Objectives of a MMS(3) Allow governance

Three leverage points:

1. Procedures and standards

2. Knowledge and skills

3. Cyberinfrastructure

Objectives of a MMS(4a) Govern effective leverage points

1. Procedures and standardsa. Standards

b. Guidelines & procedures

2. Knowledge and skills

3. Cyberinfrastructure

Objectives of a MMS(4b) Govern effective leverage points

1. Procedures and standards

2. Knowledge and skillsa. Understanding needs for knowledge & skills

b. Existing capacity

c. Strategies to access knowledge and skills

3. Cyberinfrastructure

Objectives of a MMS(4c) Govern effective leverage points

1. Procedures and standards

2. Knowledge and skills

3. Cyberinfrastructure

1. Hardware

2. Data

3. Software code

4. Workflow management

Example/experience:

Guidelines for actively managing numerical models at watershed-scale in Ontario

Ontario’s context Modelling mostly performed by consultants on behalf of agencies

Model custodian: staff shared between multiple agencies

Define:

Standards for reporting, file naming, directory structures,

Contract templates and standard clauses

Advise:

problem definition,

project scoping,

RfPs

Supports contracting, review deliverables

Long-term storage, access, archiving

https://oakridgeswater.ca/publicationsite

Summary & conclusion

Towards explicit strategies for managing model portfolios.

Largely overlooked task of public agencies that utilize models for policy support

MMS ensures / improves

Longevity and reusability

Legal defensibility and democratic transparency

Cost-effectiveness and scientific robustness

Respect organizational contexts

Organizational setups (partnerships, etc)

Capacity (management & staff)

From simple (e.g. standardizing file naming, directory structure & archiving) to highly involved (e.g. in-house software code development)

”“ For adaptive management we can

strategically build knowledge infrastructure, with models as essential tools.

THANK YOU FOR YOUR ATTENTION!

[email protected]

ThorstenArnold.com

EMS Paper, Jan 2020

[email protected]

ThorstenArnold.com