GEOB 409 Field Measurements and Data AnalysisIntroduction to data analysis! 7! Dimensions.! Strictly speaking, we sample any physical, chemical or biological variable φ in a four

Koppes / Geography 409 Introduction to data analysis

GEOB 409��Field Measurements and Data Analysis��

1


2

A classical research approach.

Documentation ‘Hands-on’ Creative Part

Hypothesis, motivation, possibilities

Experiment design��Analysis plan

Conclusions, hypothesis verification / falsification

Project Proposal Information

Lab- and field measurements

Experiment documentation

Des

ign

Expe

rim

ent

Ana

lysi

s Data analysis

Data

Analysis documentation

Report


3

Field data in physical geography.

Any natural environment is a complex, multivariate web of interacting variables.

We can never measure ��continuously everything, ��everywhere.

We sample selected data��with an appropriate ��strategy.


4

Error in Scientific Measurement means the Inevitable Uncertainty that attends all measurements��-Fritschen and Gay

Uncertainties are ubiquitous and therefore no reflection on the usefulness of the measurement or the competence of the measurer

If we are to rationally use a measurement the uncertainties must be known quantitatively


5

Data-sets and data analysis – keep it simple!

UCAR Digital Image Library


6

Boolean data Continuous data Classification

-5.24 -3.20 2.32 9.63

14.15 17.83 27.08 16.34 10.22 4.43

-1.32

Cold Cold Cool Cool Warm Warm Hot Warm Cool Cool Cold

Yes Yes No No No No No No No No Yes

What will your data look like? - Data format da

ta d

imen

sion

data

dim

ensi

on

data

dim

ensi

on


7

Dimensions.

Strictly speaking, we sample any physical, chemical or biological variable φ in a four dimensional setting, i.e. as a function of time t and space x (x, y, z).

Luckily, quite often we are not interested in all four dimensions. Likely, we focus on a single or two dimensions due to logistical reasons or because your study object allows this. You might assume that variability in one dimension is much smaller than in another one (homogeneity, stationarity critearia).

→ In your project you will implicitly include certain dimensions / exclude others.


8

Dimensions.

Basic dimension Data-set Resolution Examples of 1-D data

Time Time series Temporal resolution

• One day of 10 min temperature measurements at a climate station • One year of hourly discharge measurements from a stream

Space

Horizontal profile Spatial resolution

• Vegetation classification along a traverse. • A horizontal profile of snow water content along a line.

Vertical profile Spatial resolution

• A tethered balloon run measuring wind with height. • Temperature change in soil with depth.

(Frequency)* Spectrum Spectral resolution • A histogram of different grain sizes in sediments • Irradiance in different wavelengths

*This is strictly speaking not a basic dimension, but a transformation of time or space


9

Examples of data sets - time series.

time dimension

Example: Carbon dioxide in a forest as a function of time of a day


10

Examples of data sets - horizontal profiles.

Example: Horizontal transect through Vegetation

Example: Horizontal transect showing air temperature

horizontal dimension horizontal dimension

Ecosystems of BC / T.R. Oke (1987): 'Boundary Layer Climates' 2nd Edition.


11

Examples of data sets - vertical profiles.

vert

ical

dim

ensi

on

vert

ical

dim

ensi

on

Univ. Stuttgart / T.R. Oke (1987): 'Boundary Layer Climates' 2nd Edition.


12

Examples of two dimensional data sets.

time-space Example: temperatures in a lake as a

function of time of year and water depth

yearly course (time)

space-space (map) Example: land use

hori

zont

al d

imen

sion

horizontal dimension ve

rtic

al d

imen

sion

time dimension


13

Resolution.

• Temporal and spatial resolution: How many data-points per unit of a dimension? Temporal resolution and spatial resolution, i.e 1 measurement a day vs. 1440 measurements a day, or 1 measurement per km vs. 1000 measurements per km.

• Data depth: How accurately can we distinguish between different physical values, i.e. 0.02 vs. 0.0214523.

Illustration: Wikipedia


14

Gridded vs. irregular data

irregular regular

Voronoi tessellation

. Data Points

Nor

th (

spac

e)

East (space)

Nor

th (

spac

e)

East (space)


15

Example of a regular grid in vegetation studies.

Photo: http://www.marine.gov/


16

Integration and interpolation.

Integration refers to the process of combining or accumulating - or more generally to methods of upscaling - data from an existing set of measured data points. Interpolation refers to the process of splitting down or fill-in data to constructing new data points - or generally to methods of downscaling - an existing set of measured data points. Both can be done in time and space domains, and there are various methods.


17

Your data set?

Choose a physical parameter or a classification of interest in your potential project: Data format? Dimensions? Resolution? Regular or irregular? Assumptions?


18

Data processing.

• Correcting sensors with data from lab calibrations or field intercomparisons.

• Plausibility checks - define criteria for errors, experiment disturbances, etc.

• Flag data - remove data that fulfill the above criteria (never delete data forever, just flag it - and backup raw data!).

• Integrate or interpolate data - only if your data are not at the scale required, or if you have to compare two data sets with different resolutions.

• Select data for further analysis if you have made assumptions to fulfill certain criteria.


19

Check your data! Potential approaches?


20

Check your data - potential approaches.

• Global criteria ��(minimum, maximum, ...).

• Local criteria��(rate of change, ...).

• Statistical criteria.

• Manual data flagging.

Standard deviation

CO2-concentration


21

Analysis tools

• Describe data distribution - statistical probability of occurrence, histograms, statistical moments, ...

• Find events - peak detection, integration, ...

• Find and quantify correlations (same variable at different locations, same variable at compared different times, between two variables, correlation between model and measured values) - correlation, regressions, curve-fitting, statistical tests

• Find groups and dominating dimensions - Clustering, principal component analysis,

• Find process dominating scales - Spectral analysis finds process dominating time and length scales, wavelets.


22

Data analysis tools. Method / System Advantage Limitations Example of a system

Manual Analysis very simple, fast up to a few 10s of data-points, no large data-sets, no modelling

Calculator, paper, pen...

Spread-sheet software

simple analysis and graph tools

Limited # of data points, limited statistics, modelling, and automation & slow.

Microsoft Excel

GIS system complex spatial analysis and modelling

Expert knowledge. Expensive.

Workstation with ArcGIS

Statistical software and programming languages

complex and fast time series analysis, automation, modelling

Programming skills. Expensive.

Workstation with Matlab, R, IDL ...


23

Your data analysis?

Think about your potential project: Instrumentation, calibrations? Data checks? Analysis concept? Software needs? Hardware needs? again: Keep it simple!


24

Documentation : Meta-data

• Document your ideas and plans → proposal.

• Document your instrumentation (specifications, manufactures, serial numbers).

• Document your sampling strategy (how, where and when...)

• Document your data files (parameter, units, time-zone, location)

• Document your analysis (data filtering, data selection criteria, statistical methods)

• Document and prove your conclusions → report

log- or field note book

GEOB 409 Field Measurements and Data AnalysisIntroduction to data analysis! 7! Dimensions.! Strictly speaking, we sample any physical, chemical or biological variable φ in a four

Documents