RCT tutorial 1 RC tutorial David Verstraeten
Jan 05, 2016
RCT tutorial 1
RC tutorialDavid Verstraeten
RCT tutorial 2
Reservoir Computing
Random and fixedTrained
RCT tutorial 3
Steps in training and testing
• Create random weight matrices – Input to reservoir– Reservoir to reservoir– (Output to reservoir)
• Scale weight matrices globally to the desired value• Feed all input samples to the reservoir• Collect reservoir states• Train readout weight using linear regression or
ridge regression
RCT tutorial 4
Scaling reservoir matrix:Spectral radius
€
x[k +1] = Ax[k]+ Bu[k +1]
y[k +1] =Cx[k +1]+Du[k +1]
€
x[k +1] = f Wresresx[k]+Wres
inu[k +1]( )
y[k +1] =Woutoutx[k +1]+Wout
in u[k +1]
LTI system
RC system
€
ρ =max eig Wresres
( )( )Spectral radius
RCT tutorial 5
Training
M N P
Inputs Reservoir Outputs
T1
T2
TS
1
2
S
…
1
2
S
…
1
2
S
…
A B
RCT tutorial 6
Linear/ridge regression
A BWX =
€
wopt = (ATA)−1ATB
€
wopt = (ATA + λ I)−1ATB
Linear regression
Ridge regression€
wopt = argminwAw −b
2
€
wopt = argminwAw −b
2+ λ w
2
RCT tutorial 7
Pattern generation tasks: training
Fixed connections
Trained connections
RCT tutorial 8
Pattern generation: generation/testing
WarmupFreerunFixed connections
Trained connections
RCT tutorial 9
Leaky integrator neurons
RCT tutorial 10
Global optimization parameters
• Input scaling• Reservoir scaling (spectral radius)• Leak rate• Feedback scaling • Regularization parameter (through cross-
validation)• Not:
– connection fraction (only for spiking neurons)– reservoir size
RCT tutorial 11
RC toolbox
RCT tutorial 12
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 13
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 14
Rationale
• Reservoir Computing toolbox:• Box of tools for
– Quick-and-dirty experiments– Large scale parameter sweeps and optimizations
• Modular setup: function hooks for customization– Reservoir types– Learning rules– Scoring and evaluation methods
RCT tutorial 15
Getting started
• Download from http://snn.elis.ugent.be/rct
• Unpack, go to directory $RCT_ROOT and type:
>> install• For ‘Hello world’ experiment, type
>> rc_simulate
RCT tutorial 16
Usage
• Default settings– Are ok as starting point for many experiments– Contained in $RCT_ROOT/default_settings/default_*.m
– Look at these scripts! They give an overview of RCToolbox functionality
RCT tutorial 17
Custom settings• Two ways of overriding defaults:
– Command line>> dataset_generation_function=… @my_dataset_function();>> rc_simulate;
– Custom configuration file
>> custom_configuration=‘my_conf’>> rc_simulate;
– Example: analog_speech.m• Other examples: $RCTROOT/examples/configurations
RCT tutorial 18
Flow diagram for basic experiment
RCT tutorial 19
Structure of rc_simulate
RCT tutorial 20
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 21
Standard topology for classification/regression
InputInput BiasBias
ReservoirReservoir
ReadoutReadout
Mandatory
Optional
RCT tutorial 22
Connectivity matrix between layers
From/to Input Bias Reservoir Readout
Input 0 0 1 0/1
Bias 0 0 0/1 0/1
Reservoir 0 0 1 1
Readout 0 0 0 0
InputInput BiasBias
ReservoirReservoir
ReadoutReadout
RCT tutorial 23
Standard topology for classification/regression
Mandatory
Optional BiasBias
ReservoirReservoir
ReadoutReadout
RCT tutorial 24
Connectivity matrix between layers
From/to Bias Reservoir Readout
Bias 0 0/1 0/1
Reservoir 0 1 1
Readout 0 1 0/1
BiasBias
ReservoirReservoir
ReadoutReadout
RCT tutorial 25
Topology datastructure in the toolbox
• topology.– layer(1,n)– conn(n,n)– with n = number of layers
• Default settings: – $RCTROOT/default_settings/default_topology.m– Layer 1 : input– Layer 2 : bias– Layer 3 : reservoir– Layer 4 : readout
RCT tutorial 26
• topology.layer(1:n).– size: integer– is_trained_offline: boolean– is_trained_online: boolean– node_type: string– training_function:function pointer– nonlin_functions: array of function pointers
Topology datastructure in the toolbox
RCT tutorial 27
• topology.layer(1:n).– init_simulation: function pointer– finish_simulation: function pointer
– dt: float– regul_param: float– scoring: function pointer
Topology datastructure in the toolbox
RCT tutorial 28
• topology.conn(1:n,1:n).– is_active: boolean– creation_pipeline: array of function pointers
– scaling_factor: float– conn_frac: float [0,1]– discrete_set: float array
Topology datastructure in the toolbox
RCT tutorial 29
Topology creation pipeline
• Array of function pointers which pass weight matrices
• Generation functions– @gen_rand, @gen_1D, @gen_load
• Assignment functions– @assign_rand, @assign_discrete
• Scaling functions– @scale_specrad, @scale_constant
RCT tutorial 30
Topology creation pipeline
• Example (default for reservoir to reservoir connection):– topology.conn(3,3).creation_pipeline= {@gen_rand,
@assign_randn, @scale_specrad}
• Can be customized! Function hooks
RCT tutorial 31
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 32
Dataset generation
• Function hook• Default
– dataset_generation_function=@dataset_narma_10(input_dt, input_maxtime, n_samples)
• Signature– function [inputs outputs] = my_dataset_function(…)– inputs = cell(1,nr_samples)
• inputs{:} = matrix (M,:);• M = input dimensionality
– outputs = cell(1,nr_samples)• outputs{:} = matrix (P,:) or [] for signal generation tasks.• P = output dimensionality
• Example: 10th order NARMA
RCT tutorial 33
Data struct
• data.layer(1:n).– r : required/teacher forced states– s : simulated states
RCT tutorial 34
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 35
Training and testing
• cross-validate – Training and testing using cross-validation
• cross_validate_grid- Training and testing using cross-validation with
optimization of regularization parameter
RCT tutorial 36
Cross-validate• Example: three-fold cross-validation with
three samples.
11
22
33
TrainingTraining
TestingTesting
ResultsResults11
22
33
error fold 1
error fold 1
Fold 1
RCT tutorial 37
Cross-validate• Example: three-fold cross-validation with
three samples.
11
22
33
TrainingTraining
TestingTesting
ResultsResults11
22
33
error fold 1
error fold 1
Fold 2
error fold 2
error fold 2
RCT tutorial 38
Cross-validate• Example: three-fold cross-validation with
three samples.
11
22
33
TrainingTraining
TestingTesting
ResultsResults
11
22
33
error fold 1
error fold 1
Fold 3
error fold 2
error fold 2
error fold 3
error fold 3
RCT tutorial 39
Cross-validation
• Function hook:– train_params.cross_val_set_function– Default: @random_cross_val– Other possibilities:
• cross_val_only_training: no test set• no_cross_val: simple train and test set• random_freerun_cross_val : cross-validation for freerun tasks
• Freerun cross-validation
11
22
TrainingTraining
TestingTesting
11
22
33 33
3
3 3 Fold 1
…
RCT tutorial 40
Cross-validate with gridsearch• For optimization of regularization parameter
TrainingTraining
TestingTesting
ResultsResults
1133
error fold 1
error fold 1
Fold 1
ValidationValidation
22
11 22 33
Val. results
Val. results
error error
error error
error nerror n
…
Subfold 1
22
11
Subfold 2
n opt error fold 2
error fold 2
error fold 3
error fold 3
Fold 2Fold 3
RCT tutorial 41
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 42
Parameter sweeps• parameter_ranges = struct( 'name', {}, 'range', {});
• Example– 1D sweep:parameter_ranges = struct(
'name', {‘topology.layer(3).size’},
'range’, {10:10:100});
– 2D sweep (all combinations!):parameter_ranges = struct(
'name', {‘topology.layer(3).size’, ‘topology.layer(1).scale_factor’},
'range', {10:10:100, .1:.1:1});
RCT tutorial 43
Parameter sweeps
• Results are saved in output_directory• Set save_results=true!
• Plot results of sweep – plot_error(output_directory)
• Get variables from simulation– get_saved_data(output_directory, variables)
RCT tutorial 44
Custom scripts
• Toolbox is: box of tools you can use to build your own experiments
• Good starting points:– $RCTROOT/examples/tutorial/tut1.m– $RCTROOT/examples/framework/rc_simulate_job.m
RCT tutorial 45
Questions