13 An example of a complete agent architecture that is able to answerquestions discussed in this section 11
bution converges to the same probability for both outcomes X andY Legend is the same as in Fig 51 76
same for three consecutive time steps This correctly models struc-ture of the toy domain Legend is the same as in Fig 51 76
57 Computation of the first mem in AHMEM with RMaxS strategyon an input sequence AABAAB 78
58 Computation of the second mem in AHMEM with RMaxS strategyon input sequence AABAAB 79
59 Computation of the second mem in AHMEM with RMaxS strategyon input sequence AABAAC 80
510 Entropy of recalled episodes 81
511 Mem picked by the RMinOS in AHMEM on the sequence AABAAB 81
512 Computation of the first mem in CHMM with RMaxS strategy oninput sequence AABAAB 82
513 Computation of the second mem in CHMM with RMaxS strategyon input sequence AABAAB 83
514 Computation of the third mem in CHMM with RMaxS strategyon input sequence AABAAB Legend is the same as in Fig 57 83
515 Computation of the second mem in CHMM with RMaxS strategyon input sequence AABAAC 84
516 Entropy of recall on the level of episodes in CHMM on sequenceAABAAC when two mems (O2 = BO5 = C) are used Comparedto the same situation in AHMEM (shown in the upper right sub-figure in Fig 510) we can see that CHMM is much more uncertain 85
517 Recall of the sequenceAABAAB in AHMEM+RMaxS AHMEM+RMinOSand CHMM+RMaxS architectures 86
518 Evolution of belief entropy and surprise in AHMEM on the levelof episodes for the sequence AABAAB 89
519 Evolution of belief entropy and surprise in AHMEM on the levelof observations for the sequence AABAAB 90
520 Evolution of belief entropy and surprise in CHMM on the level ofepisodes for the sequence AABAAB 91
521 Evolution of belief entropy and surprise in CHMM on the level ofobservations for the sequence AABAAB 92
61 Influence of the amount of the training data on 1st-best recallaccuracy in AHMEM1
8 using the RMinOS strategy on the levelof observations We will denote this combination as AHMEM1
8 +RMinOS + O The ribbon shows a variance of the data Notethat more mems lead to better recall Adding more training datahelps universally when three mems are used as evidence Howeverthe training data between sequences 625 and 650 seem to harmperformance of the recall with one and two mems Also note thatthe variance of recall accuracy with three mems decreases withmore training data 100
173
62 Influence of the amount of the training data on 1st-best recallaccuracy in AHMEM1
8 + RMinOS + E0 Note higher variancecompared to recall in the same model on the level of observations(shown in Figure 61) 101
63 Comparison of recall accuracy on the level of episodes in AHMEM18
when using RMinOS and RMaxS with just one mem 10164 Recall in CHMM1
8 +RMinOS+O Approximately after using 200sequences for training the model does not improve Compare thisfigure with Figure 61 showing the same situation in AHMEM1
8which can make use of more training data 102
65 Recall in CHMM18 + RMinOS + E0 Compare this figure with
Figure 61 showing the same situation in AHMEM18 which can
make use of more training data 10266 Comparison of recall accuracy between AHMEM1
8 and CHMM1
on the level of observations when using RMinOS These figures aresuperpositions of the graphs from Figures 61 and 64 103
67 Comparison of recall accuracy between AHMEM18 and CHMM1
on the level of episodes when using RMinOS These figures aresuperpositions of the graphs from Figures 62 and 65 104
68 Comparison of RMinOS and RMaxS strategies on recall accuracyin AHMEM1
8 +O when three mems are used as evidence for recallThis is the only case where RMaxS performs better than RMinOS 105
69 Recall of observation probabilities for 10 time steps in AHMEM18+
RMaxS model with an increasing number of mems used to recon-struct the sequence 107
610 Screenshot of a simulation showing the IVA performing an actionCOOK in a virtual kitchen The action is not animated in detailin our simulation The screenshot is rendered using UT2004 (EpicGames 2004) 109
611 Recall of the stored day when three mems are used for reconstruction111612 Recall of the stored day when two mems are used for reconstruction112613 Recall of the stored day when one mem is used for reconstruction 113614 Recall of the stored day when no mem is used for reconstruction 114615 Ground truth of the stored day 115616 Comparison of recall on the level of episodes when three mems
are used AHMEM12 correctly recalls the IDuties episode whereas
CHMM1 forgets it 116617 Entropy of recall in AHMEM1
2 with three mems 118
71 Third party developerrsquos perspective of DyBaNeM 125
174
List of Tables
41 Summary of the main phases of DyBaNeMrsquos working cycle 60
51 Training examples of the toy dataset 69
61 Results of the recall experiment for all tested models encodingstrategies and the number of stored mems and level of hierarchy(observations O and first level of episodes E0) The table reportsmean recall accuracy in percentage and its standard deviation onthe 100 testing sequences when the schemata were learnt on 725sequences 97
62 Average time plusmn SD needed to compute one mem measured inmicroseconds The average is over the time needed to computethe first second and third mem in 100 testing sequences 97
63 Results of testing the hypothesis that having n mems is better thannminus 1 mems 98
64 Results of testing the hypothesis that AHMEM is a better proba-bilistic model than CHMM The legend is the same as in Table 63 99
65 Results of testing the hypothesis that RMinOS encoding strategyis better than RMaxS The legend is the same as in Table 63 99
66 Accuracy on the level of episodes for recall of the 23rd day in thetwo compared probabilistic models 117
67 Hypothetical verbalized recall of high level events based on Fig-ures 611 ndash 614 Text in italics highlights where the recall differsfrom recall when using one more mem The list line shows groundtruth that is the same as recall with three mems The modal verbldquomayberdquo is used whenever there are multiple possible episodes withhigh probability 118
71 Lossy compression algorithms for different types of data 127
175