Digits that are not Generating new types through deep neural nets Mehdi Cherti, Balázs Kégl LAL/LRI, CNRS Université Paris-Saclay Akın Kazakçı MINES ParisTech, PSL Research University
Digits that are notGenerating new types through deep neural nets
Mehdi Cherti, Balázs Kégl LAL/LRI, CNRS
Université Paris-Saclay
Akın Kazakçı MINES ParisTech,
PSL Research University
The Short Story
Why are we doing this?
How to determine the value of novelty?
The value of novelty is the blindspot of creativity research (Kazakçı, 2014).
Computational creativityDeep learning to the rescue?
• has enabled great progress in machine learning
• Also, several promising work renewed interest in computational creativity.
• Google created Magenta (nobody needs us anymore)
Gatys et al. 2015
Deep learning
Yet• main emphasis in DL research remains
learning to predict• models are based on likelihood whereas
creativity is unlikely• the initial breakthrough came from a
generative model
But the generative potential of deep nets is largely unexplored
Fitness function barrier
• For most computational creativity systems, the value function is fixed and predetermined
• This is a paradox. - And an obstacle for progress.
• Evolutionary approaches to computational creativity bears these inherent limitations:
• Explicit fitness functions reflects system designer’s preference for novelty - not the machine’s.
• We call this the fitness function barrier.
A program to go beyond the barrier
Ultimate objective: Try & get rid of hard-coded value functions; let the system develop its own
A genuinely creative system needs to be able to develop its own notion of value
Current work: Build a system
1. that can study & learn a referential set of objects (RS)
2. that can generate new objects that keep essential features of RS but generate unseen types
3. that provides an experimental bench for developing & testing various ways an agent can develop a value function
Auto-associative neural netsa.k.a auto-encoders
Learning to disassemble
Learning to build
- Auto-encoders have existed for long time (Kramer 1991)
- Deep variants are more recent (Hinton, Salakhutdinov, 2006; Bengio 2009)
- A deep auto-encoder learns successive transformations that decompose and then recompose a set of training objects
- The depth allows learning a hierarchy of transformations
The experimental setup
- Training data : MNIST, 70000 images of handwritten digits of size 28x28
- We use a sparse convolutional auto-encoder (3c1d) trained to:
- Encode : take an image and transform it to a sparse code
- Decode : take the sparse code and reconstruct the image
- Training objective is to minimize the reconstruction error
Generating new symbols
- We use an iterative method to build symbols the has never seen:
- Start with a random image x0 = r,
- and force the network to construct (i.e. interpret)
- xk = f(xk-1), until convergence
- Our method is inspired by Bengio et al. (2013)
- By contrast to them, we do not constrain the net to generate only known types (we do not consider unknown symbols as spurious)
Visualising the structure of generated images
Coloured clusters are original MNIST digits (classes from 0 to 9)
The gray dots are newly generated objects
New objects form new clusters
Using a clustering algorithm, we recover coherent sets of new symbols
A distance preserving projection of digits to a two-dimensional space (van der Maaten and Hinton 2008)
Creativity by fixations
Lear
nFo
rce
Fixa
te
Summary
A genuinely creative system needs to be able to develop its own notion of value.
Our system is a first step:• It can effectively create new types of objects preserving abstract
and semantic properties of a domain. • It provides an experimental setup that enables testing various
hypothesis.• It provides a bridge between current research on machine
learning and creativity research.
Thank you
Mehdi Cherti, Balázs Kégl {mehdi.cherti, balazskegl}@gmail.com
Akın Kazakçı [email protected]