neuralArt.io Large Scale Classification of Fine Art Paintings Mike Osorio Galvanize DSI
neuralArt.ioLarge Scale Classification
of Fine Art Paintings
Mike OsorioGalvanize DSI
Motivation
● Build an Intelligent Machine to learn and judge complex visual concepts and stylistic elements of art.
● Develop a ML model that is capable of classifying a painting’s genre and its artist.
Data Acquisition● Scraped 2000+ high resolution fine art painting images from a Picasa Web Album.● Chose Three Artists● Built a Genre Classifier to determine whether or not a painting is a portrait● Built an Artist Classifier to determine if painted by Van Gogh, Joseph Mallord Turner, or Cezanne
Joseph Mallord Turner
Van Gogh Cezanne
Modeling Process
Joseph Mallord TurnerFishermen in a squall, c 1802
Patching,Dominant Colors,
Grayscale,Denoising,
Edge Detection
Patching
SVC, KNN, Random Forest,
Gradient Boosting
Neural Networks
Feature Engineering● Resize our images to 480 x 480 pixels and extract 30 random 80 X 96 patches of the painting.● Determine Dominant Colors of the patches by performing k means clustering on the image data.● Perform the following image transformations on the patch images:
Grayscale + Total Variation Denoising
Canny Edge DetectionRaw Pixels
Patch Extraction
Joseph Mallord TurnerView the High Street, c 1834
Modeling Process
Joseph Mallord TurnerFishermen in a squall, c 1802
Patching,Dominant Colors,
Grayscale,Denoising,
Edge Detection
Patching
SVC, KNN, Random Forest,
Gradient Boosting
Neural Networks
68 % Accuracy
75 % Accuracy
MVP System for Classification of Art Painting
Raw
Vis
ual F
eatu
res
Imag
e
Res
izin
g
Pat
ch
Ext
ract
ion
GenreClassifier
(Neural Network)
ArtistClassifier
(Neural Network)
Genre:Portrait
Artist:Van Gogh
CezanneHarlequin, c 1888-1890
Feature Learning
xn
xn-1
x1
x2
Van Gogh
Turner
...
...
...
...
Input Layer
Hidden Layer 1
(512 nodes)
Hidden Layer 2
(512 nodes)
HiddenLayer 3
(512 nodes)
Output Layer
Cezanne
Further Work
● Train a large, deep convolutional neural network similar to Alex Krizhevksy’s Alexnet Model.o Expected to give more accurate classifications and allow us to classify
more artists, genres, and themes.o With a more robust classifier, we will have the capability to scale the
app to a phone app (Shazam of Art).
● Experiment with other type of image transformation technique such as wavelets.