Algorithm, and Application Deep Learning: Concept, Model,

Deep Learning: Concept, Model,Algorithm, and Application

Oleh. Prof. Zainal A. Hasibuan, PhD.

Webinar Aptikom #3

20 Juni, 2020

AGENDA• Konsep: Apa itu Deep Learning?

• Mengapa Deep Learning Menjadi Penting?

• Bagaimana Cara Bekerja Deep Learning?

• Model: Apa Saja Jenis Deep Learning?

• Algoritma: Metode Apa Saja Yang DigunakanDeep Learning?

• Aplikasi Deep Learning

• Penutup

Konsep:Apa itu Deep Learning?

Ilustrasi….Penggunaan Deep Learningfb notifications Kusrini

added 1 photo that

might include you

• Smart city bisa menjadi enableruntuk mewujudkan visi ini. Teknologidan TIK dapat digunakan untukmempertinggi efisiensi,memperbaiki pelayanan publik, danmeningkatkan kesejahteraan warga.Teknologi juga bisa membantuprogram-program kami sepertimempromosikan produk Banjarbarusehingga bisa dikenal hingga di luarnegeri.

• Smart city can be an enabler to realizethis vision. Technology and ICT can beused to enhance efficiency, improvepublic services, and improve the welfareof citizens. Technology can also help ourprograms such as promoting Banjarbaruproducts to be known abroad.

Google Translator

1

2

Mau Kemana Hari Ini?

3

• Notifikasi dari facebook• Terjemahan dari Google• Mau kemana hari ini• Dll….

Masalah 2 yang bisa

diselesaikan oleh

Deep Learning

Definisi Deep Learning…• Deep learning adalah bagian dari Machine

Learning yang bisa “berpikir” seperti manusia,dan bisa membantu manusia membuatkeputusan dalam hal identifikasi, prediksi, danprekripsi (Zainal A. Hasibuan, 2020).

• Deep learning is a subset of machinelearning where artificial neural networks,algorithms inspired by the human brain, learn from large amounts of data.

• Deep learning allows machines to solve complexproblems even when using a data set that is verydiverse, unstructured and inter-connected.

Kenapa Dibilang “Deep” Learning?

“Dangkal”, karena hanya 1lapisan tersembunyi (1 hiddenlayer)

“Dalam”, karena banyak lapisantersembunyi (many hidden layers)

Bis

a Ju

ga D

ibila

ng

“ H

igh

Lea

rnin

g”?

“Ren

dah

”, karena

hanya 1 lap

isantersem

bu

nyi (1h

idd

en layer)

“Dalam

”, karena

banyak lap

isantersem

bu

nyi (many

hid

den

layers)

Konteks Deep Learning Dengan Bidang IlmuLainnya: AI, DS, and Big Data

Mengapa Deep Learning MenjadiPenting?

Deep Learning Untuk Mengatasi KeterbatasanMachine Learning dalam Memberikan Solusi

• Banyak masalah disekitar kita yang perludiotomasi

• Kemampuan mengolah data yang lebih besar danmulti-dimensi

• Kemampuan “belajar” yang lebih rinci danberulang-ulang secara otomatis dengan fitur2yang lebih banyak

• Kemampuan “belajar” yang lebih mirip dengankonstruksi cara berpikir manusia, bertahap,banyak pilihan, tapi harus membuat keputusan.

Hampir disetiap dimensi hidup kita, ada permasalahanyg solusinya perlu diotomasi, dan bisa diselesaikan

oleh Deep Learning

Sudah terbukti dengan jumlah data yang besar, kinerja DL semakin baik

What Makes DL Become More Possible?

• Big Data (Large Data Sets)

• IoT and Sensor to Capture All Kinds of Data toProduce Big Data

• Computing Power Increased: Graphics ProcessingUnit (GPU) dan Central Processing Unit (CPU)

• The Emergence of Platforms and Software Tools

• Broadband to Guarantee Speed of Data Transfer

• Cloud Technology

• Large Memory Handling Capability

Zainal A. Hasibuan, 2020

Implementation #2: Uncover Ethnic and Sub-EthnicRelationships Using Deep Learning: Preserving

Indonesian Cultural Heritage

FeatureExtractor

FeatureExtractor

FeatureExtractor

Mid-LevelFeatures

Mid-LevelFeatures

High-LevelFeatures

Trainab

le Classifier

Deep

Learnin

g

Zainal A. Hasibuan, 2018

Suku Batak

Batak Toba, Batak Angkola

Batak Toba=Tobasa, SimalungunBatak Angkola=Sipirok, Pdg Bolak

Bagaimana Cara Bekerja DeepLearning?

Ilustrasi…

Coba tebak…siapa gerangan orang ini?

Bagaimana Deep Learning Mengidentifikasinya?

OutputInput Process

Zainal A. Hasibuan, 2020Inputlayer

N HiddenLayers

Outputlayer

Representasigambar digantike angka-2pantulanwarna RGB

Neural network

• Neuron Manusia

23

• Artificial NeuralNetworks (ANN)

64x64x3=12288 parametersWe also want many layers

Model: Apa Saja Jenis Deep Learning?

Beberapa Istilah dalam Deep Learning• A node, also called a neuron or Perceptron, is a computational

unit that has one or more weighted input connections, a transferfunction that combines the inputs in some way, and an outputconnection. Nodes are then organized into layers to comprise anetwork

• Tensors, defined mathematically, are simply arrays of numbers, orfunctions, that transform according to certain rules under achange of coordinates.

• An artificial neuron (also referred to as a perceptron) is amathematical function. It takes one or more inputs that aremultiplied by values called “weights” and added together. Thisvalue is then passed to a non-linear function, known as anactivation function, to become the neuron's output.

• the perceptron is an algorithm for supervised learning of binaryclassifiers. ... It is a type of linear classifier, i.e. a classificationalgorithm that makes its predictions based on a linear predictorfunction combining a set of weights with the feature vector.

Convolutional Neural Networks (CNN)• Jaringan Neural Konvolusional (Convolutional Neural Networks)

pada dasarnya adalah jaringan saraf standar yang telahdiperluas melintasi ruang menggunakan bobot bersama.

• CNN dirancang untuk mengenali gambar dengan memilikibelokan di dalamnya, yang melihat ujung-ujung objek yangdikenali pada gambar.

• CNNs adalah jaringan saraf yang menggunakan operasikonvolusi sebagai salah satu lapisannya.

35Sumber: https://ramprs.github.io/2017/01/21/Grad-CAM-Making-Off-the-Shelf-Deep-Models-Transparent-through-Visual-Explanations.html

Recurrent NNTime Series

How does RNN produce result?

I love CS !

Result after readingfull sentence

Evolving “embedding”

Recurrent Neural Networks (RNN)

• Recurrent Neural Networks (Jaringan Syaraf Berulang)pada dasarnya adalah jaringan syaraf standar yang telahdiperluas sepanjang waktu dengan memiliki tepi yangdimasukkan ke dalam langkah waktu berikutnya

• RNN adalah jenis jaringan saraf yang memiliki loop internal

• RNN dirancang untuk mengenali urutan, misalnya, sinyalucapan atau teks.

38Sumber: https://datathings.com/blog/post/lstm/

Revisit: Ilustrasi…Penggunaan Deep Learningfb notifications Kusrini

added 1 photo that

might include you

• Smart city bisa menjadi enableruntuk mewujudkan visi ini. Teknologidan TIK dapat digunakan untukmempertinggi efisiensi,memperbaiki pelayanan publik, danmeningkatkan kesejahteraan warga.Teknologi juga bisa membantuprogram-program kami sepertimempromosikan produk Banjarbarusehingga bisa dikenal hingga di luarnegeri.

• Smart city can be an enabler to realizethis vision. Technology and ICT can beused to enhance efficiency, improvepublic services, and improve the welfareof citizens. Technology can also help ourprograms such as promoting Banjarbaruproducts to be known abroad.

Google Translator

1

2CNN

RNN

Algoritma: Metode Apa Saja YangDigunakan Deep Learning?

10 Metode untuk Deep Learning

1. Back-Propagation

2. Stochastic Gradient Descent

3. Learning Rate Decay

4. Dropout

5. Max Pooling

6. Batch Normalization

7. Long Short-Term Memory

8.  Skip-gram

9. Continuous Bag Of Words

10.  Transfer Learning

41

(sumber:James Lee, 2017)

1. Back-Propagation

• Back-propagationadalah metodesederhana untukmenghitungturunan parsial(atau gradien)dari suatu fungsi,yang memilikibentukkomposisi fungsi(seperti padaJaring Saraf).

42

sumber: http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html

2. Stochastic Gradient Descent

• Gradient descentbertujuan untukmencapai titikminimum darisuatu fungsi,namun kadangkalaterjebak dalamminimum lokalyang tergantungpada sifat medan.

43

3. Learning Rate Decay• prosedur optimalisasi gradien stokastik untuk

menyesuaikan tingkat pembelajaran(learning rate) dapat dilakukan denganmeningkatkan kinerja dan mengurangi waktupelatihan atau dikenal dengan tingkatpembelajaran adaptif.

44

4. Dropout• Dropout adalah teknik untuk mengatasi masalah

overfitting.

• Dropout telah terbukti meningkatkan kinerja jaringansaraf pada tugas pembelajaran yang diawasi dalam visi,pengenalan suara, klasifikasi dokumen dan biologikomputasi, memperoleh hasil canggih pada banyakdataset benchmark (S. H. Khan, Hayat, & Porikli, 2019;Poernomo & Kang, 2018).

45

5. Max Pooling• Max pooling adalah proses diskritisasi berbasis sampel

yang bertujuan untuk mengambil sampel representasiinput (seperti: gambar, matriks keluaran lapisantersembunyi, dll.), mengurangi dimensinya danmemungkinkan asumsi dibuat tentang fitur-fitur yangterdapat dalam sub-wilayah yang di-binned.

46

6. Batch Normalization

• Batch normalisasimembantu sedikitproses penyetelaninisialisasi bobot danparameterpembelajaran yangcermat denganmengurangi jumlahdengan apa nilai unittersembunyibergeser di sekitar(pergeserankovarians).

47

7. Long Short-Term Memory

Jaringan LSTM memiliki tiga aspek yangmembedakannya dari neuron biasa dalamjaringan saraf berulang yaitu memiliki kontroluntuk memutuskan kapan:• membiarkan input masuk ke neuron.• mengingat apa yang dihitung pada langkah

waktu sebelumnya.• membiarkan output meneruskan ke cap

waktu berikutnya.

48

8. Skip-gram

• Skip-gramadalahmodeluntukmempelajari algoritmapenyematan kata

49

9. Continuous Bag Of Words

• Dalam modelContinuous BagOf Words,tujuannyaadalah untukdapatmenggunakankonteks yangmengelilingikata tertentudanmemprediksikata tertentu.

50

10. Transfer Learning

• Transferpembelajaranadalah saatmenggunakanCNN yang dilatihuntuk satu setdata, memotonglapisan terakhir,melatih ulangmodel lapisanterakhir pada setdata yangberbeda.

51

Aplikasi Deep Learning

Proses Analitik dengan Deep Learning

53

Aplikasi #1Deep Learning for Students’ Academic

Performance

Oleh: Ariana Yunita, Harry Budi Santoso, Zainal A Hasibuan

4,69% students Drop Out from 2013/2014 to 2014/2015 (Ristekdikti, 2016).

The biggest number is in Jakarta.

4,69% Banten (14.87%), North Sumatera (11.11%), BangkaBelitung (11.32%), Lampung (10.11%).

Several provinces with high rate of students DO morethan 10%

43,073Number of students Drop Out from period 2013/2014 to

2014/2015 in Jakarta

55

◉ This phenomenon poses significant challenges tohigher degree institutions, and they have toanalyze the way to address those issues andvarious factors that affect graduates’ success rate,as well as to predict successful graduates basedon their behaviors while studying at a university.

56

Objectives◉ To propose a predictive model for students’

academic performance

◉ To do a preliminary study using log datafrom a course to predict whether studentsget an excellent score, get an intermediatescore, or fail.

57

Research Framework

58

Demographicdata

Academicdata

Behavioral

data  input

Machine

Learning

DeepLearnin

g

process

Learning

Analytics     

Prediction

output

59

Type of Data Description of DataStructured Socio-demographic, including

∙ gender,∙ age,∙ residential area,∙ financial status,∙ type of jobs,∙ salary

Academics, including∙ pre-test (Academic Potential Test),∙ mathematics score,∙ GPA of every semester∙ GPA when graduating

Behavior, including∙ Frequency of borrowing book at

library∙ Log data from LMS (total page

view, total posting, totaldownloads)

Unstructured Log Data from LMS (discussion forum)Type of Data for Learning Analytics

Research Framework

Dataset Building Approach

Data Pre-processing

60

Log data are groupedbased on the type of

activity in LMS. Missingvalue, inconsistent data,

noise and outlier areexamined in this step.

FeatureEngineering

EDA is essential Thisstep allows to

visualize data, andshow numerical

summary in statistics,such as mean,

median, and quartile.

Feature engineering allowsto select, correlate and

examine which importantfeatures for model. In mostcases, feature selection is

needed to increase theperformance of prediction.

Exploratory Data Analysis (EDA)

List of Features

61

Features Description Type of Data DetailsGender Student gender Nominal Female/Male

Level_discussion Whether a studentposted less than theaverage of students’sposting

Ordinal High, Medium, Low

Post_in_Discussion_w1

Whether a studentposted in discussion inweek 1 or not

Ordinal 1: Yes0 : No

Post_in_Discussion_w2

Whether a studentposted in discussion inweek 2 or not

Ordinal 1: Yes0 : No

Post_in_Discussion_w3

Whether a studentposted in discussion inweek 3 or not

Ordinal 1: Yes0 : No

Post_in_Discussion_w4

Whether a studentposted in discussion inweek 4 or not

Ordinal 1: Yes0 : No

List of Features (2)

62

Features Description Type of Data DetailsRead_Discussion_w1

Whether a studentread forum discussionin week 1 or not

Ordinal 1: Yes0 : No

Read_Discussion_w2

Whether a studentread forum discussionin week 2 or not

Ordinal 1: Yes0 : No

Read_Discussion_w3

Whether a studentread forum discussionin week 3 or not

Ordinal 1: Yes0 : No

Read_Discussion_w4

Whether a studentread forum discussionin week 4 or not

Ordinal 1: Yes0 : No

Read_Discussion_w1

Whether a studentread forum discussionin week 1 or not

Ordinal 1: Yes0 : No

Post_in_Discussion_w4

Whether a studentposted in discussionin week 4 or not

Ordinal 1: Yes0 : No

List of Features (3)Features Description Type of Data DetailsView_syllabus Whether a student

viewed syllabus or notOrdinal 1: Yes

0 : NoAccess_audio Whether a student

accessed audio or notOrdinal 1: Yes

0 : NoView_announcement Whether a student

viewed announcementor not

Ordinal 1: Yes0 : No

View_lecture_slides Whether a studentviewed lecture slides ornot

Ordinal 1: Yes0 : No

Number_of_viewing_lecture_slides

How many times astudent viewed lectureslides

Interval 0-maximum numberof viewing

Level_viewing_lectures

Level of viewing lectureslides

Ordinal High: More than 15timesMedium 11-15Low: Less Than 11times

Target Whether studentachieved final scoremore than 80(class 2),between 65 and 80( l 1 l th 65

Ratio 0: less than 651: 65-802: more than 80

Architectures and How toSplit Data

64

Data Training 54Data

Validation13

Data Testing 17

TABLE . Data for Training, Validating andTesting

Inputlayer

N HiddenLayers

Outputlayer

Fig . Feedforward Neural Network with NHidden Layers

DL architectures were built usingKeras as the frontend andTensorFlow as the backend [23].This preliminary study used asupervised technique to predictthree classes. Activation functionsthat were used were relU andSoftmax. RelU was used as ahidden layer and Softmax was usedas an output layer. Furthermore,Adam [24] algorithm was used foroptimization in this study.

Place your screenshot here

Results

Using DL architecture with 2 hiddenlayers, input = 18 nodes, hidden units= 12 and 10, output = 3 nodes, themodel has been validated as can beseen in Figure at the right. It hastraining and validation accuracymore than 85%. However, when themodel is tested the accuracy is stillaround 35%.

Several reasons why the accuracy isstill low, probably because thenature of DL needs larger data toperform high accuracy.

Another reason probably becausethe features used in this study hasnot represented SAP.

65

Conclusion and Future Research◉ Firstly, this study built a dataset for SAP in the course level.

Second, this paper conducted a preliminary study using DLto predict SAP, but the results is still low accuracy becauseprobably the amount of data is not enough for DL andfeatures used in this study has not represented SAP.

◉ Based on exploratory data analysis, this study shows thatlog data could not reflect cognitive activities, especially ifthe rate of feature usage was only calculated based on thetotal hits of every activity on LMS. However, several insightscan be gained, such as students in the average achievergroup seems to have high motivation study than studentswith high achiever.

66

◉ For future research:

◉ Revise feature engineering

◉ Using larger data from students, such as social,demographic and financial factors. It is alsosuggested to build a predictive model thatinclude other data as listed in the Table “Typeof Data of Learning Analytics”.

67

Conclusion and Future Research

Aplikasi #2:UNIFIED CONCEPT-BASED MULTIMEDIA

INFORMATION RETRIEVAL SYSTEMOleh: Ridwan Andi Kambau & Zainal A. Hasibuan

Membangun Model(Training, Validasi dan Testing CNN dan RNN)

Valida

si

Training CNN& RNN

Testing CNN &RNN

Pengindeksan Terpadu dengan ModelPengklasifikasi Multimedia

Terms/Concepts

Model PengklasifikasiMultimedia

ModelPengklasifikasi Citra

ModelPengklasifikasi VIdeo

ModelPengklasifikasi Audio

ModelPengklasifikasi Teks

0-Toraja1-Batak2-Bali

3-tongkonan4-rambusolo5-rumahbolon6-Tari tortor7-PuraUluwatu8-TariPendet

DataMultimedia

Pengujian Model PengklasifikasiImage dengan kueri Tongkonan.jpg

Pada Pengindeksan Terpadu

Hal yang sama dilakukan untuk Sembilanobjek yang diklasifikasi dengan empat formatMedia

Akurasi Pengujian ModelPengklasifikasi Multimedia

Objek-objekSuku

KlasifikasiCitra

KalsifikasiAudio

KalsifikasiVideo

KlasifikasiTeks

KlasifikasiMultimedia

Akurasi Akurasi Akurasi Akurasi Akurasi Rata2

Toraja 0.74 0.67 0.74 0.96 0.78

Tongkonan 0.64 0.56 0.62 0.75 0.64

Rambusolo 0.65 0.54 0.60 0.73 0.63

Batak 0.78 0.67 0.78 0.98 0.80

Rumah Bolon 0.66 0.53 0.58 0.76 0.63

Tari Tortor 0.62 0.52 0.52 0.74 0.60

Bali 0.80 0.67 0.80 0.98 0.81

Pura Uluwatu 0.60 0.50 0.58 0.77 0.61

Tari Pendet 0.59 0.53 0.58 0.76 0.61

Rata-rata perMedia

0.68 0.58 0.64 0.82

Penutup

Which Tool is Best for Solving DeepLearning Problems?

• TensorFlow, PyTorch, Keras, Caffe are some ofthe popular tools which are being widelyused to carry out Deep Learning algorithms.

• TensorFlow is one of the best deeplearning frameworks and has been adoptedby several giants such as Airbus, Twitter, IBMdue to its highly flexible system architecture.

Deep Learning Approach to Multi-Discipline Research

DeepLearning

R&D

Zainal A. Hasibuan, 2019

Dosen Berlatar BelakangEkonomi, dan Bisnis,Sistem Informasi dll

Dosen berlatar belakangSistem Informasi, Data Science

Dosen berlatarBelakang IlmuKomputer/TI/SE

Business A

nalyst

System Analyst

Pro

gram

mer

Tech

nica

l Ana

lyst

Dosen berlatarBelakang IlmuKomputer/TI/SE

Dosen2 Doktor dan Professor dariEkonomi, Bisnis, Komputer, dan berbagaiSektor: pertanian, pariwisata, dll

Research Objectives

• Tersedia dataset dari big data untuk berbagaisektor

• Menghasilkan algoritma baru dalam DeepLearning

• Terbangunnya berbagai sistem otomasiberdasarkan Deep Learning:– Identifikasi dan klasifikasi perilaku pelanggan– Prediksi produksi pangan– Deteksi berbagai penyakit– Dan lain-lain