CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

CS230:Lecture3ThemathematicsofdeeplearningBackpropagation,Initializations,Regularization

KianKatanforoosh

I– BackpropagationII– InitializationsIII– Regularization

I- Backpropagation

Problemstatement

A– LogisticRegressionbackpropagationforonetrainingexample

B– LogisticRegressionbackpropagationforabatchofmexamples

Question:Youhavetrainedananimalclassifier.Canyoutellwhatpartoftheinputledtothisprediction?

II- Initializations

Problemstatement

Inclass,you’veseenthat:

Thegoal:

Let’sprovethat: !"# "[%&'] = !"# "[%] → !"# +[%] = 1-[%&']

Let’sprovethat: !"# "[%&'] = !"# "[%] → !"# +[%] = 1-[%&']

!"# "[%] = !"# .[%] = !"# / +',1% "1%&'2 345

16'Checkpoint:

We’veshownthatforeverylayerl: !"# "[%] = -[%&']!"# +[%] !"# "[%&']