DEPARTMENT OF HEALTH AND HUMAN SERVICES • National Institutes of Health • National Cancer Institute Frederick National Laboratory is a federally funded research and development center operated by Leidos Biomedical Research, Inc., for the National Cancer Institute. NIH.AI Workshop on Hyperparameters Optimization Intro to hyperparameter sweeps techniques George Zaki, Ph.D. [C] BIDS, Frederick National Lab for Cancer Research (FNLCR) July 18, 2019
17
Embed
NIH.AI Workshop on Hyperparameters Optimization Intro to ...hpo-introduction.pdf · HPO and High Performance Computing (HPC) •HPO requires good amount of compute resources •HPC
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DEPARTMENT OF HEALTH AND HUMAN SERVICES • National Institutes of Health • National Cancer InstituteFrederick National Laboratory is a federally funded research and development center operated by Leidos Biomedical Research, Inc., for the National Cancer Institute.
NIH.AI Workshop on Hyperparameters Optimization
Intro to hyperparameter sweeps techniquesGeorge Zaki, Ph.D. [C]BIDS, Frederick National Lab for Cancer Research (FNLCR)
July 18, 2019
Model’s Parameters
• Are fit during training• Result of model fitting or training• Also optimized during training
• Examples• Slope and intercept in linear modeling
• Weights and biases in Neural Nets
What are Hyperparameters?
• Parameters of your system with no straightforward method on how to set their values:– Usually set before learning process– Is not directly estimated from the data
deepai.org
Examples of Hyperparameters
• The depth of a decision tree• Number of trees in a forest• Number of hidden layers and neurons in a neural network, • Degree of regularization to prevent overfitting• K in K-means• Learning rate schedule in Stochastic Gradient Descent (SGD)• …. Activation functions