Linear Models (I) Rong Jin Review of Information Theory What is information? What is entropy? Average information Minimum coding length Important inequality Distribution…