From Fort Lewis College to Big Data

Marilyn WaldmanMarilyn Waldman

Software EngineerSoftware EngineerBoulder, Colorado

@mdwaldman

From Fort Lewis College to Big Data

Where do you fit into thelandscape of computing?

My journeyHistory of ComputingSpark Streaming

- Mapping the Brain at ScaleReading, Writing and Computation whereComputation is Math + Computing Skill

Demonstration

1973-19771973-1977

Chemistry

Introduction to ChemistyOrganic ChemistryPhysical ChemistryAdvanced Organic Chemistry

Math

Pre-CalculusCalculusDifferential EquationsProbability and StatisticsAbstact Algebra

Fort Lewis College Biology

Introduction to BiologyA&PBiochemistry

General Courses

WritingHistorySpanishPsychology

Department of Computer Science

Dr. Andrzej EhrenfeuchtDr. Andrzej Ehrenfeucht

Dr. Eugene Myers Dr. David Haussler Dr. Gary Stormo

1980 - 19831980 - 1983

Course of Study at CU

Prerequisites

Linear AlgebraFinite MathematicsProgrammingLanguagesAlgorithms and DataStructure

Course Work

Operating SystemsNumerical AnalysisOperations ResearchProgramming LanguagesAutomata Theory

Field of Study -Distributed Programming

Introduction to Big Datawith Apache Spark

70,000 enrolled10,000 passed

Scalable Machine Learning

50,000 enrolled7,000 passed

- WWII

Father of computingBroke the codeTuring MachineAutomata Theory

AlanTuring

Enigma

: Redline credit

- Dennis Ritchie and Ken Thompson, June 1972

"...the number of UNIX installations has grown to 10, with more expected..."

UNIX - 1969

Administration

Professor Evi Nemeth - Godmother of Unix System

1974 - Intel Releases the 8080 processor

Personal ComputerBeginning of PC and commodityhardware Ed Roberts - Father of the PC

..."most of you steal your

software"

Bill Gates quotes

1973 Robert Metcalfedevised the Ethernet Network AdministrationBasedon - ALOHAnet Norman M.Abramson

Credit: Watch the video

1980 - University of Colorado

proposes that DNA is a four letter alphabetDr. David Haussler and Eugene Myers compete on the HumanGenome Project - 1990 to 2003

Dr. Andrzej Ehrenfeucht

HINT: No Internet, no formal databases, no Iphones, no IPads, no PCs

d3 demo

1990 - The World Wide Internet

HTTP (HyperText Transfer Protocol)URL (Uniform Resource Locator) HTML(HyperText Markup Language)

AWSHPIBMOracle

Big Data Applications

Real-time - IOT, StreamingMachine Learning - Data Analytics

Ted DunningA New Look at Anomaly Detection

Spark Micro Batch

Event Driven: Storm, Samza

Real-time Analytics at Pinterest

Floating SensorsFrom Berkeley

Co-Aerial-Ecologist: Robotic WaterSampling and Sensing in the Wild

Can Spark Streaming survive ChaosMonkey?

Credit

@CodeNeuro

BringingBringingNeuroscience andNeuroscience and

Data ScienceData ScienceTogetherTogether

Click Here

The EndThe End

Thank you