This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1/30
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
Big Data: Data Wrangling Boot CampBD Tools and Techniques
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
An overview
Vocabulary
Data Sources – where data comesfromIngestion – how data ispre-processed for acceptanceData Sea/Lake – where data livesProcessing – how data isprocessed prior to storageData warehouse – transition fromSQL to NoSQLAnalysis – extracting informationfrom dataUser interface – how the userinteracts with the information Image from [1].
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
Each is different for a reason
Vocabulary (1 of 2)[12].
array – generalize operations onscalars to apply transparently tovectors, matrices, andhigher-dimensional arrays.client side – languages are limitedby the abilities of the browser orintended client.compiled – languages typicallyprocessed by compilers, thoughtheoretically any language can becompiled or interpreted.concurrent – languages providelanguage constructs forconcurrency.curly-bracket – languages have asyntax that defines statementblocks using the curly bracket orbrace charactersdeclarative – languages describe a
problem rather than defining asolution.extension – languages embeddedinto another program and used toharness its features in extensionscripts.functional – languages defineprograms and subroutines asmathematical functions.generic – language is applicable tomany domains.imperative – languages may bemulti-paradigm and appear in otherclassifications.impure – languages containingimperative features.interactive mode – languages act asa kind of shell
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
Each is different for a reason
Vocabulary (2 of 2)[12].
interpreted – languages areprogramming languages in whichprograms may be executed fromsource code form, by an interpreter.iterative – languages are builtaround or offering generators.list-based – languages are a type ofdata-structured language that arebased upon the list data structure.metaprogramming – hat write ormanipulate other programs (orthemselves) as their data or that dopart of the work that is otherwisedone at run time during compiletime.object-oriented (class-based) –
support objects defined by theirclass.object-oriented prototype-based –languages are object-orientedlanguages where the distinctionbetween classes and instances hasbeen removedprocedural – languages are basedon the concept of the unit andscopereflective – languages let programsexamine and possibly modify theirhigh level structure at runtime.scripting – another term forinterpreted
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
Each is different for a reason
What does the future hold?
“If languages are not defined by taxonomies, how arethey constructed? They are aggregations of features.Rather than study extant languages as a whole, which con-flates the essential with the accidental, it is more instruc-tive to decompose them into constituent features, whichin turn can be studied individually. The student then hasa toolkit of features that they can re-compose per theirneeds.”
S. Krishnamurthi [6]
New languages will be created all the time to fit needs.
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
What have we covered?
Looked at how Amdahl’s Law canimprove performanceLooked at single and multithreadedprogramsLooked at some of the many OpenSource Big Data tools that areavailableLooked at how and why somelanguages are better than othersfor a particular application
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
References (1 of 4)
[1] Josef Adersberger, Big Data Landscape Q3/2016, email,2016.
[2] Gene M Amdahl,Validity of the single processor approach to achieving large scale computing capabilities,Proceedings of the Spring Joint Computer Conference, ACM,1967, pp. 483–485.
[3] John T. Bell, Threads, https://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/4_Threads.html, 2013.
[4] James Gleick, Chaos: Making a New Science, Random House,1997.
Intro. Amdahl BD Processing Languages Q & A Conclusion References Concepts
References (4 of 4)
[13] Peter Van Roy et al.,Programming Paradigms for Dummies: What Every Programmer Should Know,New Computational Paradigms for Computer Music 104(2009).