SAS Viya : A Journey in Statistics Canada’s Universe
Karine DésiletsTeam Leader and Technical AdvisorSystem Engineering DivisionStatistics Canada
November 20th, 2018
The Importance of Data
“The world’s most valuable resource is no longer oil, but DATA” The Economist, May 2017
Source : Image taken and adapted from another Statistics Canada presentation
SAS enable organizations to thrive in the analytics economy
ANALYTICAL ECONOMY
Data+
Analytics+
Collaboration
CanapésStatistics Canada Analytical Economy
First CourseViya 3.3 Architecture and Clients
Loading Data in SAS Viya
Programming Languages and Snippets
Main CourseDeployment of SAS solutions – Macro Catalogs and Source Control in CAS (Informatics)
In-memory and multi-threaded algorithms (Mathematicians)Classifications and Proc Cardinality (Analytics)
DessertPros and Cons of SAS Viya in CAS and Statistics Canada
R&D – Future as extra
Conclude with improvements
Menu
Statistics Canada Analytical EconomyC
ompu
ter S
cien
ce • Architect• Programmers• System Analysts• BI Analysts• Platform
Administrator• Algorithmic
Specialists• Databases Experts• User Experience• Researchers • Security
Anal
ytic
s
• Economists• Data Scientists• Geographers• Statisticians• Sociologists• Industry Analysts• Environmental
Analysts• Criminology• Researchers
Met
hodo
logi
sts • Mathematicians
• Statisticians• Researchers
• Includes:• Statistical Methods• Spatial and Temporal
Estimation• Non-Sampling Variance • Outlier Detection• Benchmarking• Interpolation• Calendarization• Longitudinal Data Analysis• Time Series
What is Viya ? Which part is related to Statistics Canada?
SAS Viya
SAS Studio 4.x Series
SAS Cloud Analytic Services (CAS)
SAS Visual Analytics
SAS Visual Data Mining and Machine Learning
SAS Econometrics
SAS Visual Forecasting
SAS Optimization
See more details with : An Introduction to SAS Viya : What is it and what it isn’t! byFrank Scott Analytic Solutions Architect, Canadian Public Sector, SAS Institute (Canada)
SA
S V
IYA
3.3
-Fu
ll D
eplo
ymen
t
Source : support.sas.com (https://documentation.sas.com)
As an employee – What do you need to know ?
Source : support.sas.com (https://documentation.sas.com)
SAS Viya 3.3 is compliant to SAS programming run-time environment (SPRE)
(Deployment)
The parallelisation occurs at the procedures level.
Viya Clients – What do you need to know ?
What is a server without a client?
Web Client
Mobile Client
SAS Home on Viya
SAS EG
SAS DI
Clients
SAS® Clients
9
CAS-L CAS ProcCAS Actions
Pyth
on
Rest Interface using JSON
Lua
Base SAS Compliant
Java
R
That’s Great! All those languages in a SAS program ?
SAS Viyaand
ProgrammingLanguages
The Reality – What can you do ?
SAS Studio 4.x ,SAS EG & SAS DI 9.4M5• CAS actions, CAS-L Language
and new CAS Procedures• Data Step, DS2, FedSQL can run
in CAS Server (no mention of proc sql)
• Base SAS code minimally compliant to execute on SAS Workspace Server
Python 2.7.x & 3.4.x, Lua 5.2/5.3, R 3.1.x • CAS actions, CAS-L Language
and new CAS Procedures
• SAS provides the SAS Wrapper for Analytics Transfer (SWAT)
• Comparison of models built in these different languages in SAS Model Manager
Java 8REST API• CAS actions, CAS-L Language
and new CAS Procedures
• SAS Java Interface for Viya) –with a client JAR file
• With REST API, CAS Offers an embedded HTTPs server that can be accessed for running actions
Major Programming Differences
Data Steps• Process in CAS server• Process in a SAS session• Parallelize With Group By• Take advantage of the in-memory
infrastructure :SMP ModeMPP Mode
No Proc sort neededNo rows level access
Macros• Supported in SAS Viya• Process in SAS session• The macro language itself do
not run on CAS.• Macro with CAS procedures run
in-memory
User-Defined Formats• Stored in Catalogs and process
in a SAS Session
• Stored in a Format library on the CAS server. Format libraries can be associated to a CAS session
• PROC FMT2ITM
SAS Viya : What does it Look Like ?
SAS Viya : What does it Look Like ? (2)
SAS Viya : What does it look like ? (3)
Yes but Viya is great, it gives you access to a Snippets bank of SAS code
15
What is a Snippet ?
SAS Code snippets enable you to quickly insert SAS code into your program and customize it to meet your needs.
Want to use Viya ? What do you need ?
Options:
• SAS Data Explorer – Manage Data
• Programming Interface
• SAS DI / SAS EG
• Probably other tools
How will I load the Data in Viya ?
17
Wait … A More Complex Scenario
I am an IT Folks!I’ve got generalized SAS Solutions.
I want to Deploy SAS solutions and have it accessible to my clients…
What are the options!
IT Perspective – What is the SAS Actual Deployment Model in Relation with Viya?
SAS Macro Catalogs
Dependant of :• Operating System• SAS version Compliant
SAS Programs
I am a SAS Studio User!
19
I am a super Methodologist dealing with computer-intensive calculations and I want the results within seconds!
How will I use In-Memory Multi-Threaded Procedures efficiently in Viya ?
Steps to Follow:
1. Create a CAS connection2. Bind a libname the your current
Data and CAS session3. Load the Data in-memory4. Process the tremendous
Calculations5. Save your Results in Data steps or
report it !
Demo of SAS Code
21
We are a bunch of Analytics people dealing with Classifications and we would like to know if the data we got respect the classification.
How I could use Proc Cardinality efficiently in Viya ?
Immigrant Status
Hou
seH
old
Type
Total Income
Legal Marital Status
National Occupational Classification (NOC)
Inco
me
Sour
ce
North American Industry Classification System (NAICS)
North American Products Classification System (NAPCS)
Gender / Sex
Afte
r Tax
-Inco
me
Duration of Commute
First Official Language Spoken
Proc Cardinality – A Data Prep Procedure
Price Data from Jan98 to Dec02 – 3 Regions – 17 Products
Pros, Cons and Other Considerations
ProsIn-memory process data faster (RAM)
If not in-memory, process on Workspace server (ROM) – no need to modify vintage programs
SAS PROCs and language has been clean-up for in-memory processing. Ex: No more proc Sort
ConsThe In-memory infrastructure is expensive in comparison to Disks
All applications need to be tested with the new language and/or new infrastructure
Load data in-memory could have processing time degradation
Not all PROCs are compliant to CAS
Other ConsiderationsWith Viya – What would be the versioning of SAS in Cloud
Current STC network B bandwidth need to be adjusted – Some SAS Viya interface not loadable
Cloud Agnostic – GoC need a solution that can work with whichever the Cloud provider chosen
24
Hey ! I heard SAS is doing some R&D in order to haveSAS GRID MANAGER and SAS VIYA Combined Together
Come with me, we will look at that…
SAS GRID Manager and SAS Viya : a Perfect Pair
Statistic Canada SAS GRID allows multi threading at the Jobs Level on disk
SAS Viya allows threading at the procedure level with in-memory processing
Combine Both ? – WOW! What Super Powers you would have! Ken Gahagan : A perfect pair :https://www.sas.com/en_us/webinars/sas-grid-manager-sas-viya-a-perfect-pair.html
• SAS products centralized on a Hub with the Viya Platform• New possibilities with in-memory versus disk-based• CAS-L Language, CAS Procedures and CAS Actions
Source of Enhancements / Reflexions:
Compliance for the Future Vision• Statistics Canada is part of Government of Canada Plans and Roadmap Need to be compliant to GoC and Treasury Board of Secretariat, do have to
respect The Business Needs for Security (BNS) and IT Security Guidelines (ITSG).• Exploration of other components by experts in those fields
Conclusion
• SAS/EG Functionalities• SAS Studio visual access to folder /home/&userid
and writing (W) access to other directories
• Source Control • Password Management
27
SAS® 9.4 and SAS® Viya® 3.3 Programming Documentation - Orientationhttps://documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.3&docsetId=pgmmvaov&docsetTarget=pgmsasov.htm&locale=en
An Introduction to SAS® Viya® 3.3 Programminghttps://documentation.sas.com/api/docsets/pgmdiff/3.3/content/pgmdiff.pdf?locale=enSAS-Viya-3.3-Adminstration-Fundamentals-Course-Notes
SAS VIYA Free-Learning Introductory Web Videos:https://support.sas.com/edu/schedules.html?id=4936&ctry=CA#s1=1
https://documentation.sas.com/api/docsets/whatsdiff/3.2/content/whatsdiff.pdf?locale=en
DataStep in Sas Viya : Essential new Features, https://support.sas.com/resources/papers/proceedings17/SAS0118-2017.pdf
Getting Your SAS® 9 Code to Run Multi-Threaded in SAS® Viya® 3.3 https://blogs.sas.com/content/sgf/files/2018/01/TechnicalWhitePaper_SASViya_Jan2018_FINAL.pdf
I strongly recommend the reading of …
Statistics Canada : One hundred years and counting
THANK YOU!For more information please visit, www.statcan.gc.ca
#StatCan100
KARINE DÉSILETSTechnical Advisor and Team Leader
System Engineering DivisionStatistics Canada100 Tunney’s Pasture DrivewayOttawa, Ontario K1A [email protected]