Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools

Composing,Reproducing,andSharingSimula5ons

BruceChilders(contact:[email protected];.edu)DebashisGanguly,WilliamC.GarrisonIII,DavidWilkinson,BruceR.Childers,AdamJ.Lee,andDanielMosse’Dept.ofComputerScience,UniversityofPi;sburgh

ModSim2016,Sea/le,Washington,August12,2016

Star5nganewproject…

2

n  You’vegotanewidea:PIMencryp?onenginetosupportaccesscontrols

l  Accesscontrols,applica5onsusingaccesscontrolsl Memoryac5vityandarchitecturesimulator

n  Star5ngpoint(TheIntrospec5veScien5st)l  Hmmm.Iwonderwhatsimulatorsexist?l  Oh,personYmadethismodel!CanIleverage&reuseitformywork?l  Oops.Somethingismissinginpaper.Whatwerethosesimparams???l  Yikes!Wait!Myresultsaredifferent???

Considerthis:Natureasked1,576scien?sts...over70%saidthey’dtriedandfailedtoreproduceanothergroup’sexperiments.

--Scien?ficAmerican,May28,2016

Itreallyshouldn’tbethatdifficult...

3

n Modelingandsimula5onhasadvantagesl  Frameworks(common?)forbuildingsimula5onsl  Implementedmodelsassimula5oncomponentsl  Parameters,inputs,testcases,scripts,outputs

n Wehavealltheingredients!!

PortunoDRAMsim2

TheIngredients

4

n  Portuno:Accesscontrolsimulator/analyzern  SST:HostPortuno&composewithmemorysimulator

l  Instan5ateasimulatorusedforexperimentsl  Shareandcompose(reuse)simulaConmodelswithinSST

n  OCCAM:Create,runandshareexperimentsl  Defineworkflowsofdata,tools,resultsforexperimentsl  Usesimulators/tools&experiments(w/resultsandvisualiza5ons)l  Shareandcompose(reuse)toolsforexperimentworkflowsl  RepeatabilityandmodificaConofexperiments

+ =SC16Tutorial:SolvingandSharingthePuzzle,Sunday,

Nov.13

Portuno:AccessControl

5

GrantAccess

IssueKeys

Users

Admin Set of files

Encrypt

Encrypt

Encrypt

Portuno:AccessControl

6

Re-issueKeys

RevokeUser2

Users

Admin Set of files

Re-encrypt

Re-encrypt

• Whatistheoverheadofre-encryp5on?• Whichaccesscontrolsystemisbestfit?

Portuno:AccessControl

7

PortunoSystem1

System3

System2AccessControlSystems

Applica?onPayload

SystemConstraints

Expressiveness

CostMetrics

Suitabilityanalysisofaccesscontrolsystemsforgivenpayloadundersystemconstraints

•  Monte-Carlosimulator•  Inputscharacterizelikelihoodofindividualbehaviors•  Individualinstancesindependentandparallelizable

8

n  Open,mul5scale,andscalableparallelexecu5onn  Highlymodularframeworkthatisextensiblen  APIstomonitor/measurerun-5mesta5s5csn  Composeandintegratesimulators/models

èIdealframeworktobuildourco-designsimulatorl  PortunoimplementedasaJavaframeworkl  IntegratePortunointoSSTl  Reuseexis5ngmemorysimulators(DRAMsim2,orourown,HMMsim)

Aparallel,discrete-eventsimulaConframeworkforscalabilityandflexibility

Portunointegratedwith

9

SST::Link

SST::ComponentManagerEventHandler

SST::Component

Worker

EventHandler

SST::EventWorkPacket

JobScheduler

JobWorkers

SerializedParameters

Workloadtracesmappedto

implementa5on

MeasureCostAccrualbasedonCostFunc5onsunderconstraints

ExperimentswithPortunoinSST

10

• Howtodefine,run&shareexperiments?• Howtomaintainprovenance?• Howtolimitburdenofpeerreviewandovercomelimita5onsofpageconstraints?

SSTPortuno

IntegratePortuno&SST

Portuno

11

n  Community-supportedexchangeforcura5onofsimula5on&experimentalresultsincomputerarchitectureandMORE!

Federated Group Collaboration •  Collaborateonexperiments&tools•  Dynamic,interac5veobjects•  Leverageexperiments&tools

http://occam.cs.pitt.edu ACI-1535232 CNS-1305220 CCF-1148646

web portal

software services

Digital Resources

dispatch

retrieve

Repository

artifact

benchmark

tutorial

experiment

web portal

software services

Digital Resources

dispatch

retrieve

Repository

artifact

benchmark

tutorial

experiment

web portal

software services

Digital Resources

dispatch

retrieve

Repository

artifact

benchmark

tutorial

experiment

OCCAM

OCCAM OCCAM

Demand Resources

Digital Libraries

Completeco-designenvironment

12

OCCAMSST

FinalIntegraBonSST

SSTPortuno IntegratePortuno

withSST

Portuno

OCCAM

SST

PortunoIntegrateSSTwithinOCCAM

① Create,runandanalyzeresults(throughsimplifiedGUI)② Collaborateandsharetoconductexperiments③ Reusetoolsandexperimentstoaccelerateresearch④ Repeatabilitywithprovenance⑤ EnablesnewpublicaBonmodelsthroughaccess(demo!)

Whatdoesthisgiveus??

13

Composing, Reproducing, and Sharing Simulations

Area: ModSim Methods and Tools

Debashis Ganguly, William C. Garrison III, David Wilkinson,

Bruce R. Childers, Adam Lee, and Daniel Mosse

Department of Computer Science

University of Pittsburgh

Pittsburgh, Pennsylvania USA

Every year, research groups around the world contribute papersand artifacts to the computer science literature. In many areas, sim-ulation and modeling play key roles in bringing about these newcontributions. Simulation is used to test and validate new ideasprior to their implementation, and thus, the artifacts (software, datasets, benchmarks, etc.) used in simulation are fundamental to theempirical valuation of a research hypothesis.

Oftentimes, the primary focus of a paper is on the validation ofa central hypothesis, and the details surrounding the artifacts usedduring this process are sometimes scarce. Many researchers do notintend to build a foolproof software component to share with thecommunity. Artifacts may end up limited in scope or usability, andhidden assumptions may make the artifact difficult (if not impossi-ble) to reuse, extend, or compose. Many artifacts take a tremendousamount of effort to build and validate and, as such, may remain pri-vate to the research groups that invested in developing them in thefirst place. This limits their availability, increases the difficulty ofvalidating claims made in papers based on these artifacts, and limitsthe ability of others to build upon prior work.

Figure 1: State size vs. Number objects

Addressing this situation necessitates sharing and reproducibil-ity1. While this problem cuts across most CS disciplines, the mod-

case study is openly hosted in the OCCAM collaborative repository(http://occam.cs.pitt.edu) and integrated with Sandia’sStructural Simulation Toolkit (http://sst-simulator.org).

Our simulator, Portuno, conducts cost analyses to explore thesuitability of different access control approaches for a given appli-cation workload. Portuno has been used in an array of analyses, in-cluding evaluating group-centric approaches to information sharingand exploring the communication, computation, and administrativeoverheads associated with cryptographic enforcement of role-basedaccess controls (RBAC) on untrusted cloud platforms. Portuno usesprobabilistic actor-based models of user, administrator, and systembehaviors to generate application traces. These abstract traces arethen mapped into traces in concrete access control systems: thosethat are candidates for implementing the application. Costs are thenaggregated over these candidate system traces. Portuno supportsa wide range of design choices in its actor models, initial systemstates, and other parameters of an experiment. As such, openlysharing the choices that have been made and allowing other re-searchers to modify these choices can lead to a better understandingof the trade-offs among different access controls techniques.

To compose Portuno with other simulations, share the infrastruc-ture, and disseminate the experimental outcomes, it is integratedwith SST and incorporated in OCCAM. SST acts as the driver ofthe underlying access control models, which are implemented inJava. This is a novel use of SST as a backbone for probabilisticmodeling in an area other than computer systems simulation. Italso illustrates interoperability between SST and Java models.

The combination of OCCAM, SST, and Portuno leads to a seam-less environment that is more capable than the sum of its parts. Thisintegrated approach offers the capability to quickly define, run, vi-sualize, and share simulation artifacts and results over a huge de-sign space. It supports an end-to-end workflow for modeling andanalyzing access controls under a variety of scenarios, making iteasier to (a) use Portuno for access control analysis, (b) inspectand augment experiments done by others, and (c) modify Portunoin a contained environment. Simulation results are available fromOCCAM in dynamic, interactive analyses, representing the “pa-

OurModSim16PDFabstractGraphaddedwithlinktoexperiment

Observa5onsfromcurrentwork

14

ObservaBonCryptographically-enforcedRBACpoliciesareprohibi5ve,eveninmoderatescale

How to decrease the overheads associated with enforcing cryptographic access controls in cloud environments?

OCCAM

SST

Portuno

Workinprogress

15

n  Pushre-encryp5ontothedata(don’tbringbulkdatatotheprocessortore-encrypt)

n  Co-designsupportinHybridMemoryCubes(HMC)andtrustedexecu5onenvironmentslikeIntel’sSGX

Portuno GenerateTrace

HMC

Enc.Module

SST

VisualizeResults

Benefit of architectural enhancement to speedup the management of files on untrusted infrastructure.

① Composi5onwithcommonframework② Accelerateresearchbysharing③ Increaseimpactthroughaccess

16

Por5onsofthismaterialisbasedinpartuponworksupportedbytheNa5onalScienceFounda5onunderGrantNumbersCCF-142331,CNS-1012070,ACI-1535232,CNS-1305220andCCF-1148646.Anyopinions,findings,andconclusionsorrecommenda5onsexpressedinthismaterialarethoseoftheauthor(s)anddonotnecessarilyreflect

theviewsoftheNa5onalScienceFounda5on.

TheEndBruceChilders

UniversityofPi;[email protected];.edu

Severalpiecesofclipartinthispresenta?onarefromopenclipart.org