MOVES Curriculum Update · MOVES Curriculum Update LTC John Morgan 23 MAY 2017 ... MV-4025 (3-2) Cognitive and ... • Observed CS/IA techniques used at NTC and Cyber OPFOR operations;

MOVES Curriculum Update

LTC John Morgan23 MAY [email protected]

• 2014– MOVESProgramAssessmentconductedbyJohnsHopkinsUniversity/AppliedPhysicsLaboncontractfromMCMSO /NMSO

– Broadbottom-upsurveyofallUSMC8825(M&SOfficers)• Currentjobrequirements• Importantconceptstounderstand• RecallofclassestakeninMOVES• MOVESteamhelpedquantifylevelofmasteryrequired

Background

2

• 2015– “Modeling,VirtualEnvironments,andSimulationCurriculumAssessmentFY2015InitialAssessment”(NSAD-R-15-011,Feb2015)• Establishedbaselineof93necessary“knowledgeitems”(“KI’s”)andlevelofmasteryneededforeach

• KI’sgroupedintoeightESRs• MOVESidentified30totalissuestoaddress

– Primarily:courses’contentandfocus

Background

3

• 2016–AMSO Project($330K)

Background

4

• 2017–NavalResearchProgramProject($150K)–NMSO Project($300K)

Background

5

6

QUARTER COURSES SEMINARS

RefresherSummer

(15.5credithrs.)

CSR-100(2-2)RefresherforBeginning

Programming

MA-1113(4-0)SingleVariableCalculusI

MA-2025(4-1)LogicandDiscrete

Mathematics

MA-1114(4-0)SingleVariableCalculusII

withMatrixAlgebra

Quarter1Fall

(19.0credithrs.)

CS-2072(4-2)FundamentalObject

OrientedPrograminginJavaScript

MV-3101(4-0)IntroductiontoDoD

Modelingand Simulation

MA-3042(4-0)LinearAlgebra

OS-3111(4-0)Probability&Statisticsfor

HSI andMOVES

MV-2921(2-0)IntroductiontoMOVES

Quarter2Winter

(20.0credithrs.)

CS-2173(4-2)JavaasaSecondLanguage

MV-3202(3-2)IntroductiontoComputer

Graphics

MV-3025(4-1)Artificial

IntelligenceforSimulation

OS-3113(4-1)DataAnalysisforHSIand

MOVES

MV-3922(2-0)IntroductiontoVirtual

EnvironmentTechnology

Quarter3Spring

(20.5 credithrs.)

CS-3600(4-2)Introduction toComputer

Security

MV-3302(4-1)IntroductiontoDiscrete

EventModeling

MV-3203(3-2)Graphical Simulation

OS-3112(4-2)StatisticsandDesignof

Experiments

MV-3923(2-0)IntroductiontoResearch

inMOVES

Quarter4Summer

(18.0 credithrs.)

MV-4025(3-2)CognitiveandBehavioralModelingforSimulations

MV-4002(4-1)SimulationandTraining

MV-4502(2-4)SimulationDevelopment

Practicum

MV-3500(3-2)Internetwork

CommunicationsandSimulation

MV-4924(1-1)Research Seminarin

MOVES

Quarter5Fall

(18.0 credithrs.)

OA/MV-4655(4-0)IntroductiontoJointCombatModeling

MV-4503(2-4)SimulationInteroperability

Practicum

MV-4920(4-0)AdvancedTopicsin

ModelingandSimulation

MV-4001(4-1)HumanFactorsof Virtual

Environments

MV-4924(1-1)ResearchSeminar in

MOVES

Quarter6Winter

(19.0 credithrs.)

MV-4501(2-4)SimulationApplication

Practicum

MN-3331(5-1)PrinciplesofAcquisitionandProgramMgmt.

MV-4657(3-2)Modeling&Simulationfor

SSTROperations

MV-0810(0-8)ThesisResearch

MV-4924(1-1)Research Seminarin

MOVES

Quarter7Spring

(17.5 credithrs.)

SE-3100(3-2)FundamentalsofSystem

Engineering

MV-4460(4-0)ManagementofM&S

Development



MV-4924(1-1)ResearchSeminarin

MOVES

# OBJECTIVE RESULTS1 Reviewallcourse

materialstoensurethattheyarecongruentwiththenewESRs

• CS2072,MV2921,MV3025,MV3203,MV3922,MV4001,MV4025,MV4501,MV4502,MV4924reviewedandnewsyllabusgenerated

• MV3302reviewed, decreasedcoverageongenerationofrandomvariates,increasedcoverageofdataanalysis,andupdatedsyllabus

• MV4002reviewed course,incorporatednewLVCmaterials,andupdatedsyllabus

FY16 AMSO Project

7

# OBJECTIVE RESULTS2 ReviewallMOVEScourses

foradditionalplaceswhereCyberSecurityandInformationAssurance(CS/IA)canbecovered.

• ObservedCS/IAtechniquesusedatNTCandCyberOPFORoperations;discussedCS/IAwithSMEsatNPS;incorporatedupdatesintoclasses.

• Increasedcoverageofcybersecuritypolicyincludingdifferencesbetweentheservices.

• IncreasedcoverageofCS/IAprinciplesthatarefollowedwithinorganizations,thatareappliedinsimulationenvironments,andexamplerepresentationsofCS/IAinsims.

• ArrangingforCS/IAguestspeakers.

FY16 AMSO Project

8

# OBJECTIVE RESULTS3&4

ConductresearchtodeterminewhichMissionCommand/C4Idevicesandprogram ofrecordsimulationcanbeusedwithintheMOVEScurriculumandtherequirementstoimplement

• SimIS didanexcellentjobofseeking,obtaining,andsynthesizingalloftheinformationthatwerequested.

• Hugeamountofrawinformationgatheredfromhundredsofsourceswithreferences.

• WeareworkingonproducingarecommendationforAMSO,NMSO,andMCMSMOforfuturefielding.

FY16 AMSO Project

9

Simulation SuitabilityforClassroomEnvironment

ArchitectureManagementIntegrationEnvironment(AMIE)

Suitable

NextGenerationThreat System(NGTS)

Suitable

MAGTF TacticalWarfareSimulation(MTWS)

NotSuitable– basedonrecommendationofPM(recommendsitevisit)

CombinedArmsCommandandControlTrainerUpgradeSimulation(CACCTUS)

NotSuitable– basedonrecommendationofPM(recommendsitevisit)

FrameworkforAssessingCostandTechnology(FACT)

Suitable

Simulation Study Results

10

Simulation SuitabilityforClassroomEnvironment

VirtualBattlespace3(VBS3) SuitableStandardInterfaceforMultiplePlatformLinkEvaluation(SIMPLE)

Suitable

OneSemi-Automated Forces(OneSAF)

Not Suitable– significantHW/SWoverhead(VMWaremaybeanoption)

ArmyLowOverhead TrainingToolkit(ALOTT)

Suitable

JointSemi-AutomatedForces(JSAF) SuitableVRForcesbyVTMAK Suitable

Simulation Study Results

11

C4I Device Study Results

12

AFATDS CPOF JBC-P DCGS-A C2PC

CharacteristicsFiresPlanning

FireCoordinationForeignFears

SituationawarenessCollborationMulti-echelon

ForceTrackingSystemCollboration

Lower-echelonMobile/Portal

IntelSystemComplexArchitecture800+datasourcesLimited-echelon

TacticalCOPClientJTCWModule(legacy)

FieldSupportAvailableSimplehardware

SupportAvailableSupportAvailable

LotsofconfigurationSupportAvailable

LotsofconfigurationSupportUnavailableSimplehardware

Operate Nothingsignificant Difficulttooperate Nothingsignificant Difficulttooperate NothingsignificantSustain Nothingsignificant Heavierlicenses Nothingsignificant Highercosts* Highercosts*

SuitabilityInteroperable

PopularPossiblereplacement

NotasinteroperablePopular

PowerfulComplex

Replaced

C4I Device Study Results

13

GCCS-TCO JTCW MTC2 ADSI GCCS-J

CharacteristicsTacticalCOPServicesAnalysisk/Planning

TacticalCOPClientCOPEditing

SituationAwarenessMulti-echelonMulti-domain

COP/FusionLinkControlCOPEditingMulti-Service"AllShips"

COP/FusionAnalysis/Planning

FieldSupportUnavailableLotsofconfiguration

SupportUnavailableSimplehardware

SupportAvailableSoftware-only

SupportAvailableComplexhardwareLotsofconfiguration

SupportUnavailableLotsofconfiguration

Operate Nothingsignificant Nothingsignificant Nothingsignificant Nothingsignificant NothingsignificantSustain Highercosts* Nothingsignificant Nothingsignificant Heavierlicenses Highercosts*

SuitabilityInteroperablePopularCOP

InteroperablePopulartacticalCOP

clientDescoped Possiblereplacement

InteroperablePopularCOP

• Higher-echelon,interoperable– GCCS-J

• Lower-echelonCOPclients,popular,multi-Service,directLVCstimulation,littlebitoffunctionaldiversity– JBC-P,JTCW,AFATDS

• C2PCreplacedbyJTCW• RumorsofCPOFreplacement,difficulttouse• DCGS-Ainfeasible• ADSItryingtobereplacedbyLMMT• MTC2recentlyde-scoped

C4I Device Findings

# OBJECTIVE RESULTS6 Identifyanddevelopnew

coursecontentforMV4001(HumanFactorsofVirtualEnvironments)coveringTraditionalandNon-TraditionalHumanSystemInterfaces

• Addedinformationtothecourseaboutmostrecentresearcheffortsinthisdomain,aswellasillustrationsandanalysisofarangeofnoveldevices(bothinputandoutput)andideasfornewtypesofinterfacesanduserinteractionthathaveemergedrecently.

• Purchasedseveralexamplesofnoveldevices;planedandexecutedaseriesofhands-onexperienceswithstudents.Studentshavealreadystartedusingthemforthesisresearchandclassassignments.

FY16 AMSO Project

15

MOVES Gadgets Den:Resource for Student Thesis and Classroom Work

Amela Sadagic, PhD

3D Scanning, Input Controllers

Ø Lab assignments and hands-on experience for MOVES courses:

• MV3922, MV4001 & MV4924

Ø Others:• CS3004

Devices: Tango Tablet (2), iSense 3D scanner for iPhone 6 (3), iSense for iPad Air2 (1), Sense 3D scanner (1), Leap Motion VR bundle (5), Bluetooth Telescopic Controller (14), Bluetooth Gamepad for Android (10), Bluetooth Gamepad (4)

VR Headsets


• MV3922, MV4001 & MV4924


Devices: Samsung Gear VR (1), VR One by Zeiss (1), VR Box (4), VR Beatz (11)

Other Devices

Devices: KOR-FX game ‘jacket’ (1), LG 360 camera (3)


• MV3922, MV4001 & MV4924


Oculus Rift: DK1, DK2, Production Unit

LT Geoghegan’s thesis

(1st place SECNAV Innovation 2015 Award)

LT Greunke’s thesis

(3rd place SECNAV Innovation 2015 Award)

Ø Lab demos and hands-on experience for MOVES courses:

• MV3922, MV4001 & MV4924


HTC Vive + VR-ready PC workstation

Ø Sqn Ldr Faisal Rashid MOVES thesis.Ø Lab assignments and hands-on experience for

MOVES courses:

• MV3922, MV4001 & MV4924


Hololens

Lab assignments and hands-on experience for MOVES courses:

• MV3922, MV4001 & MV4924

Others:• CS3004• Support of thesis work for

DA students LCDR Teti and MAJ Scott Fennel

Future: MOVES student thesis AR work

ONR Funding: Microsoft Surface Tablets + 3D Models

Capt Arthur

MOVES thesis

Maj Gibson

MOVES thesis

Collection of Mobile VR Headsets

Virtual Sand Table: Still going strong since 2009!


coursecontentforMV4001(HumanFactorsofVirtualEnvironments)toenhancecoverageofHumanPerformanceEvaluation

• Coursecontentreviewedandupdatedaddinginformationaboutmostrecentresearcheffortsinthisdomain.

• MOVEShasacquiredseveraldevicesthatallowresearcherstomeasureEEGresponses.

• MOVESacquiredaneye-trackingsystemthathasbeenusedinseveralstudentthesisandresearchprojectefforts.Itwillalsobeusedtoenrichcourseofferings,planandexecuteasetofhands-onexperiencesbythestudents

FY16 AMSO Project

26


coursecontentforMV4460(ManagementofModelingandSimulationDevelopment)toenhancecoverageofCyberPolicyandCyberSecurity/InformationAssurance.

• GatheredthemostcurrentinformationthatpertainstoNavalsystems.Therearesomesubtledifferencesbetweentheserviceswhichwewillpointoutinthecourse.

• FilteredthegeneralcybersecuritypoliciesforinformationthatpertainsdirectlytoM&SofficerswithafocusonshepherdinganM&Ssolutionthroughthedevelopmentprocessandgetitonoperationnetworksforuse.

• Cybersecuritypolicyinformationhasbeengatheredandrevised.

FY16 AMSO Project

27


coursecontentforMV4460(ManagementofModelingandSimulationDevelopment)toenhancecoverageofDIACAPtothenew“RiskManagementFramework”standard.

• Identifiedafourmodule“course”thatSPAWARhasontheNETCLMSthatcoverseverythingwewillneedforthepresent.Wecapturedthematerial andmadeitavailableforourstudentsoutsideoftheLMS.

• ThefirsttwomodulesmeettheESRrequirements.Thelasttwoaremoreofthenutsandboltsofthesystemsandformsthatyouhavetowadethroughtogetasystemcertified.

FY16 AMSO Project

28


coursecontentforMV4502(SimulationDevelopmentPracticum)toenhancecoverageofValidationandVerification(V&V).

• IncreaseemphasisonVV&AstandarddocumentationfromMIL-STD-3022;inparticulartheRequirementsTraceabilityMatrix(withemphasisonhavingstudentsformulateacceptabilitycriteria,metrics,andmeasurementsforspecificsimulationrequirements,andunderstandingtheBasisofComparisonexpressedinthesimulationconceptualmodelasthefoundationformodelvalidation).

FY16 AMSO Project

29


coursecontentforMV4502(SimulationDevelopmentPracticum)toenhancecoverageofValidationandVerification(V&V).

(continued)• ProvideinstructionontheVV&Aoverlayto

theFederationDevelopmentandExecutionProcess(IEEERecommendedPractice1516.4-2007;thereisalsoarecommendedpracticeforVV&AofDistributedInteractiveSimulationfederations,IEEE1278.4-2003,thatcanbeexamined),anditsevolutiontoanoverlaytotheDistributedSystemEngineeringandExecutionProcess(DSEEP).

FY16 AMSO Project

30


coursecontentforMV4924(CurrentTopicsinModeling,VirtualEnvironmentsandSimulation)toenhancecoverageofSantos.

• NewmaterialthatoutlinescapabilitiesofSantossimulationandunderlyingmodelshasbeencreated.Weplantomergetwoclasses(twohourslongclassactivity)duringwhichstudentswillbeexposedtofollowingmaterialsandactivities:o BasicsofSantossystemo Introductiontotheuserinterface,ando Hands-onexperiencewiththesoftware

includingexecutionofasimpletaskwiththehelpofsessionmoderator

FY16 AMSO Project

31

# OBJECTIVE RESULTS15 FundStudentTravelto

ConferencesinordertoimproveMarketResearcheducation

• Identifiedasetofconferences,meetingsandorganizationsthataregoodcandidatesforstudentstoattendinordertoimprovetheirabilitytoconductMarketResearch.o Inter-service/IndustryTraining,Simulation,

andEducationConference(I/ITSEC)o IEEEVirtualRealityo IEEE3DUI(co-locatedwithIEEEVR)o GameDevelopersConferenceo SiliconValleyVirtualRealityConference&

Expoo SiliconValleyVirtualReality(SVVR)Meetup,

monthlymeetingorganizedinoraroundSanJose,http://svvr.com/

FY16 AMSO Project

32


ConferencesinordertoimproveMarketResearcheducation

(continued)• Additionalevents:

o ACMSIGGRAPHconferenceo ACMSIGCHIconferenceo IEEEInternationalSymposiumonMixedand

AugmentedReality(ISMAR)o ArtificialIntelligenceandInteractiveDigital

Entertainment(AIIDE)o BRiMS:BehaviorRepresentationinModeling

andSimulationo IEEEConferenceonComputational

IntelligenceandGames(CIG)o SimulationInteroperabilityWorkshopsby

SimulationInteroperabilityStandardsOrganization(SISO)

FY16 AMSO Project

33


ConferencesinordertoimproveMarketResearcheducation(11studentsinFY16cohort,1tripeach)

• StudenttriptotheSimulationInteroperabilityWorkshop2016(SIW2016)inOrlando,CAo Thiswasaverysuccessfultriponall

accounts.o SIW organizerssaidthattheywouldlike

tohaveourstudentsthereeveryyear.o Studentfeedbackontheconceptof

conductingthesetripswasalsoverypositive.

o Student tripreportswerecollected.

FY16 AMSO Project

34

# OBJECTIVE RESULTS16 Developcoursematerials

forMV3025(ArtificialIntelligenceforSimulation)

• Developedlaboratoryassignmentsthatwouldhelpstudentsaccomplishimplementationofafull-blownA*searchalgorithmthattheycouldapplytosimulatedcombatunitstoengageenemyforceshiddenbehindterrain.

• EachlabisanindividualproblemthatrequiressolvingaparticularproblemwithskillsetsleveragingtheJavascriptprogramminglanguage.

FY16 AMSO Project

35

# OBJECTIVE RESULTS16 Developcoursematerials

forMV3025(ArtificialIntelligenceforSimulation)

(continued)• Theselabscover:

o DataStructureso Sortingo Searchingo OtherToolsandTechniques

• MostoftheprogramsuseaDataDrivenDocument(D^3)interactivevisualizationlibrary(writteninjavascript)toillustrateprogramsequenceandresultsingraphicalform.

FY16 AMSO Project

36

# OBJECTIVE RESULTS17 DevelopaSiteVisit

ConceptandconductatestvisittoNTCforpossibleinclusioninfutureiterationsofMV4501(SimulationApplicationPracticum)aspartoftheinstructiononLive-Virtual-Constructivetrainingevents.

• Developedan NTCSiteVisitconceptthatisbasedonattemptingtoexposestudentstoasmanyMOVESESRsaspossible.

• ConceptincludesLearningObjectivesinthefollowingbroadareas:o LVCArchitectureo LVCTrainingo LVCExerciseDesigno LVCAARso SimulationSupportedTraining

FY16 AMSO Project

37

# OBJECTIVE RESULTS17 DevelopaSiteVisit

ConceptandconductatestvisittoNTCforpossibleinclusioninfutureiterationsofMV4501(SimulationApplicationPracticum)aspartoftheinstructiononLive-Virtual-Constructivetrainingevents.

• ThefirstNTC sitevisitwasconducted24-27FEB2017usingNRPfunding.o Overallthetripwasagreatsuccess.o Needtoaddressissueofforeign

studentsbeingabletoattendo Needtofindafundingsourceforfuture

iterations

FY16 AMSO Project

38

• FY17NRPProject– $150K– ConductedNTCSiteVisitinFEB2017(8students,2faculty)

– DevelopingrecommendationsforwhichsimulationsandC4Idevicesshouldbeaddedtothecurriculum

– ContinuingdevelopmentofnewMV3025(AIforSimulation)

– UpdatingMV4025(CognitiveandBehavioralModelingforSimulations)

– UpdatingMV4657(M&SforSSTROperations)– InitiatingdevelopmentofcoursetoreplaceOS/MV4655

Ongoing Efforts

39

• FY17NMSOProject– $300K(stillunfundedduetooperatingunderCRA)– UpdatingMV3922(IntotoVETechnology)– UpdatingOS3122(StatisticsandDesignofExperiments)– UpdatingMV3203(GraphicalSimulation)– UpdatingCS2072(FundamentalObjectOrientedProgramminginJavaScript)

– UpdatingMV4001(HumanFactorsofVEs)– UpdatingMV4924(CurrentTopicsinMOVES/ResearchSeminar)

Ongoing Efforts

40

• FY17NMSO Project(Continued)– AddingguestlecturetoMV4002(SimulationandTraining)– CompletingdevelopmentofcoursetoreplaceOA/MV4655(newcoursetitle“ComputationalCombatModels”)

– UpdatingtheLVCSiteVisitConceptforMV4501(SimulationApplicationPracticum)

– FundingalumniandguestspeakertraveltotheMAWG– M&SCommunityOutreach– Providingstudentthesissupport(~$30Kfortravelandprogrammingsupport)

Ongoing Efforts

41

QUESTIONS?

MOVESCurriculumUpdate

42

BACKUPSLIDES

MOVES Curriculum Update · MOVES Curriculum Update LTC John Morgan 23 MAY 2017 ... MV-4025 (3-2) Cognitive and ... • Observed CS/IA techniques used at NTC and Cyber OPFOR operations;

Documents