Author: Benedict Delannor Student ID: BD87160 Exam Number: 402541 Supervisor: Hartanto Wijaya Wong Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines Submitted in partial fulfillment of the requirements for the degree of B.Sc. in Economics and Business Administration At Department for Business Studies Aarhus School of Business and Social Sciences Aarhus University May 1, 2012
49
Embed
Simulation Modeling of a Stylized Game Asset Pipeline ...pure.au.dk/portal/files/45289864/Bachelor_Thesis_Benedict_Delannor... · Simulation Modeling of a Stylized Game Asset Pipeline:
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Author: Benedict Delannor
Student ID: BD87160
Exam Number: 402541
Supervisor: Hartanto Wijaya Wong
Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Submitted in partial fulfillment of the requirements for the degree of
B.Sc. in Economics and Business Administration
At
Department for Business Studies
Aarhus School of Business and Social Sciences
Aarhus University
May 1, 2012
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 2 of 49
Abstract
The creation of computer games is a complex process that can take years to finish. Many risk
factors influence this process which makes schedule durations difficult to predict. Since
commitments to deadlines can be costly if schedules are too tight, game companies can
benefit from having a reliable tool for evaluation of schedule duration and milestone
deadlines.
A stylized game asset pipeline model and simulation program for sampling simulated
schedule durations under uncertainty was developed for this thesis in order to investigate its
potential applicability as a support-tool for setting milestone deadlines through selection of
percentile ranks from simulated distributions of schedule durations.
The thesis offers a comprehensive visual presentation of the developed model while
briefly describing the simulation program that has been developed in relation to this thesis
featuring dynamic rescheduling and preemption of tasks. A baseline is established and
analyzed before the model is simulated with four different risk factors in isolation and in
combination and compared against the baseline.
The simulation results show that on average each risk factor in the model adds time to
the schedule duration. However, risk factors in combination do not add time to the baseline
duration equivalent to the sum of the individual factors because of overlapping effects.
Simulation modeling and programming is an intricate and time consuming task and
models can easily get overly complicated in the effort to create a good fit with the system it is
modeling. Still, it might be possible that a simulation model can be developed with enough
detail and accuracy to reflect a real game asset pipeline and evaluate the robustness of
deadlines for the benefit of game companies. Even if the level of detail needed to achieve this
is yet unknown.
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
1.1 PROBLEM STATEMENT .................................................................................................................................... 8
1.2 LITERATURE REVIEW (SHORT FORM) ................................................................................................................. 8
1.4 GENERAL ASSUMPTIONS ................................................................................................................................. 9
4 SIMULATION MODEL ............................................................................................................................ 15
4.1 MODEL ..................................................................................................................................................... 15
TABLE 3 - PRODUCTION TEAM ..................................................................................................................................... 31
TABLE 4 - GAME ASSET REQUIREMENTS ......................................................................................................................... 32
TABLE 5 - STANDARD TASK DURATIONS ......................................................................................................................... 32
TABLE 6 – BASELINE –WORKER AVERAGE (IN STANDARD DAYS) .......................................................................................... 36
TABLE 7 – BASELINE - CHARACTER ARTIST AVERAGE (IN STANDARD DAYS) ............................................................................ 37
TABLE 8 – BASELINE - ENVIRONMENT ARTIST AVERAGE (IN STANDARD DAYS) ........................................................................ 37
TABLE 9 – BASELINE – ANIMATOR AVERAGE (IN STANDARD DAYS) ...................................................................................... 37
TABLE 10 – BASELINE - VFX ARTISTS (IN STANDARD DAYS PER WORKER) .............................................................................. 38
TABLE 11 – BASELINE – SCRIPTERS (IN STANDARD DAYS PER WORKER) ................................................................................. 38
TABLE 12 – BASELINE - LEVEL DESIGNERS (IN STANDARD DAYS PER WORKER) ........................................................................ 38
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 5 of 49
TABLE 13 – BASELINE - TECH ARTISTS (IN STANDARD DAYS PER WORKER) ............................................................................. 39
TABLE 14 – DURATION, SLACK AND RANK – TASK DURATION VARIABILITY (DURATION IN STANDARD DAYS) ................................ 40
TABLE 15 – DURATION, SLACK AND RANK – MACHINE BREAKDOWNS (DURATION IN STANDARD DAYS) ...................................... 42
TABLE 16 – DURATION, SLACK AND RANK – WORKER ABSENCE (DURATION IN STANDARD DAYS) .............................................. 43
FIGURE 20 – WORK RATIO .......................................................................................................................................... 35
FIGURE 22 –DISTRIBUTION OF TOTAL DURATION (TP1) ................................................................................................... 40
FIGURE 23 – DISTRIBUTION OF TOTAL DURATION (TP2) ................................................................................................... 40
FIGURE 24 –DISTRIBUTION OF TOTAL DURATION (MP1) .................................................................................................. 41
FIGURE 25 – DISTRIBUTION OF TOTAL DURATION (MP2) ................................................................................................. 41
FIGURE 26 –DISTRIBUTION OF TOTAL DURATION (WP1) .................................................................................................. 42
FIGURE 27 – DISTRIBUTION OF TOTAL DURATION (WP2) ................................................................................................. 42
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 6 of 49
FIGURE 28 –DISTRIBUTION OF TOTAL DURATION (QP1) ................................................................................................... 44
FIGURE 29 – DISTRIBUTION OF TOTAL DURATION (QP2) .................................................................................................. 44
FIGURE 30 –DISTRIBUTION OF TOTAL DURATION (AP1) ................................................................................................... 45
FIGURE 31 – DISTRIBUTION OF TOTAL DURATION (AP2) .................................................................................................. 45
FIGURE 32 –PLOT OF PERCENTILE RANK VS. SLACK (AP1) ................................................................................................. 46
FIGURE 33 – PLOT OF PERCENTILE RANK VS. SLACK (AP2) ................................................................................................ 46
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 7 of 49
1 Introduction
The computer games industry is a multi-billion dollar industry with a few large companies
and a host of small to medium-sized companies all fiercely competing on technology,
innovation, and product differentiation in a hit-driven market. Only a few game companies
make really big profits while the vast majority of them are struggling.
Most game companies are reliant on contracting with publishers for financing of the
production and marketing of their games in full or in part. Usually these contracts ensure
payment releases to the game company dependent on timely milestone deliveries and often
penalties are installed for not meeting milestone deadlines. Such penalties may force
developers to cut corners and lower quality to meet milestone deadlines which may prove
even costlier for the game company in the long run. It puts the company in a precarious
situation where they will incur costs no matter what they do if their initial estimates have
turned out to be too tight.
Game production setups are as diverse as games are different. Games exist in many
different themes and genres, and on many different platforms. In fact, no two game
productions are exactly the same. Even if game code and assets to some extent can be reused
the drive to push technology and be innovative makes for an ever changing production
environment. Estimating schedule durations and establishing robust deadlines is a feat which
many game companies fail to deliver on. Not only is the production of games inherently
complex it also faces many sources of risk and uncertainty which makes for a huge challenge
in planning and scheduling. Many game companies use the critical path method (CPM) and
simple tools like Microsoft Project for scheduling, and by adding slack (allowance) to their
schedules they try to accommodate the impact of uncertainty. This is seemingly an effective
approach. However, it has led to many a headache among managers as projects often do not
meet deadlines, even with high levels of slack in the schedule. Causes for this can be many.
Managers and producers have different perspectives, knowledge and agendas. Managers tend
to want shorter development times and producers typically want as robust a schedule as
possible. Negotiating schedules with regards to milestone deadlines easily becomes a game of
give and take. Producers add extra slack to their schedules because they know managers will
demand shorter development time regardless, and managers anticipate that producers will add
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 8 of 49
extra slack so they demand shorter development time regardless, without knowing the exact
details and impact on the schedule.
Game companies would benefit from having a tool that could provide robust and
realistic deadlines that both managers and producers could accept and believe in; chances are
that such a tool would be adopted by game companies and that more game development
projects would finish on time in the future.
1.1 Problem Statement
The aim of this thesis is to develop a simulation model of a stylized game asset
pipeline that can be used to demonstrate how schedule durations can be simulated and
deadlines chosen from associated percentile ranks from the simulated distribution output. For
this purpose, a simulation model and program is developed featuring dynamic rescheduling
and preemption of tasks. The simulation program is used to establish a deterministic baseline
(a.k.a. predictive schedule) before simulating scenarios incorporating stochastic variation.
Firstly, the baseline is analyzed by decomposition of resources and time usage before
comparison. Secondly, the impact of four risk factors on schedule duration, in isolation and in
combination, is compared against the baseline, with focus on the slack needed for the
baseline to meet desired percentile ranks. Finally, the plausibility that a simulation model
could be developed with enough detail to estimate schedule durations for setting realistic
milestone deadlines for actual game asset pipelines is considered.
1.2 Literature Review (Short Form)
Carter (2004) gives a practical introduction to the game asset pipeline.
Chandler (2010) describes aspects of game production from a producer’s perspective.
Law (2007) gives a comprehensive walkthrough of the process of making a discrete-event
system.
Hulett (2009) offers a practical approach to analyzing schedule risk.
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 9 of 49
Herroelen and Leus (2004) review possible procedures for generation of robust schedules and
offer a framework for identifying proper scheduling methods
Herroelen and Leus (2005) review the fundamental approaches for scheduling under
uncertainty
Rubinstein and Kroese (2008) give a rigorous introduction to simulation and the Monte Carlo
Method.
Vieira et al. (2003) describes a framework for understanding rescheduling strategies, policies,
and methods.
Note that the above literature list is not a comprehensive list. Only the most essential sources
are reviewed here. References for all sources can be found in the References section on the
last page of the thesis.
1.3 Approach
The process of acquiring data for this thesis was iteratively exercised in four stages. Firstly,
the model was conceived. Secondly, the model was developed and tested. Thirdly, data was
created using the developed simulation program. Finally, data was validated and analyzed.
1.4 General Assumptions
The model is stylized and is assumed to be a closed system, i.e. no interference from outside
the modeled system.
Independence is assumed between all stochastic variables in the model.
Modeling Programming Simulation Data Analysis
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 10 of 49
1.5 Delimitation
The thesis strictly focuses on the game asset pipeline. Any other organizational aspect is
ignored. No optimization will be performed on schedules or resources or any other aspect of
the simulation for that matter. Stochastic variables and distributions have been simplified as
the correctness of distributions is not a concern for this thesis.
1.6 Definition of Terms
Table 1 – Definition of Terms
Term Description
Deadline Robustness Deadline robustness is defined here as the percentile rank of the simulated distribution of schedule
durations. E.g. a deadline robustness of 99% means the 99th percentile of the associated simulated
distribution of schedule durations. This approximates to a 99% certainty that the simulated schedule
duration will not be longer than the duration at the 99th percentile, given a high number of
simulated observations.
Realistic Deadline A deadline is softly defined as realistic if there is a “good chance” that the deadline can be met
under real circumstances. The “good chance” is up to the manager or producer to define and is done
by choosing a suitable percentile rank for calculating the slack percentage and deadline (schedule
duration).
Artist Artist is the term for a worker working with creating art assets for the game.
- Character Artist An artist who specializes in creating models, maps and textures for character assets
- Environment Artist Environment artists create environment assets.
- Animator Create rigs on models and animations
- VFX Artist Creates visual effects for various game assets
- Scripter Scripters create game logic (scripts). Even though scripters technically do programming they are
part of the asset production team.
- Tech Artist The tech artist bridges the gap between programmers and artists and in model developed in this
thesis tech artist also handle the technical conversion of art assets to file formats usable in the game
build
- Level Designer The level designer designs levels in a level editor which requires game assets and relies on dummy
assets for those assets that are not finished yet.
Asset or Game Asset An asset is a final object creation (game part) going into the game. There are six types of assets:
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 46 of 49
AP1 shows a duration target range of 129.23 standard days while AP2 has a range of
only 117.60. The biggest impact comes from task duration variability.
Interestingly, in terms of slack percentage the range for AP1 is [19.60%; 52.07%]
which is quite a bit wider than the range for AP2 which is [27.64%; 43.00%]. Therefore
smaller changes in the slack percentage are needed for AP2 to cause the same rank change.
This means that overall higher robustness can be achieved by having two consecutive and
overlapping pipelines rather than one.
Using a standard of 25% slack added to the baseline is not enough to reach the 99%
percentile when applying the risk factors as defined for the model. Naturally these will not be
the same in real applications. The actual numbers here are not as important as the method
proposed. Managers and producers must model their actual pipelines and use risk factors that
fit their situation. Also they must choose the percentile that they feel comfortable with.
Figure 32 –Plot of Percentile Rank vs. Slack (AP1)
Figure 33 – Plot of Percentile Rank vs. Slack (AP2)
Calculations using distribution percentile ranks allows the producer and manager to
either evaluate the robustness of a selected slack percentage or pick a percentile rank and
calculate the needed slack percentage. The mean of one pipeline in AP2 produces the same
ranks and slack percentage although the range and standard deviation is halved.
The standard deviation for task duration variability is quite high compared with the
other factors.
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 47 of 49
7 Validation
The simulation program has undergone extensive debugging and heuristic testing including
validation of output in Excel and SPSS by use of checkpoint calculations and visual
inspection of variable distributions. Furthermore validation of the simulation output has been
done by creation of a simple meta model through linear regression.
All tests have shown output data to be consistent and valid for the model. Even if the
model itself, the model structure and input variables, cannot be validated empirically for
obvious reasons, and comparison with the baseline may not be statistically sound due to the
low number of simulation runs (1000), the results presented are valid for the purpose of this
thesis.
8 Conclusion
The results show that each risk factor affects the total schedule duration to some extent so on
average schedules have longer durations than the baseline. Still it is no trivial matter to assess
the impact beforehand when risk factors are combined. By simulation it is possible to capture
the dynamic nature of the production pipeline model and gauge the results of combinations of
risk factors that may exercise overlapping effects.
Simulation appears to be a strong tool for gaining valuable and detailed information
about how a milestone deadline is impacted by combinations of various risk factors. Even if
this thesis only analyzes the effect on deadlines the simulation program may store
distributions of many other variables that could potentially be analyzed as well to support the
decision-making process. The simulated distributions and robustness metrics for deadlines
allow managers and planners to jointly engage scheduling at a more informed level and may
help in defining realistic deadlines in real production settings.
Even though simulation may be a very useful tool for defining realistic deadlines in
games productions, the initial cost of setting them up can be high. Especially, simulation
modeling and programming in a general-purpose language is a time consuming endeavor. So
before taking on simulation as a support-tool for decision-making a cost-benefit analysis
should be completed.
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 48 of 49
Based on the experience with modeling and programming the simulation developed
for this thesis it does seem very plausible that a model with high enough detail to capture all
relevant dynamics of the game production could be developed. Obviously, the simulation
model must exhibit a close fit with the actual production system in order for the results to be
useful in decision-making. Exactly how close a fit is needed is yet unknown. This is
definitely something that could be interesting to test in future studies.
9 Further Studies
There are mainly three things that would be interesting to do in the future:
1. Enhance the model to include costs.
2. Expand the model to encompass the whole games development process.
3. Detail the model to fit an existing production pipeline and run empirical tests.
Incorporating costs in the model should be fairly simple, but interesting nonetheless.
Creating a model for the whole game development process is somewhat more
involving but the benefit from doing so possibly outweighs the effort.
One would be required to look into an actual production pipeline using empirical data
in order to create a highly detailed simulation model, taking into account all conceivable risks
and uncertainties, to approximate a good fit. The model should include elements like
information flow, learning curve, worker morale, crunch-time, over-time, etc. and be vary to
match real workflows, in order to capture enough information for reliable simulation results.
Simulation of actual days using a calendar to map workers’ scheduled time off and an
empirical model for sickness would be needed as well as simulation of scheduled breaks and
micro-breaks and startup times. Dynamic resource allocation for workers leaving, being laid
off or hired, partial re-use of asset from other productions – the list goes on. Having studied
all these elements it is possible that findings will show that some of these elements are
negligible and approximations can be found to create a good model fit with the real world
without too many variables in the simulation.
BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines
Author: Benedict Delannor Page 49 of 49
10 References
AUTODESK. 2012. Modern Pipeline [Online]. Available: http://usa.autodesk.com/adsk/servlet/index?id=16116742&siteID=123112 [Accessed April 24, 2012].
BARMBY, T. A., ORME, C. D. & TREBLE, J. G. 1991. WORKER ABSENTEEISM: AN ANALYSIS USING MICRODATA. Economic Journal, 101, 214-229.
CARTER, B. 2004. The Game Asset Pipeline, Delmar Cengage Learning. CHANDLER, H. M. 2010. The Game Production Handbook, Jones and Bartlett Publishers. GUSMÃO, F., ORTEGA, E. & CORDEIRO, G. 2011. The generalized inverse Weibull distribution.
Statistical Papers, 52, 591-619. HERROELEN, W. & LEUS, R. 2004. Robust and reactive project scheduling: a review and classification
of procedures. International Journal of Production Research, 42, 1599-1620. HERROELEN, W. & LEUS, R. 2005. Project scheduling under uncertainty: Survey and research
potentials. European Journal of Operational Research, 165, 289-306. HULETT, D. 2009. Practical Schedule Risk Analysis, Gower Publishing Company. LAW, A. 2007. Simulation Modeling and Analysis, McGraw-Hill Higher Education. MATSUMOTO, M. 1998. Mersenne Twister: A 623-Dimensionally Equidistributed Uniform Pseudo-
Random Number Generator. ACM Transactions on Modeling and Computer Simulation, 8, 3-30.
MSDN. 2010. Introduction to the C# Language and the .NET Framework [Online]. Microsoft. Available: http://msdn.microsoft.com/en-us/library/z1zx9t92.aspx [Accessed May 1, 2012].
RUBINSTEIN, R. Y. & KROESE, D. P. 2008. Simulation and the Monte Carlo Method, John Wiley & Sons.
VIEIRA, G. E., HERRMANN, J. W. & LIN, E. 2003. Rescheduling Manufacturing Systems: A Framework of Strategies, Policies, and Methods. Journal of Scheduling, 6, 39-62.
WIKIPEDIA. 2012. Polymorphism in object-oriented programming [Online]. Available: http://en.wikipedia.org/wiki/Polymorphism_in_object-oriented_programming [Accessed April 30, 2012].