Predictive Auto Scaling of In-Memory Data Stores TEAM MEMBERS: ● AKSHAY RAUL ● RAHUL IYER ● SHWETHA KALYANARAMAN
Predictive Auto Scaling of In-Memory Data Stores
TEAM MEMBERS:● AKSHAY RAUL● RAHUL IYER● SHWETHA KALYANARAMAN
Motivation
● Caching is a major part of many application infrastructures
● Service needs to be resilient, reliable, and scalable
● Costly to maintain multiple nodes to service clients
● Auto-Scaling to the rescue!
What are In-Memory Data Stores?
● Type of non-relational database
● Relies on memory for storage
● Helps in minimizing response time
Introduction to Redis
● In-Memory data store
● Simple key value store
● Supports sharding and replication
● Lightweight and multi-platform
● Provides a lot of performance metrics
Setup Infrastructure: AWS
● ECS (Docker containers running Redis)
● Easy to scale
● redis-cli to capture Redis’ metrics
● collectd to capture system level metrics
● Aggregated metrics to be sent out to the prediction engine
Load Testing
● Huge set of supported operations
● Easy to program
● Varied workload intensities
Metrics Considered
● CPU/Memory usage from Redis
● Request Latency
● System’s CPU/Memory usage from collectd
Approach
● Based on Utilization factor
● Predicting Future Workload → Predicting the need for Autoscaling
● Using Prediction Algorithms
Prediction Algorithms
● Recurrent Neural Network
● ARIMA MODEL
RNN
● Make use of sequential information
● Output is dependant on previous computations
● RNNs are used in a lot of areas:
○ Speech Recognition
○ Machine Translation
RNN
ARIMA Model● Stands for Autoregressive integrated moving average
● Model is fitted to time series data either to better
understand the data or to predict future points in the
series (forecasting).
● Used in short-term forecasting that requires 40 historic
data points
● It uses autocorrelation to correlate data points which are
at specific time interval apart
ARIMA Model● AR: Autoregression. A model that uses the dependent
relationship between an observation and some number of lagged observations.
● I: Integrated. The use of differencing of raw observations (e.g. subtracting an observation from an observation at the previous time step) in order to make the time series stationary.
● MA: Moving Average. A model that uses the dependency between an observation and a residual error from a moving average model applied to lagged observations.
Algorithm Skeleton
Utilization_Factor=(Load Present+Predicted Load)
If Utilization_Factor > 0.70
Autoscale!!
If Utilization_Factor <0.30
Scale down!!
Benefits of Auto Scaling
● Better fault tolerance
● Better availability
● Better cost management
● Optimization of resource needs
Work Distribution● AKSHAY - Data Cleaning, Data Selection and Training the ARIMA
Model for prediction. Experimenting by varying lag observations, raw observations and moving average component to evaluate the performance of the model
● SHWETHA - Data Pre-Processing and Training of Recurrent Neural Network for prediction of incoming load. Experimenting with different hidden layers and compare analysis with ARIMA model
● RAHUL - Setting up the Redis cluster and will be working on load testing to capture metrics. Also, will manage the addition and removal of nodes as per the prediction.
QUESTIONS?
References● Chung-Horng Lung Ali Yadavar Nikravesh, Samuel A. Ajila. 2015. Towards an Autonomic Auto-Scaling Prediction
System for Cloud Resource Provisioning. IEEE (2015). https://doi.org/10.1109/SEAMS.2015.22● Shu Tao Nikos Anerousis Arijit Khan, Xifeng Yan. 2012. Workload characterization and prediction in the cloud: A
multiple time series approach. (2012). https://ieeexplore.ieee.org/document/6212065● Guo Ma Bin Xie, Guanyi Sun. 2017. Prediction-based autoscaling for container-based PaaS system. IEEE (2017).
https://ieeexplore-ieee-org/document/8239026● Emiliano Casalicchio and Vanessa Perciballi. 2017. Auto-Scaling of Containers: The Impact of Relative and Absolute
Metrics. 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2017). https://doi.org/10.1109/fas-w.2017.149
● Yao-Tsung Yang Li-Der Chou Chia-Wei Tseng, Ming-Shiun Tsai. 2017. A Rapid Auto-Scaling Mechanism in Cloud Computing Environment. In Int’l Conf. Grid, Cloud, & Cluster Computing (GCC’17) CSREA Press. https://csce.ucmss.com/cr/books/2017/LFS/CSREA2017/GCC6087.pdf
● Zhenhuan Gong, Xiaohui Gu, and John Wilkes. 2010. PRESS: PRedictive Elastic ReSource Scaling for cloud systems. 2010 International Conference on Network and Service Management (2010). https://doi.org/10.1109/cnsm.2010.5691343
● Miguel-Alonso J. & Lozano Lorido-Botran, T. 2014. A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments. Grid Computing (2014). https://doi.org/10.1007/s10723-014-9314-7
● Marty Humphrey Ming Mao. 2011. Auto-Scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows. IEEE (2011). https://ieeexplore.ieee.org/document/7027559
● Alexei Karve Andrzej Kochut Muhammad Wajahat, Anshul Gandhi. 2016. Using machine learning for black-box autoscaling. IEEE (2016). https://ieeexplore.ieee.org/document/7892598
References● Abhishek Dubey Nilabja Roy and Aniruddha Gokhale. 2011. Efficient Autoscaling in the Cloud using Predictive
Models for Workload Forecasting. IEEE (2011). https://ieeexplore.ieee.org/document/6008748● Maria Alejandra Rodriguez and Rajkumar Buyya. 2018. Containers Orchestration with Cost-Efficient Autoscaling
in Cloud Computing Environments. CoRR abs/1812.00300 (2018). arXiv:1812.00300 http://arxiv.org/abs/1812.00300
● Moumita Mitra Manna Sunirmal Khatua and Nandini Mukherjee. 2014. Prediction-based Instant Resource Provisioning for Cloud Applications. IEEE (2014). https://doi.org/10.1109/UCC.2014.92
● Dehai Zhao Faming Gong Weishan Zhang, Bo Li and Qinghua Lu. Workload Prediction for Cloud Cluster using A Recurrent Neural Network. IEEE (2016) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8281183&tag=1
● Hao Zhang, Bogdan Marius Tudor, Gang Chen, and Beng Chin Ooi. 2014. Efficient In-memory Data Management: An Analysis. Proc. VLDB Endow. 7, 10 (June 2014), 833–836. https://doi.org/10.14778/2732951.2732956