Fixing the Models that COVID Broke
Predictive models rely on the principle that the patterns and behaviours of the past will most likely repeat in the future. The more stable and repeatable the pattern, the better.
If every year the sale of hotdogs and burgers go up in August, then a model will likely advise a supermarket to stock extra hotdogs this summer. So, what happens when in 2020, to paraphrase Yeats; “All changed, changed utterly: A terrible beauty is born.”
The pandemic has completely changed the way we live and work. Lockdowns, travel bans, the complete shutdown of certain sectors of business, physical distancing, widespread furloughs … all of these things have changed how we shop, where we work from, how and if we travel, how we behave and all of that translates to changes in patterns in data, rendering many pre-COVID-19 models fragile, if not completely useless.
Anyone working with models won’t be surprised to find that shifts in the data feeding into models built before the crisis has sent some models off the deep end. Models are expected to degrade over time – no model lasts forever, and models regularly need re-building as shifts in the market, environment or customer behaviour happens.
With COVID-19, this has just happened more quickly and more radically than before. Before throwing out all models/analytics and wringing your hands in despair, we’ve put together a few tips below on how you can ‘fix’ the models COVID-19 broke.
- Evaluate, triage, plan, act: To determine what needs to be prioritised a business might look at how a model is used, whether a model is material in key tasks such as building financial reports, to what degree COVID-19 has had an impact (not all models will be affected equally) and how direct that impact is on critical business. Other considerations include regulatory focus or the potential of upstream models to “contaminate” other critical models. By performing model-triage you can focus limited analytics resources on the most critical models and focus attention on where it’s most needed and will give highest returns. As with any analytical work, and especially during these times, it’s vital the business is involved in these decisions as they are closely linked to potential shifts in business strategy.
- Identify if a shift/new trend is temporary and likely to die out post-COVID-19 (such as a surge in demand for flour as people took up baking during lockdown) or likely to be more long-lasting or ‘the new normal’ (such as customers move to online shopping). This will shape your business strategies as well as your approach to analytics.
- If a model is not vital to business activities during these times, and is performing poorly, consider switching it off/pausing until the situation stabilises.
- Override or switch-off auto-replenish if a model automatically orders stock and predictions are wildly off. Return to manual orders for the time being.
- There are various approaches you can take to improving model performance such as: try tuning a models’ parameters, re-train the model on more recent data or limit data to only post-COVID-19 time periods.
- Investigate using a different model – SPSS has an Expert Modeler function that allows the tool to choose the optimal time series model based on the data. Other tools such as IBM Modeller have a function to auto-select the best algorithm.
- Identify future demand drivers and include them in the models. This could mean including disease and lockdown information in the model – separate forecasts can be created, and the outputs added to existing models, or use past case numbers/lockdown events as features in the model.
- Revisit the time period for predictions: if a model has been forecasting for longer time periods ie on a quarterly basis, it might be useful to build another model that forecast on a weekly or daily basis. Short term forecasting can allow more rapid reaction to changing events.
- Re-evaluate the importance of your model inputs: retail has seen a massive shift to online shopping, if a model previously relied on footfall figures, including new/previously unused digital data sources such as mobile-app usage could be more useful
- Beware one-size-fits-all modelling: whereas previously a model may have been used to forecast at a group level ie sales for Europe, considerable differences now exist between countries based on varying government reactions/strategies, the impact of the disease on the population in each area etc – consider that a more regional or granular approach may now be needed.
- Promote agile iterative analytics: sometimes creating a quick solution, test small, fine-tune later can help a company react quickly especially in volatile times. A simple moving average of the sales of the past three days might be enough to allow a rapid response to a shock event that would otherwise not have been caught while waiting for a full re-build of a more complicated predictive model. Having a strong analytics department that is closely tied to the business will allow your company to keep track and respond in an uncertain world.
- Check assumptions made during model build/data engineering steps: we’ve seen unprecedented events this year such as the collapse of the airline, travel and entertainment sectors. If your model relies on certain assumptions such as a minimum number of flights per day, or that crude oil prices would never drop below zero dollars a barrel, and doesn’t allow for the alternatives, that can have tremendous implications for model predictions.
- Widen where you apply analytics: As companies moved to providing services to consumers online, and their workforce moved to remote-working, this has had a massive impact on many companies internal IT infrastructure, with many companies worrying about business continuity. Issues faced were companies realising their systems couldn’t cope with the increase in web traffic, their systems weren’t set up for remote-working or simply they didn’t have enough stock such as laptops for workers. Analytics can be employed to ensure business continuity. Various models such as time series, clustering models and anomaly detection models can be used to model network usage – allowing your IT department scale up systems (easily done if you are already using cloud based technology such as Azure), they can flag anomalous and potentially threatening activity that could result in downtime to your IT systems, even traditional t-tests/ANOVA can be used to analyse worker-satisfaction survey results to monitor how workers are coping during these times.
Being able to gather and illustrate data accurately and communicate in a way that everyone can understand and react to is down to the tool that is deployed, and this is where data analysis software such as SPSS can deliver real business benefits.
Version 1’s experienced consultants are on hand to help you understand your SPSS needs – from consultancy and training to finding the best software and license type for your analytical and usage requirements. Contact us to discuss your requirement and identify the best SPSS solution for you.
Take a look through our SPSS Articles covering a broad range of SPSS product and data analytics topics.
The Value of Codebook in IBM SPSS Statistics
Codebook is a valuable resource that presents a comprehensive overview of dictionary information about variables and multiple response sets within a dataset. It contains crucial data details, such as variable names, variable labels, variable types, value labels, missing values, and summary statistics. The summary statistics for nominal and ordinal variables and multiple response sets include counts and percentages, while the summary statistics for scale variables comprise mean, standard deviation, and quartiles.
What’s New in SPSS Statistics v29.0.1?
SPSS Statistics v29.0.1 was launched in April 2023 with the inclusion of the Overview Tab functionality, as part of the SPSS Base module. This post will take a high-level look at these changes plus cover off the key enhancements in v29.0.1.
What’s New in IBM SPSS Statistics v29?
Looking to quickly learn what’s new in IBM SPSS Statistics 29? Rather than reading a list of changes and new procedures, there is a quick way to tour what’s new and access help. Read our post for more info.
What’s New in SPSS Statistics v27?
SPSS Statistics v27 was launched in June 2020 with the biggest change compared to v26 being the inclusion of Data Preparation and Bootstrapping as standard functionality, as part of the SPSS Base module. This post will take a high-level look at these changes plus cover off the key enhancements in v27.
Fixing the Models that COVID Broke
Models are expected to degrade over time – no model lasts forever. With COVID-19, this has just happened more quickly and more radically than before. This post summarises a few tips on how you can ‘fix’ the models COVID-19 broke.
Internal Fraud Can Erode Organisations – Predictive Analytics to the Rescue!
If fraud occurs in an organization, salvaging the situation is paramount. How can this be done? The answer is Predictive Analytics! Read our blog post to find out more.
What’s new in IBM SPSS Statistics Version 26
IBM SPSS Statistics Version 26 includes new statistical tests, enhancements to existing statistics and scripting procedures, and new production facility capabilities. Read our post to find out more.
Infographic: Advanced Analytics for Healthcare
Healthcare systems face multiple challenges, including ageing populations, evolving healthcare needs and rising costs. Take a look at our Infographic on Advance Analytics for Healthcare for more information.
Lost in Translation? 8 tips so Multi-Lingual Text Analytics isn’t too scary!
Were you ever faced with identifying insights from open ended text data? From call centre notes, open ended questions on surveys, complaint letters, and/or online reviews? Read our blog to find out more.
Infographic: What is Forecasting?
What is Forecasting? Take a look at our Infographic on Data Audit: Advanced Analytics for more information.
How to Choose the Right SPSS License for You
Are you a super user fully emerged in analytics, a lone wolf carrying the load, or someone who dabbles in the odd analytics project? Whether you are part of a team or a single user, whatever your analytical requirements are, there is an SPSS license for you! Read to find out more.
How You Can Predict Device Failure
How can manufacturing companies accurately measure the lifetime of a device or even take into account the effect of external factors? Read our article on predictive maintenance to find out more.
How to Plan your Analytics Strategy
Do you have a business crying out for answers to critical questions but find it difficult to generate those answers? In this article we highlight the key issues to address when planning your Data Mining strategy.
The Importance of Strategic Alignment for Successful Analytics Initiatives
If you have ever found that your company has engaged with Analytics initiatives that have not generated the level of success that was envisaged – despite […]
5 Steps to Getting on the Right Track in MRO Inventory Management
Inventory Management is a concern for many organisations, large and small. This article will show you how organisations can take some light touch, manageable steps to reduce MRO inventory levels and costs while maintaining their maintenance SLAs and downtime KPIs.
Deployment in Advanced Analytics Projects
At Version 1, we’ve seen organisations perform best when they take the before, during and after of the deployment of analytics initiatives into consideration. This article will highlight these considerations for the best possible project outcome.
Analytics, Simplicity and Value – A complicated relationship?
Descriptive analytics is illustrated as the foundation and bedrock of analytical activities but limited in the business value potential. This is where the simplification is lacking, in our experience. This blog will share more on this topic.
Advanced Analytics and the quest for Return on Investment in 5 Key Points
Advanced Analytics as a core enterprise capability is there to serve the business to make better-informed decision. In this article, we present some of the challenges and good practices linked to measuring ROI for Advanced Analytics Programmes or Projects.
Creating an Analytics Function
This article takes a look at 4 analytics functions that are essential to consider, especially in organisations where analytics can be used for a variety of applications across multiple lines of business.
What is Analytics?
What is Analytics? Analytics is being discussed more and more often in top table conversations, but why do so many organisations still struggle to get to grips with it? Is it fundamentally down to complexity, or at least the perception of complexity? Read our post to find out more.
Infographic: Data Audit – Advanced Analytics
Forecasting predicts future values of a particular quantity based on previously observed values of that quantity. Take a look at our Infographic on What is Forecasting for more information.
Infographic: Technology Review – Advanced Analytics
Technology Review – Advanced Analytics Take a look at our Infographic on Technology Review: Advanced Analytics for more information.
Is Your Organisation Analytically Mature?
Is it too easy to think of analytical maturity as a step ladder? How can you reliably position yourself on that ladder? Are you too critical or too kind? Have you taken all the key aspects (perhaps even the unknown unknowns in the analytical world) into account? Read our blog to find out.
7 Tips for Transitioning Data Insights into Business as Usual
“Progress is impossible without change, and those who cannot change their minds cannot change anything” George Bernard Shaw The Irish playwright was on the money – […]
The Three Pillars to Actionable Insight
What are the secrets to success with analytics? The truth is that there are many factors and forces at work that help build a successful analytics project and/or programme. The ultimate goal is Actionable Insight. Read the blog post for more information.
“… The beauty of me is that I’m very rich.”
I have absolute belief in the richness of data and its ability to enable us to make evidence based decisions. When we ask ourselves When? What? Where? Why? or Who?, data can provide the answer.
Are you into extinguishing fires or are you into preventing fires?
As a specialised Advanced Analytics Solution company, the majority of the relationships that we build with our customers start with a crisis of some sort. This article takes a look at how we can support customers in addressing these challenges.
Why Analytics makes people nervous?
Why Analytics makes people nervous? The reasons behind, and some hints to help you during difficult conversations and challenging situations! Read our post to find out more.
The 5 Mistakes You Make when You Start with Analytics
When you start with Analytics you might feel overwhelmed, you have all that incredible power in your hands but don’t know how to wield it most effectively. So just how do you go about getting the most from this valuable asset? You can start by avoiding these five common mistakes.
Enhance Customer Experience in Insurance with Analytics
Reports from the insurance industry consistently highlight that the quality of customer experience remains the biggest factor driving customers to remain loyal or switch to another insurance provider. Hence, the focus should be on how to improve the quality of the customer experience rather than reducing fraud. Read our post to find out more.