Forecasting Systems
Forecasting systems consist of the following components: exploratory data analysis, data processing pipelines, time series models. and forecasting metrics wrapped into a coherent framework, acting together with the objective of producing automated and reliable forecasts for enhanced decision making across an organization.
Exploratory Data Analysis looks at things as simple as the sample size, but can go into more depth answering questions such as are there statistically significant lags that can affect the future forecast, or are there seasonal patterns that exist, and do these patterns have a high probability of persisting into the future.
Data Processing Pipelines can be seen as the life blood of the models. Without clean data, the models would be processing bad data. The adage “garbage in “ and “garbage out” would be an apt description for this particular circumstance.
Time Series Models can be seen as the engine that drives the predictions. The often used analogy of data is oil and the algorithm the engine, seems fitting. Models(Algorithms) in this stage, after exploration and research, can be calibrated to accurately generalize forecasts into the future. The word “generalize” here means, that the model should not “overfit” or “underfit” the data. This is a statistical phenomena, that needs careful consideration.
Another important issue arises, when you “over-forecast” and “under-forecast”. For example, if you own a retail store and want to meet demand, if you “under forecast”, you will not have enough inventory to match demand. This would lead to lost sales. If your models, “over-forecast” the data, this would lead to excess units in inventory, increasing storage costs. The “optimum” level of units you want in inventory, can be achieved by well calibrated forecasts.
Of course, every system should have metrics to monitor model performance. These metrics, show the temperature of the environment, as measured by model. When the metrics show underperformance, models may need to be recalibrated to fit the environment in which they are operating in. This could mean shelving old underperforming models, for newer models that might fit that particular economic environment. It also could mean that another round of research maybe necessary in order to research and test models, to add into future rotations.
