(5/2/20) There are probably as many different opinions on models as there are models themselves. None can forecast with total accuracy and if one could you wouldn’t know it until it is too late. Still we need some forecasting tools to guide policy and responses.
As evident throughout this blog, we have been developing a model based primarily on the most reliable data available, deaths. From death rate and cumulative deaths and other measurable variables, we can compute number of cases (prevalence) and new cases (incidence) as well as forecast these values as well as deaths into the future. A useful outcome of these tools is the ability to forecast dates when it is safe to resume some level of work and social activity.
However, how do we know if a model is performing well? That usually means running it on historical data. Even though, like trying to predict the stock market, past performance is no guarantee of future results. We are here in the middle of an epidemic and so we can gain some indication based on the most recent historical performance. We can also compare ourselves to other models. However, all models are different and give different results. Still we have been benchmarking to what is emerging as the gold standard model and is now clearly the most widely followed model by the press and government agencies. This is the model by the University of Washington’s (UW) Institute for Health Metrics and Evaluation (IHME). The details of this model have not yet been published and are not well known. We believe we have many similarities such as depending strongly on death statistics.
The IHME and our model also make updates to forecasts based on new data. This would seem to be a sensible approach, but many “expert” modelers and epidemiologists consider this strategy a shortcoming because the forecasts can bounce around. Well I began my model for the very reason that these so-called experts were pontificating things that seemed far fetched and their models seemed to be too complicated with many variables that are hard to measure or define. So, these models may be good for ultimately understanding pandemics better (after the fact), but we are in the middle of a pandemic and need some real-time guidance and forecasts so I believe the strategy that I, and apparently IHME, are taking are valuable. There is a need for many models as they all make different assumptions or are trying to measure or forecast different properties.
So, to cut to the chase the plots below show the forecasted total final deaths for the various hotbed countries and U.S. states that my model is following alongside the IHME model forecasts over the same time period.
The following are key observations:
- The two models would appear to be more similar than dissimilar based on the relative qualitative agreement on death forecasts, e.g., the order of severity for different countries and states.
- Both models fluctuate, but not excessively relative to an epidemiological model that is calculating based on first principles.
- Both models seem to be on a slight rise in forecasting deaths. We think that is because both models a priori assume that the rise and fall behavior are symmetric, but evidence is now showing the decline is slower. We know how to modify our model to do this, but will not be able to do so before the pandemic mostly plays out. Instead we are applying an asymmetry factor to the downside to compensate. We do not know if IHME is doing anything similar.
- Our latest forecasts are all above those of IHME, but not greatly making us think that the difference is due to our including the asymmetry factor for the first time in the last update.
- Not shown in these plots but evident in the Weekly Update postings is that we tend to read the death rate data such that we believe countries and states are not as far past the peak as IHME indicates. We think this is due to our using a Gaussian function to monitor peaking, whereas they use an error function (integral of a Gaussian) to do so, which we believe is less sensitive to detecting changes, e.g., peaking. We both apparently use the error function to forecast total deaths but from different parts of the curve depending on how we read displacement from the peak.
I just want to point out that what I’ve been saying for a couple weeks now – that we clearly aren’t in decline as the virus spreads through neighboring states even with NY in recission – has finally been picked up by university models and the media. Victory lap! I’m not sure why this was so obvious to me as an armchair observer yet nobody else seemed to be seeing this (or they were and lying about it, which is worse).
I haven’t seen your numbers, just your hunch that this will continue as it spreads. But this has been known for awhile and I have commented multiple times that we will see a peak and a slow decline as other areas become infected latter and will trigger additional pseudo-Gaussian growth and decay, but these will involve lesser and lesser populated areas. Of course we could get rebound outbreaks from easing social restrictions too early, but that is another matter. So the evidence in my forecasts are bearing out in the death rate statistics where NY is now about 2-3 weeks past its peak, NJ 1-2 weeks, as well as the other first line hotbeds 3+ weeks past (WA, MI, LA, etc.), but the U.S. is only 1-2 weeks past peak and declining slowly due to these newer, but smaller hotbeds, e.g., MA, CT, PA, IL, GA. Can’t post those plots here, but they are shown in my last weekly update posting and a new one will be out tomorrow. So my observations are things are progressing as forecasted with no major changes in the final death forecasts; adjustments have been within a factor of 2x magnitude (see above plots), which is pretty tight in this forecasting game. But the trend is toward the upper end of that 2x range.
BTW don’t pay any attention to any death forecasts coming out of the administration. They are posturing to try to make themselves look good in the end and they never reveal what their models are.
Further to this, forecasted numbers are indeed growing. This is due to the asymmetry of the rise and fall, the fall is much slower due partially to secondary outbreaks of other locales as Brent above as indicated. Another factor is that the reported deaths in the past were more under-stated than anticipated due to late nursing home reported figures, but also due to many deaths that were not COVID-19 confirmed, but are now being attributed to it. Both of these factors will add 50-100% to the forecasts of only a couple of weeks ago both internationally and domestically. New update coming out tomorrow 5/7/20.