Flu’s spread is unpredictable. Can AI yield more reliable forecasts?

Flu season reliably arrives around this time every year — but where the virus heads and how it will spread can seem wildly unpredictable. Now, artificial intelligence is playing a bigger role in trying to change that.

One startup is using data collected from thermometers to develop algorithms to derive insights about flu activity that it could sell to retailers, consumer-goods makers, and perhaps even health systems. Academic researchers are refining sophisticated AI models, including using machine learning and statistical methods to recognize patterns and map out future trajectories.

There’s tremendous value in being able to accurately predict the flu’s path, and in particular, when the season will peak. Researchers and companies argue that better flu forecasting could help public health officials more accurately time their messaging or mobilize resources when a dangerous strain is likely to hit a given region hard. Hospitals could plan ahead, making sure they’re staffed up and have beds available as the worst stretch of flu season in their area approaches. Retailers and manufacturers could plan for high demand for cold and flu medications. People who are the most vulnerable to flu — including the elderly, people with asthma, and young children — could be warned when they need to take extra precautions, like avoiding public spaces.

advertisement

Despite AI’s promise, there are plenty of obstacles that could prevent it from improving flu forecasting: The underlying data used to build these algorithms can be messy, low-quality, and unrepresentative of who’s actually sick with the flu. It’s not clear whether there’s money to be made in developing these algorithms. Nor is it clear whether public health officials will embrace them to inform their decision-making.

Still, interest in high-tech flu forecasting is growing. One sign of the boom: In 2013, when the Centers for Disease Control and Prevention first invited researchers to use their data to forecast the flu season weekly, researchers submitted 11 different models. Last flu season, there were 38.

“It’s garnering more interest, and that’s really what you want. You want lots of ideas so that the best practices can emerge,” said Roni Rosenfeld, who leads the machine learning department and flu forecasting efforts at Carnegie Mellon University in Pittsburgh.

That influx of ideas has been a boon for modern flu forecasting. There’s plenty of room for improvement. The CDC’s traditional flu surveillance systems measure flu activity only after it has already happened. There’s also a weeklong lag between when the cases are reported to the CDC and when that data are reported publicly.

But the spread of the flu is notoriously wily, and has proven difficult to predict, even with high-tech tools. The most notable example: the failure of Google (GOOGL) Flu Trends. A decade ago, researchers at Google launched an effort to “nowcast” the flu by combining big datasets on Google searches for flu-related information with flu tracking data from the CDC. For that project, they used a type of statistical modeling that was a precursor to machine learning and other AI techniques that are widely used today.

Flu proved too tricky to forecast with that approach. Flu viruses aren’t the only ones that circulate during winter and that can cause colds and “flu-like symptoms.” Online searches about flu-like symptoms might or might not be about influenza itself — which means search data generate a lot of noise. Google’s forecast ended up dramatically overestimating flu cases. The initiative was shuttered, becoming a cautionary tale about the limitations of big data.

Researchers now believe there is a better way to accurately predict the flu’s path — and that might happen with the help of AI. Rosenfeld’s research group — which has frequently topped the CDC’s list of most accurate flu forecasting — has developed two primary systems to predict the spread of flu. One relies on machine learning and computational statistics to make predictions using past flu season patterns and data from the CDC’s surveillance system. That can be challenging, given flu’s inherent unpredictability.

“There’s an amount of uncertainty and there is a good amount of regularity,” Rosenfeld said. “The real heart of forecasting is being able tell to the two apart,” he said.

But machine learning systems aren’t always good at dealing with curveballs. Take the flu season two years ago, which appeared to hit a peak around December 2017. Historically, there’s been a dip in flu activity in January after that high. Everyone expected that downturn — but it didn’t come.

“It surprised the flu professionals, it surprised the CDC, it surprised the models. It surprised the humans too, but they adjusted to it. They adjusted faster,” Rosenfeld said.

The other system developed by Rosenfeld and his colleagues is by and large, run by humans — and humans are much better than machines at dealing with surprises, Rosenfeld said.

That model is based on a wider phenomenon known as the “wisdom of the crowd”: Rosenfeld asks volunteers to look at historical trajectories of flu for a particular location, scan data for the current flu season, and use their finger or mouse to draw a line. All together, the projections from the “crowd” are fairly accurate.

The hitch: Each person has to predict the trajectory for flu in every state or region in the U.S. — every single week. The crowd gets bored.

“They will do it happily for the first time and the second time and the third time, but after that, they tend to lose their motivation,” Rosenfeld said.

Another key dilemma for academics is demonstrating the value of their approaches to public health officials and decision-makers.

“The challenge is to see how they can incorporate what we’re sharing with them every week to change the way they perform their activities. In other words, we can do a lot of math, a lot of technical progress — but if it doesn’t become useful to change behavior in any meaningful way in our societies, then the efforts lose their value or their relevance,” said Mauricio Santillana, a leading flu forecasting researcher at Boston Children’s Hospital and Harvard University.

Santillana led a team that developed ARGONet, which uses both machine learning and two flu detection models. The models pull from historical flu data, electronic medical records, Google searches, and patterns of flu spread in neighboring areas. To teach the machine to accurately predict flu’s spread, the researchers fed it real flu data and predictions from the two models.

Santillana said that he and his team are working with officials at the CDC to help inform future directions for their research; he’s been in touch, too, with public health officials in France and Africa eager to learn more about his work and help guide how it can be most helpful.

How good is the underlying data?

It’s a truism in AI research that a model can only be as good as the underlying data used to build it. And for flu forecasters, sourcing the right data is an ongoing challenge.

Experts are quick to point out limitations in the datasets often used in the field. There are data from social media postings and search engine queries, which can be skewed by so-called “panic searches” — healthy people who go online because they’re worried about getting sick. There are data from insurance claims and electronic medical records, which only reflect the subset of sick people who actually went to seek medical care. There are also data on sales of cough and cold medicines, as well as weather patterns; this information offers some signal but is a few steps removed from what’s really useful to flu forecasters: who’s sick now and who’s going to get sick next.

Inder Singh would know. He and his team at Kinsa, the San Francisco startup that he leads as CEO, initially tried mining existing datasets to try to find patterns that could help them forecast the flu.

“Even if you take all of those data points together, when we analyzed it, you simply could not get to what you wanted — you want timeliness, you want accuracy, and you want geographic precision,” Singh said. “We felt — and I think many people still feel — that the amount of signal that you can extract from those existing datasets were at a limit.”

Instead, Singh and his team decided to collect their own data. Kinsa sells smart thermometers — used in homes, schools, and workplaces — that act as an entry point for Kinsa’s real product: an app that acts as a command center for patients to communicate with the company about their illness. The thermometers beam in temperature readings and also sync up to Kinsa’s software; when a user has a fever, an AI chatbot asks them about other symptoms and gives them guidance about how to respond and seek care.

The system gives Kinsa granular, real-time information about where people are experiencing flu-like symptoms. It’s hard to tell how representative Kinsa’s users are of the general population of sick people, but the company has lots of data: More than 2 million people in the U.S. have used the company’s systems since they were launched in 2013, representing tens of millions of illness episodes, according to Singh.

Kinsa is using that data to build predictive AI models; it’s in the early stages of trying to refine those algorithms by also adding in weather data and CDC’s historical data, which are derived from clinical data.

The company’s next step: Building out a business selling useful insights that it’s generated from those predictive models. Singh is careful to stress that the company isn’t selling its data to third parties. Rather, third parties pay Kinsa to tell them, via an emailed alert or in a dashboard with visualizations, whether the influenza-like illness level in a given county is high or low, whether it’s severe, whether it’s long-lasting, whether it’s growing quickly — and what’s likely to happen in two weeks.

Kinsa has a partnership with a large retailer to forecast upcoming influenza-like illness in a given area and likely demand for certain products, whether that’s cough and cold medicines or disinfectants. And the startup is eager to ink more such deals. Singh said he sees the most promise in marketing Kinsa’s insights to retailers and the makers of consumer goods. He’d like to sell to health systems, too, though he acknowledged that the notoriously long sales cycles involved in striking a deal with a hospital is likely to make that harder.