Why Hadoop is Changing Healthcare

While the healthcare industry grapples with interoperability, a new breed of predictive solutions is hitting the market. These programs analyze disparate data—data from multiple sources in multiple formats—to predict adverse health events. The structure of these technologies enables them to churn through massive amounts of information to build and tune models that ultimately predict individual illnesses and population health events. Using a similar approach to a Google or an Amazon search, these applications find meaningful and oftentimes hidden patterns within data to predict outcomes.

Predictive Analytics Healthcare, Hadoop, Clinical Decision Support, Big Data

Underlying many of these predictive technologies is Hadoop - an open-source framework that enables the processing of large and diversified data elements (SAS 2014). Let's breakdown this definition:

  • Open-source: At the core, open-source means that an application's source code is publically available and freely redistributed (Open Source Initiave 2014). Developers can use and contribute to the ongoing growth of the program. Hadoop uses its open-source architecture to speed development and the application's relevance across diverse use cases.
  • Processing: Hadoop leverages a distributed model to enable processing across multiple, parallel machines with the goal of speeding the time it takes to return results. These parallel machines are called "nodes." More nodes translate into more power.
  • Diversified: Hadoop can handle large volumes of data very quickly. This data doesn't have to be uniform; it can be unstructured. Examples of unstructured data include videos, social media feeds, and images. Hadoop doesn't require data to be pre-processed. It can consume a variety of data elements at the same time and house it for use.

Hadoop is important to healthcare because it, along with other complimentary open-source platforms, has advanced big data development within the industry (Raghupathi and Raghupathi 2014). Tools like Hadoop have made it easier for developers and commercial entities to mine healthcare data and develop models that can predict health events with high levels of accuracy and reliability. These predictions go far beyond Clinical Decision Support (CDS) frameworks and established statistical tools to account for millions of variables related to an individual patient, his/her community, and the general population. Raghupathi and Raghupathi in their February 2014 publication Big Data Analytics in Healthcare: Promise and Potential explain the process this way:

"Certain developments or outcomes may be predicted and/or estimated based on vast amounts of historical data, such as length of stay (LOS); patients who will choose elective surgery; patients who likely will not benefit from surgery; complications; patients at risk for medical complications; patients at risk for sepsis, MRSA, C. difficile, or other hospital-acquired illness; illness/disease progression; patients at risk for advancement in disease states; causal factors of illness/disease progression; and possible co-morbid conditions."

There are many other components outside of Hadoop that enable predictive technologies. However, Hadoop along with other supporting open-source applications including MapReduce and MongoDB, have driven development and innovation in a space that has historically lagged behind other industries in predictive analytic adoption. And the reality is that new value-based models of reimbursement along with penalties that reduce provider payments for adverse events such as hospital-acquired conditions are spurring the need for predictive applications. For providers, prevention over treatment is the mantra – but we can't prevent what we can't predict.

Works Cited
Open Source Initiave. The Open Source Definition (Annotated). 11 11, 2014. http://opensource.org/osd-annotated (accessed 11 11, 2013).
Raghupathi, Wullianallur, and Viju Raghupathi. "Big data analytics in healthcare: promise and potential." Health Information Science and Systems. 2 7, 2014. http://www.hissjournal.com/content/2/1/3 (accessed 11 12, 2014).
SAS. Hadoop What It Is and Why It Matters. 11 11, 2014. http://www.sas.com/en_us/insights/big-data/hadoop.html (accessed 11 11, 2014).

By Jvion Health