A reflection on my experience in the Division of Cardiac Critical Care at SickKids
27OCT2017 (20 minute reading time)
From October 16th through 27th 2017, I had the immense privilege of completing an observership within the Cardiac Critical Care Unit (CCCU) at the Hospital for Sick Children (SickKids) in Toronto, Canada. I must admit that the CCCU was overwhelming in many aspects. Overwhelmingly complex, yet overwhelmingly specialized. Overwhelmingly busy, yet overwhelmingly efficient. Just imagine placing a four-year old into the cockpit of a Boeing 787 Dreamliner and telling him to, “Go for it. Take off, and deliver us to our destination safely.” I’m the four-year-old. The CCCU is the Boeing. Need I say more?
Established in 2006, the Division was the first of its kind in Canada. It has about a half dozen staff physicians, dedicated nursing and interprofessional staff, about a dozen fellows, and a 23 bed critical care unit. While rounding with the fellows and staff physicians, I saw and was taught about children with the most complex medical pathology in the country and witnessed the administration of commensurately advanced and ground-breaking treatments; however, although this would be an incredibly fascinating topic, it will not be the topic of my reflection here. Rather, I would like to reflect on the primary objective of my visit to SickKids: to interpose myself at the interface where critical care medicine meets data science and, thereby, learn about the application of data science in medicine. While I am a far cry from an expert in either field, I will attempt to share a few thoughts I have had over these two weeks including why big data in medicine interests me, why it should interest the medical community, and my perspective of the data science program at the SickKids CCCU.
My professional background is in mechanical engineering, and I’m currently completing my fourth year of medical school. Risking over-generalization, I will admit that my experiences in the classroom and clinic during medical school have taught me that the practice of medicine is more of an art than a science. As a trainee, I have not found peace with this notion but derive comfort in that, given time and practice, I will become a better artist. For the time being, I prefer the scientific approach to clinical problems, an approach inculcated during my training and practice as an engineer, an approach that is broad and multifaceted, team-oriented, step-wise, calculated, and endears a clear understanding of the background, problem definition, specifications, and real-world constraints. The scientific approach may not be time efficient but is incumbent upon trainees because they lack the knowledge and experience to approach medicine otherwise. I was introduced to helpful ideas related to this dilemma early in my medical training by two faculty members in particular.
First, Dr. David Haase, an infectious disease physician, taught me about the maturational pathway of trainees, which could be applied to any field of practice: we all begin as unconsciously incompetent and, with training, progress to become consciously incompetent, then unconsciously competent, and, finally, consciously competent. A trainee in the incompetent stages attempting to practice medicine as an art would be nothing short of guess work; trainees must take the scientific approach. As competence is gained over years of study and practice, the trainee matures, and the transition from the scientific to artistic approach becomes possible while maintaining a good outcome.
Second, under the tutelage of Dr. Pat Croskerry, a global pundit in critical thinking in medicine and the Director of the Critical Thinking Program at Dalhousie Medical school, I have come to appreciate the subject of metacognition, that is, thinking about how we think – remember, I am an engineer with no formal psychology background, so Dr. Croskerry’s lectures seemed quite novel to me. The two most important physician behaviors are knowledge and decision making, yet our decision making ability in medicine is unacceptably poor relative to other high-stakes fields, e.g., aerospace engineering. The diagnostic failure rate in medicine is approximately 15% and, accordingly, diagnostic error is by far the leading cause of medical harm resulting in legal action. Recognizing that at a fundamental level, how we think determines how we make decisions, Dr. Croskerry taught us about dual process theory, which I found quite helpful. Basically, when we think and make decisions, we operate between two systems: the intuitive (system 1) and the rational (system 2). System 1 is fast, informal, subjective, context-dependent, qualitative, and flexible. System 2 is slow, formal, objective, context-dependent, quantitative, and rigorous. As humans, we spend more than 95% of our time in system 1, which is where most heuristics, biases, and errors occur. The human mind is a cognitive miser, which refers to our tendency to take the path of least resistance in our approach to problem solving. The remarkable efficiency of system 1 enables this and saves many lives; however, it is a double-edged sword because system 1 plays a major role in medical error, which is the cause of a significant proportion of deaths, for example, 9.5% and 3rd highest cause of all deaths in the US according to a study from Johns Hopkins published in the BMJ in 2016. Trainees begin by functioning mainly in system 2 (rational) and eventually progress to system 1 (intuitive) as functions are repeated time and time again and pattern recognition develops. The systems can override each other, and we can toggle between systems in our decision making process, e.g., critically reviewing the history, physical, lab, and imaging data to “double check” my gut feeling that the diagnosis is pneumonia (system 2 overriding system 1), versus following my gut feeling and choosing to empirically treat this patient for pneumonia now because the context is so dire such that I cannot wait any longer to make a fully informed decision (system 1 overriding system 2). In either system, decision making can be compromised by fatigue, sleep deprivation, and cognitive overload.
I liken system 1 thinking to the artistic approach and system 2 thinking to the scientific approach. Ideally, all our decisions would be made in system 2 and, being the best decisions, would yield the best results; however, there are at least two problems with this. First, system 2 thinking is time consuming, which makes it impossible for the human brain, operating solely in system 2 and under extreme time constraints, to make the best decisions rapidly. Second, system 2 thinking is cognitively demanding, and this demand is amplified to exponential proportions as more variables are added to the problem (Hick’s Law postulates that time on a task is positively correlated with the number and complexity of choices), which makes it impossible for the brain to arrive at the best decision every time with an error rate of zero. Also, like the relationship between distance, velocity, and time, the time and cognitive requirements of system 2 thinking on the brain are not independent: increasingly complex problems (distance) demand increased cognitive effort, which, due to a limited cognitive processing speed so intrinsic to being human (velocity), forces a slower arrival at a decision (time). By extension, we can appreciate that system 2 thinking is less suited for high-acuity, time-sensitive, data-intense situations commonly encountered in critical care or emergency medicine, for example. Fortunately, system 1 thinking, with its best-selling feature of pattern recognition, allows us to overcome these two major supply-demand problems (time, cognition) and performs quite well most of the time.
So why the discussion about how we think and make decisions? Let me bring this back to why data science in medicine has piqued my interest. Given that we want to always make the best decisions all the time, let us assume that we can only think in system 2 – our decision making process would be formal and objective, but despite always making the best decisions, the time taken to reach our decision could be hours, days, months, or even years depending on the cognitive demand of the clinical problem. What if we could compress this computational time so that it was less than a second? Not only would it be fast, it would be faster than system 1 thinking. This is precisely where data science comes in, and this fascinates me.
With data science techniques, we can collect, process, and store huge amounts of physiologic data from our patients, including “real-time” variables and wave forms (e.g., heart rate and variability, respiratory rate, central venous pressure, systolic and diastolic blood pressure), lab values (e.g., hemoglobin, arterial blood gases, lactate, SvO2), and treatment parameters (e.g., ventilator settings, medication infusion rates). We can then use data science to develop patient risk analytics engines on this dataset, and these engines can then be used to collect, process, and analyze data simultaneously and enable healthcare providers to make the best decisions at the bedside. One such example is the inadequate oxygen delivery (IDO2) index by Etiometry, Inc. (Boston, USA), which received U.S. Food & Drug (FDA) 510(k) clearance in 2016. Using a software model of human physiology, the IDO2 integrates 18 measurements of 9 physiologic variables and Bayesian inference to continuously adjust the risk of inadequate oxygen delivery based on current and previously acquired data. The IDO2 index enables healthcare providers to rapidly, effortlessly, and accurately assess the likelihood that the patient is experiencing inadequate oxygen delivery defined as a mixed venous oxygen saturation (SvO2) less than 40% – intervention may then immediately ensue to prevent morbidity and mortality. By the way, critical care units are “oxygen-delivery-centric” – everything evolves around oxygen delivery. The IDO2 index was made possible by applying data science techniques to a huge data set – nearly 2,300 patients and over 10,000 measurements of venous blood gases – to develop the original analytical engine. The engine is now packaged with Etiometry’s FDA-approved software, T3 Data Aggregation & Visualization, which allows healthcare providers to collect, store, and visualize ICU data in essentially real-time at the bedside.
To think through such computations in system 2 thinking would take hours, days, months, or even years, so I have likened data science to a time warp. But it’s not like any time warp. This time warp is bipolar. By bipolar, I mean that it can be viewed as both a time dilation (slowing down of time) and a time contraction (speeding up of time) at the same time. As a time dilation, under the pressure of needing to make the best decision within a matter of seconds, big data allows us to consider the whole picture as if everything were in slow motion and we were thinking in system 2. As a time contraction, big data harnesses super-computing power to transform an analysis that would take us years to compute in system 2 into a matter of milliseconds, allowing us to make the best and maximally informed decision, now. Now, that is cool, but data science has much more to offer. Thus far, I have only discussed decision making as a response – a reactive process. What if, in medical decision making in the acute setting, we were able to transition from being passive and reactive (the traditional approach in the acute setting), to being predictive and active? What if, instead of letting things happen to us, we happened to things? For example, what if we were able to predict that our patient would have a cardiac arrest within minutes and, instead of the arrest happening to the patient, we happen to act within seconds to change the clinical course of the patient and prevent the arrest? Spoiler alert: we already can, kind of. Kennedy et al., (Houston, USA) are using time series trend analysis enabled by support vector machine algorithms to encode physiologic deterioration – a time dependent process – in PICU patients and thereby predict and prevent cardiac arrest (Pediatr Crit Care Med 2015;16:e332-e339). The data team here in the CCCU at SickKids is also using data science to build models that predict cardiac arrest. Now, that is super cool, and this sort of potential inherent in data science is expressly why the medical community, at large, should “listen up”.
It caught my ear in 2015 when Dr. Peter Laussen, Chief of the Department of Critical Care Medicine at SickKids, gave a grand rounds talk on data science in the pediatric CCCU at the IWK Health Centre in Halifax. After his talk, I stayed to chat with him about a similar vision that I had in the context of my global health work, and he graciously invited me to complete an elective rotation in data science at the SickKids CCCU. Now, I’ll share a bit about what I learned about the data science program within the CCCU.
I learned that thus far, data science is a very small component of the CCCU but, despite its small size, it is quite complex, increasing in magnitude, and gaining clinical relevance with time. Over a few years, the data science team has grown from one person with an idea to a multidisciplinary team of at least seven individuals from varied clinical, professional, and research backgrounds. The program is supported by only one full-time software engineer and three part-time data scientists, which was surprising to me.
How does the data flow from being a signal recorded by a device attached to or embedded in the patient to a data point in storage? From bedside to byte stored? In brief, the physiologic data collection devices (e.g., arterial lines, CVP line, BP cuffs, SpO2 probes, near infrared spectroscopy, respiratory flow transducers, thermometers) and the ventilator settings (e.g., PEEP, vent rate, Ppeak, TV, etCO2) are connected to a bridge that interfaces with a Philips IntelliVue bedside monitor. To date, aggregate infusion pump data inaccessible for proprietary reasons imposed by the vendor. The data displayed on the IntelliVue monitor is sent to the Philips central server via Ethernet, which then sends data to three places: a central monitor at the nursing station for monitoring in real-time by nursing staff not at the bedside, the electronic medical record (EMR) for charting of patient vital signs at regular intervals, and to the T3 (tracking, trajectory, and trigger) system allowing visualization of the patient’s current status and past course and the IDO2 by any healthcare provider at any computer with SickKids intranet connectivity and a secure hospital login. The data team also siphons all the data from the IntelliVue monitor via serial connection to a middleware medical device integration service called ViNESTM by True Process, which acts as a device gateway, streamlining the high frequency (500Hz) waveform, derived value, and device parameter data for intelligent and queryable storage before it is sent to the CCCU database. An additional source of data feeding into the CCCU database is the EMR, which ultimately links time-stamped laboratory values (e.g., arterial blood gases, lactate, hemoglobin) with the physiologic and device settings data. The EMR also feeds into T3 in the same manner, allowing for continuous visualization of trends and current time-synced physiologic monitor data, lab data, and device settings anywhere the user logs in. Very impressive.
I learned of a few barriers that the data science team has encountered in their journey thus far. From a human factors engineering standpoint, I’m told that something seemingly as easy as convincing CCCU staff (nurses and respiratory therapists) to connect the ventilator to the Philips bridge via an Ethernet cable when ventilation was initiated was much easier said than done. Eventually, the team achieved buy in, and this critical step is now routine practice in the CCCU. Lin et al., (BMC Medical Informatics and Decision Making (2017) 17:122) recently published a human factors study evaluating T3 in the SickKids CCCU to identify interface usability issues, to measure ease of use, and to describe interface features that may enable or hinder clinical tasks. The finding that I found most interesting by Lin et al., was clinicians’ mistrust in the IDO2 due to “lack of transparency and published evidence” – only one of seven clinicians in the study were familiar with what the IDO2 even was, which indicates that perhaps ignorance and lack of knowledge was more cause for distrust than the cited reasons. From the hospital systems management, ethical, and medicolegal perspectives, another barrier the team has faced includes the issue of prospectively collecting identifiable patient data, without consent from the patient or his/her substitute decision maker, for various clinical, research, quality assurance, and quality improvement purposes. What are the managerial, ethical, and legal implications of a data program and how can these be adequately addressed? Who knows at this point, but what I do know is that it is imperative for a budding data team to have an advocate with influence at the level of hospital management to help overcome these barriers.
I am not privy to the expenses associated with initiating and supporting the data science program, but I do know that the program runs on a very small budget, yet another barrier. One would think that a program with such unprecedented potential at a leading pediatric hospital would be well funded; however, remember that we are at the dawn of the interface between medicine and data science. Data science and medicine are two soon-to-be lovers that have only begun to embrace – the potential of what their relationship could produce remains yet unrealized. Like an overeager but well-meaning parent, the data science team has accompanied these two on their first date here in the CCCU. Medicine is responsible for paying the bill but is broke tonight for obvious reasons. The parents will have to float the cost for now, but it’s an expensive evening out and funds are tight at home.
Enough analogies. Although budget limitations can slow progress, it can make things more efficient, which is exactly what is happening at SickKids. I happened to be at the CCCU during a special time for the data science team. On October 19th, 2017, the team collected, processed, and stored their trillionth data point. A trillion is a million millions! 1,000,000,000,000! How do we store this much data? How many bytes would it occupy? How much would it cost us to store this much data? I don’t know the answers to these questions, but what I do know is that the CCCU data is securely stored at the Centre for Computational Medicine (CCM) shared by SickKids and the University Health Network in Toronto. The CCM boasts a 268 computer node cluster network with 34 terabytes of RAM and is capable of providing performance of 80 Gb/s steady throughput from the computer nodes to a 2.4 petabyte (1 petabyte = 1,000,000 gigabytes) storage cluster. Woah! The SickKids CCCU data team has access to the CCM, which allows for “distributive computing” – the computer-world parallel to “teamwork”, fitly expressed by the old adage, “many hands make light work”. Distributive computing is necessary given that data science operations on such colossal datasets require an enormous volume of computational power. Regarding the trillion data points – in its raw format, it would occupy a huge amount of memory (initial size: in the ballpark of 50 terabytes!), which would be prohibitively expensive to securely store using commercially available data servers; however, using a technique called “lossless compression”, the data science team has invented a novel way to compress the data they collect at a 100 to 1 ratio for space-efficient, cost-efficient, and readily accessible storage (final size: a mere 400 gigabytes!). With lossless compression techniques, no information is lost during the compression and, therefore, all the original information is exactly the same upon decompression. Incredible!
A few more points about the data science program:
Collecting data for machine learning purposes is distinct from collecting it for research or QA/QI purposes. With machine learning in mind, we must be much more diligent to minimize errors and ensure data continuity. Data quality and fidelity are not a given, so ensuring data quality is a job in itself. Also, for predictive analytics, you need to link what you are measuring to what you are trying to predict, which requires a secondary data collection stream of outcomes, which introduces yet another source of error: human error, e.g., digit preference, transcription error, labelling error. Interestingly, humans are not the only ones guilty of digit preference – I learned that the measurement devices we use have digit preference too.
We also need to be wary of systematic error introduced by a measurement device itself as this, by compromising data integrity and accuracy, could undermine every decision we make based on the data, whether clinical or research. One example is the arterial line blood pressure measurement, which is a second-order dynamic system possessing a natural frequency and damping coefficient influenced by multiple factors including the catheter, extension tubing, stopcocks, flush devices, transducer, amplifier, and recorder. Arterial line blood pressure measurement was first reported in the literature in 1949, and as the medical field adopted the technology over the subsequent decades, it was incumbent upon physicians to understand the physics underlying the technology and morphology of the blood pressure waveform it produced. However, in more recent years, having received little-to-no teaching on the underlying physics of waveform generation, students and staff healthcare providers pay little-to-no attention to the quality of the waveform and focus solely on the quantity (numbers) displayed next to the waveform, i.e., the systolic BP, diastolic BP, and mean arterial pressure (MAP). Relying on these values and ignoring the waveform morphology can lead to significant medical error. A “whip” waveform morphology indicates the system is under-damped and is overestimating the systolic BP and underestimating the diastolic BP. Conversely, when a clot forms in the catheter tip or air bubbles enter the extension tubing, the result is an over-damped system and can be recognized on the waveform by the loss of the dicrotic notch. My point is that although data science may perform system 2 thinking for us some day, this does not mean we can shut off our brains. Quite the contrary. We need to be more vigilant in understanding the tools we use. Data science is a powerful tool with unprecedented complexity, and, because inherent to its use are benefits and risks like any other medical treatment, it is critically important for healthcare providers that wield it to do so responsibly. This includes a good understanding of where, how, and how much error may be introduced and how this might affect outputs and thus our decisions – not an easy task in the face of an ever-widening chasm between two ever-specializing fields.
Finally, I learned that the data science program is truly a team effort at its core. I was impressed by the diverse, but complimentary, range of professional backgrounds that constitute the data science team. Drawing from various fields unrelated to healthcare, the team has made the data science program what it is today and has poised it well to have tremendous impact in the near future. On reflection, this diversity makes sense to me and is a necessity for the excursion the team is taking into uncharted territory. Mark my words, the SickKids CCCU data team is a team to watch.
In closing, I would like to extend a heartfelt thanks to Dr. Laussen, the data science team, and the Division of Critical Care for their warm hospitality over the past two weeks.
B.Sc. Engineering (Mechanical/Biomedical)
Med4, Dalhousie Medical School