Automated, camera-based neurological deficit identification

Introduction

Facial weakness, limb weakness, limb ataxia, and abnormal eye movements are common signs of nervous system damage caused by stroke, traumatic brain injury, and neurodegenerative diseases [1]. Our project aims to develop an automated, quantitative, digital screening tool for neurological disease that would (1) quickly identify neurological deficits thereby expediting the diagnostics and treatment process, and; (2) improve outcomes by minimizing inter-operator variability and operator error resulting in more accurate diagnosis and triage of neurological patients. Once developed, our system will provide real-time, clinically relevant, triaging information for cases when a neurological event happens and access to neurological expertise is limited.

The system will evaluate for signs of neurological disease from a video of a standard patient examination. We aim to deploy the system on commonly available camera-enabled devices (e.g. laptop computers, tablets, phones, etc.), without the need for additional specialized hardware, and without the need to transfer image/video data from the device to servers (all processing to be performed on the device). An overview is shown below.

Applications

Stroke . In the United States, stroke is a leading cause of death and disability with more than 795,000 people suffering a stroke every year[2]. The mainstay treatment of acute ischemic stroke (AIS) is rapid reperfusion therapy with intravenous tissue plasminogen activator (tPA) and/or endovascular thrombectomy (EVT), which substantially decrease the odds of death or disability[3]. Numerous studies have shown that reduction in time to treatment by as little as fifteen minutes significantly decreases mortality and disability in stroke patients[4-6]. Recognition of stroke in the prehospital setting by emergency medical services (EMS) leads to earlier and more frequent treatment with acute stroke therapy[7-10]. However, the identification of stroke by non-neurologists can be challenging, as illustrated by the finding that paramedics fail to detect stroke in high as 56% of patients when not using a diagnostic aid [11].

Traumatic brain injury . Traumatic Brain Injury (TBI) includes concussion, subconcussion, and most exposures to explosive blasts from improvised explosive devices [12]. One study indicates that the prevalence of TBI among returning service members range from 15.2% to 22.8%, affecting as many as 320,000 troops [13]. The delay in diagnosis increases the risk of developing Alzheimer's disease, Parkinson's disease, motor neuron disease, and stroke [12,14,15]. For instance, a recent study demonstrated that TBI was associated with 10.21, 4.61, and 2.32 times greater risk of stroke at three months, one year and five years after the injury, respectively [16].

Engineering

To differentiate normal examinations from abnormal ones, we utilize an ERT(Ensemble of Regression Trees) model [17] for extracting facial location from movie frames and an PAF (Part Affine Fileds) model [18] for extracting pose. Once appropriately localized, features (e.g. optical flow, local binary patterns, etc.) can be extracted and machine learning can be utilized to regress the correct label (diagnosis) of a given video examination.

Data acqusisition

At the University of Virginia we’re acquiring a "neurological deficit" data set comprised of 200 videos for each of the following neurological deficits: facial weakness, limb weakness, limb ataxia, nystagmus, and gaze deviation. An additional 200 videos of non-stroke controls will be recruited from healthy volunteers. This video database will be sorted into derivation and validation sets to develop and test the individual deficit detecting algorithms.

Preliminary results

Our current prototype for the proposed tool for facial deficit detection, called F-DIT-V, achieves a classification accuracy of 92.9%, precision of 93.6%, recall of 92.8%, and specificity of 94.2%, which is higher and more reliable compared with the LBPTOP[19] and the LSTM[20], which are widely used in previous and current studies for activity recognition.

The prototype has been implemented as a live, real time tool, able to run on modest hardware such as commonplace laptops. Demonstration of the BANDIT (brain attack and deficit identification tool) on a volunteer mimicking a normal examination, as well as left and right deficits are shown below.

--- prototype ---

References

[1] “Traumatic brain injury and Concussion” https://www.cdc.gov/traumaticbraininjury/symptoms.html
[2] “Stroke facts.” https://www.cdc.gov/stroke/facts.htm
[3] William, Jpowers, et al. "2015 American Heart Association/American Stroke Association focused update of the 2013 guidelines for the early management of patients with acute ischemic stroke regarding endovascular treatment A guideline for healthcare professionals from the American H." International Journal of Cerebrovascular Diseases 23.11 (2015): 801-814.
[4] Saver, Jeffrey L., et al. "Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke." Jama309.23 (2013): 2480-2488.
[5] Sheth, Sunil A., et al. "Time to endovascular reperfusion and degree of disability in acute stroke." Annals of neurology 78.4 (2015): 584-593.
[6]Liebeskind, David S., et al. "Early arrival at the emergency department is associated with better collaterals, smaller established infarcts and better clinical outcomes with endovascular stroke therapy: SWIFT study." Journal of neurointerventional surgery 8.6 (2016): 553-558.
[7] Abdullah, Abdul R., et al. "Advance Hospital Notification by EMS in Acute Stroke Is Associated with Shorter Door-to-Computed Tomography Time andIncreased Likelihood of Administration of Tissue-Plasminogen Activator." Prehospital emergency care 12.4 (2008): 426-431.
[8] McKinney, James S., et al. "Hospital prenotification of stroke patients by emergency medical services improves stroke time targets." Journal of Stroke and Cerebrovascular Diseases 22.2 (2013): 113-118.
[9] Mosley, Ian, et al. "The impact of ambulance practice on acute stroke care." Stroke 38.10 (2007): 2765-2770.
[10] Gladstone, David J., et al. "A citywide prehospital protocol increases access to stroke thrombolysis in Toronto." Stroke 40.12 (2009): 3841-3844.
[11] Smith, Wade S., Marshal Isaacs, and Megan D. Corry. "Accuracy of paramedic identification of stroke and transient ischemic attack in the field." Prehospital Emergency Care 2.3 (1998): 170-175.
[12] McKee, Ann C., and Meghan E. Robinson. "Military-related traumatic brain injury and neurodegeneration." Alzheimer's Dementia 10.3 (2014): S242-S253. [13] Wojcik, Barbara E., et al. "Traumatic brain injury hospitalizations of US army soldiers deployed to Afghanistan and Iraq." American journal of preventive medicine 38.1 (2010): S108-S116.
[14] Lee, Yi-Kung, et al. "Increased risk of ischemic stroke in patients with mild traumatic brain injury: a nationwide cohort study." Scandinavian journal of trauma, resuscitation and emergency medicine22.1 (2014): 66.
[15] Liu, Shih-Wei, et al. "Increased risk of stroke in patients of concussion: a nationwide cohort study." International journal of environmental research and public health 14.3 (2017): 230.
[16] Chen, Yi-Hua, Jiunn-Horng Kang, and Herng-Ching Lin. "Patients with traumatic brain injury: population-based study suggests increased risk of stroke." Stroke 42.10 (2011): 2733-2739.
[17] King, Davis E. "Dlib-ml: A machine learning toolkit." Journal of Machine Learning Research10.Jul (2009): 1755-1758.
[18] Cao, Zhe, et al. "Realtime multi-person 2d pose estimation using part affinity fields." arXiv preprint arXiv:1611.08050 (2016).
[19] He, Shu, et al. "Quantitative analysis of facial paralysis using local binary patterns in biomedical videos." IEEE Transactions on Biomedical Engineering 56.7 (2009): 1864-1870.
[20] Yue-Hei Ng, Joe, et al. "Beyond short snippets: Deep networks for video classification." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.