Facial weakness, limb weakness, limb ataxia, and abnormal eye movements are common signs of nervous system damage caused by stroke, traumatic brain injury, and neurodegenerative diseases [1]. Our project aims to develop an automated, quantitative, digital screening tool for neurological disease that would (1) quickly identify neurological deficits thereby expediting the diagnostics and treatment process, and; (2) improve outcomes by minimizing inter-operator variability and operator error resulting in more accurate diagnosis and triage of neurological patients. Once developed, our system will provide real-time, clinically relevant, triaging information for cases when a neurological event happens and access to neurological expertise is limited.

The system will evaluate for signs of neurological disease from a video of a standard patient examination. We aim to deploy the system on commonly available camera-enabled devices (e.g. laptop computers, tablets, phones, etc.), without the need for additional specialized hardware, and without the need to transfer image/video data from the device to servers (all processing to be performed on the device). An overview is shown below.

centered image

The overview of our system.


Stroke . In the United States, stroke is a leading cause of death and disability with more than 795,000 people suffering a stroke every year[2]. The mainstay treatment of acute ischemic stroke (AIS) is rapid reperfusion therapy with intravenous tissue plasminogen activator (tPA) and/or endovascular thrombectomy (EVT), which substantially decrease the odds of death or disability[3]. Numerous studies have shown that reduction in time to treatment by as little as fifteen minutes significantly decreases mortality and disability in stroke patients[4-6]. Recognition of stroke in the prehospital setting by emergency medical services (EMS) leads to earlier and more frequent treatment with acute stroke therapy[7-10]. However, the identification of stroke by non-neurologists can be challenging, as illustrated by the finding that paramedics fail to detect stroke in high as 56% of patients when not using a diagnostic aid [11].

Traumatic brain injury . Traumatic Brain Injury (TBI) includes concussion, subconcussion, and most exposures to explosive blasts from improvised explosive devices [12]. One study indicates that the prevalence of TBI among returning service members range from 15.2% to 22.8%, affecting as many as 320,000 troops [13]. The delay in diagnosis increases the risk of developing Alzheimer's disease, Parkinson's disease, motor neuron disease, and stroke [12,14,15]. For instance, a recent study demonstrated that TBI was associated with 10.21, 4.61, and 2.32 times greater risk of stroke at three months, one year and five years after the injury, respectively [16].


To differentiate normal examinations from abnormal ones, we utilize an ERT(Ensemble of Regression Trees) model [17] for extracting facial location from movie frames and an PAF (Part Affine Fileds) model [18] for extracting pose. Once appropriately localized, features (e.g. optical flow, local binary patterns, etc.) can be extracted and machine learning can be utilized to regress the correct label (diagnosis) of a given video examination.

Data acqusisition

At the University of Virginia we’re acquiring a "neurological deficit" data set comprised of 200 videos for each of the following neurological deficits: facial weakness, limb weakness, limb ataxia, nystagmus, and gaze deviation. An additional 200 videos of non-stroke controls will be recruited from healthy volunteers. This video database will be sorted into derivation and validation sets to develop and test the individual deficit detecting algorithms.

Preliminary results

Our current prototype for the proposed tool for facial deficit detection, called F-DIT-V, achieves a classification accuracy of 92.9%, precision of 93.6%, recall of 92.8%, and specificity of 94.2%, which is higher and more reliable compared with the LBPTOP[19] and the LSTM[20], which are widely used in previous and current studies for activity recognition.

The prototype has been implemented as a live, real time tool, able to run on modest hardware such as commonplace laptops. Demonstration of the BANDIT (brain attack and deficit identification tool) on a volunteer mimicking a normal examination, as well as left and right deficits are shown below.

--- prototype ---

centered image

Demo of proposed video classification tool: normal case

centered image

Demo of proposed video classification tool: left deficit

centered image

Demo of proposed video classification tool: right deficit


[1] Uribe, Omar, et al. "Automated Detection of Facial Weakness Using Machine Learning." 2018 International Stroke Conference.
[2] Uribe, Omar, et al., American Neurological Association Annual Meeting, (2018).
[3] McDonald, Mark, et al. "Comparison of Human and Machine Learning Based Facial Weakness Detection." 2019 International Stroke Conference.
[4] Zhuang, Yan, et al. “Pathological facial weakness detection using computational image analysis,” 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI), IEEE, 2019.
[5] Zhuang, Yan, et al. "F-DIT-V: An Automated Video Classification Tool for Facial Weakness Detection." 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI). IEEE, 2019.