Detecting Clinically Relevant Emotional Distress and Functional Impairment in Children and Adolescents: Protocol for an Automated Speech Analysis Algorithm Development Study

Background Even before the onset of the COVID-19 pandemic, children and adolescents were experiencing a mental health crisis, partly due to a lack of quality mental health services. The rate of suicide for Black youth has increased by 80%. By 2025, the health care system will be short of 225,000 therapists, further exacerbating the current crisis. Therefore, it is of utmost importance for providers, schools, youth mental health, and pediatric medical providers to integrate innovation in digital mental health to identify problems proactively and rapidly for effective collaboration with other health care providers. Such approaches can help identify robust, reproducible, and generalizable predictors and digital biomarkers of treatment response in psychiatry. Among the multitude of digital innovations to identify a biomarker for psychiatric diseases currently, as part of the macrolevel digital health transformation, speech stands out as an attractive candidate with features such as affordability, noninvasive, and nonintrusive. Objective The protocol aims to develop speech-emotion recognition algorithms leveraging artificial intelligence/machine learning, which can establish a link between trauma, stress, and voice types, including disrupting speech-based characteristics, and detect clinically relevant emotional distress and functional impairments in children and adolescents. Methods Informed by theoretical foundations (the Theory of Psychological Trauma Biomarkers and Archetypal Voice Categories), we developed our methodology to focus on 5 emotions: anger, happiness, fear, neutral, and sadness. Participants will be recruited from 2 local mental health centers that serve urban youths. Speech samples, along with responses to the Symptom and Functioning Severity Scale, Patient Health Questionnaire 9, and Adverse Childhood Experiences scales, will be collected using an Android mobile app. Our model development pipeline is informed by Gaussian mixture model (GMM), recurrent neural network, and long short-term memory. Results We tested our model with a public data set. The GMM with 128 clusters showed an evenly distributed accuracy across all 5 emotions. Using utterance-level features, GMM achieved an accuracy of 79.15% overall, while frame selection increased accuracy to 85.35%. This demonstrates that GMM is a robust model for emotion classification of all 5 emotions and that emotion frame selection enhances accuracy, which is significant for scientific evaluation. Recruitment and data collection for the study were initiated in August 2021 and are currently underway. The study results are likely to be available and published in 2024. Conclusions This study contributes to the literature as it addresses the need for speech-focused digital health tools to detect clinically relevant emotional distress and functional impairments in children and adolescents. The preliminary results show that our algorithm has the potential to improve outcomes. The findings will contribute to the broader digital health transformation. International Registered Report Identifier (IRRID) DERR1-10.2196/46970

-The proposed approach of using Gaussian Mixture Model, EM, and RNNs was presented but no justification was given as to why they expect these approaches will be accurate.
-There is little preliminary analysis of the data to suggest that this architecture will actually work.

Suggestions: None
The panel assigned the following overall ranking to this proposal: Competitive The summary was read by/to the panel and the panel concurred that the summary accurately reflects the panel discussion.

Title:SBIR Phase I: Automated Emotional Distress Severity Classification Using Speech Analytics and SFSS for SUD and OUD-Related ACE and Trauma
Institution:TQIntelligence, Inc.

Review:
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.
This SBIR Phase I project will develop a voice-based tool to classify emotional distress severity for children and adolescents ( families with low socio-economic status) at-risk of trauma due to Adverse Childhood Experiences. The research work will focus on testing and validating two machine learning models to identify and predict emotional disorder severity.
Strength: + Emotion analysis fro voice is an interesting research area for machine learning.
Weaknesses: -The major research and innovation of this project is to use machine learning to classify emotional disorder severity from voice, which is task 1.4. However, only a very small part of the proposal description is spent on discussing the technical approach. Many problems and concerns are not addressed, e.g., any existing methods? why will the proposed method work? The Gaussian Mixture Model and LSTM are mainly used in speech recognition (sequence data), which convert speech to text. Why will it work on detecting emotion?
-As the main innovation and risk is on developing machine learning algorithms, it is disappointing and confusing why so much discussion and budget is on data collection, therapist training, and personnel with general management and startup experience.
-Different from speech recognition, personal talking style can affect an automatic emotion identification tool significantly. Personalization should be a factor in building such a tool.
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.
Strength: + The proposed work will have positive impact to society. + The project has recruited two Georgia Based Behavioral Health Organizations as Pilot sites for product development.

Review (PI Copy)
Weaknesses: -The commercialization and marketing plan is lack of details and not convincing.
-It is disappointing and unrealistic to focus hiring on sales while a solid product is still under development.
-The CVs do not follow NSF requirements.
Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable

Summary Statement
The proposed work has positive impact to health and society in general. However, there are serious flaws with technical discussion, project plan and priority setting, innovation, personnel, and budget in this project.

Review:
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.
The goal of the project is to develop an automated way to detect emotional distress and functional impairments in children and adolescents. The PIs will leverage a software app called TQI App that has already been developed to collect the data, and analyze the voice samples using ML techniques to detect children in need and at high-risk.
Strengths: + The idea of extracting distress signals using voice samples is quite interesting.
+ The availability of a channel for collecting data using the TQApp is a strength.
+ Close ties to orgs (with support letters) such as Family Ties, Georgia Hope provides them adequate patients to collect appropriate data to carry out the proposed work.
+ The team seems to be extremely strong with the PI being an expert on psychological disorders with both research and implementation. It also contains seasoned business and technological leadership coupled with several experts who are leaders spanning technology and healthy data analysis. Weaknesses: -The proposed approach of using Gaussinan Mixture Model, EM, and RNNs was presented but no justification was given as to why they expect these approaches will be accurate. There is also no preliminary analysis of the data to suggest that this architecture will actually work.
In the context of the five review elements, please

Review (PI Copy)
evaluate the strengths and weaknesses of the proposal with respect to broader impacts.
The project has the potential to detect and provide care to the highly vulnerable (and large) population of children of substance-abuse parents, that are often prone to mental and psychological disorders if left untreated. The technological advances made by this project will also help address the shortage of trained mental health professionals in providing mental care to families etc.
Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable Summary Statement I really liked the proposal. They have identified a nice problem that can be potentially solved with ML/AI techniques. The team is also extremely strong. They also have strong ties with organizations to access data required to carry out the project. The only remaining risk is whether the proposed technology solution for analysing voice samples will be accurate, which I think is worth funding.

Review:
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit. Strength: 1. This SBIR project proposes to develop a voice-based emotional distress severity classification tool for children and adolescents at-risk of trauma due to Adverse Childhood Experiences (ACE). 2. Their tool will help mental health providers to detect the severity of emotional distress early, efficiently, and automatically.
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.
1. The proposal will improve the timely and objective measurement of mental health issues and vulnerabilities in children and adolescents using innovative technology. 2. The proposal will divert high-risk youth from joining the next cohort of a population with a deadly Opioid Use Disorder.
Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable This proposal will detect and predict mental health disorder severity of at least 2 million American children and adolescents whose parents with OUD each year, with the capability of providing feedback at the point of care for patients, families, and mental health providers.

Summary Statement
In this proposal, the PIs plan to develop a voice-based emotional distress severity classification tool for children and adolescents at-risk of trauma due to Adverse Childhood Experiences (ACE). This proposal will help 2 million American children and adolescents whose parents with OUD each year. The tool can also help mental health providers to detect the severity of emotional distress early, efficiently, and automatically. I have a little concern on their experience of development of deep learning based

Review:
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.
The investigators propose to develop a machine learning algorithm that detects clinically relevant emotional distress in speech samples from at-risk youth receiving mental health and family preservation services. The algorithm to detect and predict emotional disorder severity is likely to be preliminary and may require additional data and funding to develop a more robust classification system. + Benchmarks and evaluation metrics are included.
-The innovation appears to be modest and is not clearly articulated. There is a great deal of prior work on emotion detection that appears to be overlooked.
-The approach in general is not rigorous.
-The PI has limited publications and research track records. The speech processing expertise appears to lie outside the company.
-The bio-sketches should be in NSF format.
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.
The proposed study, if successfully executed, will have a broader impact on community health and families.
The broader impacts on other disciplines is not highlighted.
Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable -It is not clear how the revenue is calculated.