Original Paper
Abstract
Background: Recent studies have identified significant gaps in equity, diversity, and inclusion (EDI) considerations within the lifecycle of artificial intelligence (AI), spanning from data collection and problem definition to implementation stages. Despite the recognized need for integrating EDI principles, there is currently no existing guideline or framework to support this integration in the AI lifecycle.
Objective: This study aimed to address this gap by identifying EDI principles and indicators to be integrated into the AI lifecycle. The goal was to develop a comprehensive guiding framework to guide the development and implementation of future AI systems.
Methods: This study was conducted in 3 phases. In phase 1, a comprehensive systematic scoping review explored how EDI principles have been integrated into AI in health and oral health care settings. In phase 2, a multidisciplinary team was established, and two 2-day, in-person international workshops with over 60 representatives from diverse backgrounds, expertise, and communities were conducted. The workshops included plenary presentations, round table discussions, and focused group discussions. In phase 3, based on the workshops’ insights, the EDAI framework was developed and refined through iterative feedback from participants. The results of the initial systematic scoping review have been published separately, and this paper focuses on subsequent phases of the project, which is related to framework development.
Results: In this study, we developed the EDAI framework, a comprehensive guideline that integrates EDI principles and indicators throughout the entire AI lifecycle. This framework addresses existing gaps at various stages, from data collection to implementation, and focuses on individual, organizational, and systemic levels. Additionally, we identified both the facilitators and barriers to integrating EDI within the AI lifecycle in health and oral health care.
Conclusions: The developed EDAI framework provides a comprehensive, actionable guideline for integrating EDI principles into AI development and deployment. By facilitating the systematic incorporation of these principles, the framework supports the creation and implementation of AI systems that are not only technologically advanced but also sensitive to EDI principles.
doi:10.2196/63356
Keywords
Introduction
Artificial intelligence (AI) is a branch of engineering and computer science that creates systems capable of performing tasks that typically require human intelligence such as visual perception, speech recognition, and language translation, while also analyzing complex datasets to identify patterns, make predictions, and solve problems across various domains [
- ]. AI technologies are increasingly being applied in all sectors of our society including health care systems [ - ]. These systems could be helpful in improving the effectiveness of health care delivery and facilitate achieving quadruple aims of care, that is, improved patient experience, better outcomes, lower costs, and improved clinical experience [ ]. The introduction of AI in medicine and dentistry has shown promising potential [ , , ] in screening, detection, and treatment of diseases [ , ]. AI enhances the use of electronic health records, supports clinical decision-making, and enables the development of sophisticated robots and devices to enhance care delivery and alleviate workload in health care systems [ , ].The deployment of AI across various sectors has underscored the importance of ethical governance and the integration of equity, diversity, and inclusion (EDI) principles. Drawing on prior research from various industries, it is evident that AI technologies can both mitigate and exacerbate existing social biases, depending on their design and implementation [
, , ]. This dual potential of AI, as highlighted in studies like Obermeyer et al [ ], emphasizes the necessity for incorporating EDI principles from the onset of development to ensure technologies promote inclusivity and prevent social inequality. Moreover, research on algorithmic accountability underscores the need for transparency and explainability in AI systems, requiring designs that provide traceability of decisions and clear data use to comply with international regulations [ ]. Additionally, insights into data privacy stress the importance of robust data protection practices to prevent technology misuse and protect personal information, ensuring AI applications uphold individual rights and freedoms [ ]. These principles, drawn from a broad array of previous research, constitute a comprehensive framework essential for the ethical deployment of AI technologies across all domains.Serious concerns have been expressed about how AI might undermine the basis and essence of humanity [
]. Hypothetical syllogisms, which are arguments containing at least 1 hypothetical proposition as a premise, suggest a dystopian future with unfairness enacted by AI [ ]. Recent studies also revealed that EDI concepts are not adequately implemented in the building of AI systems for health and oral health care [ , - ]. For example, the use of AI in health care raises concerns about the privacy and security of sensitive medical information, particularly for patients who are vulnerable and may have a history of discrimination and mistrust in the health care system [ ]. EDI is a conscious effort to include a diverse range of social identities and perspectives, including those typically marginalized in decision-making on social and health issues that impact their lives.By definition, equity strives to treat everyone fairly with the recognition that each person has different circumstances and allocates the exact resources and opportunities needed to reach an equal outcome [
]. Diversity means including a range of people and perspectives such as those from different racial and ethnic backgrounds, genders, sexual orientations, ages, abilities, socioeconomic statuses, religions, and cultural backgrounds. Inclusion means the act of social inclusion of groups that are normally marginalized as well as the social values they subscribe to [ ]. Paradoxically, some evidence suggests that AI can lead to fairness and efficiency by reducing the likelihood of human bias, but our decisions will determine how the future unfolds [ ].In this study, we argue that ethical and fair AI can be achieved only by integrating the EDI concept in the development and application of AI and machine learning models [
, , , ]. However, it is critically important that the building of AI systems be participatory in nature and includes diverse experts and stakeholders like system developers, health professionals, researchers, and policy makers, among other end users [ ]. Unfortunately, this research shows that studies on the application of AI in health care have rarely included relevant stakeholders in the design, development, and implementation of AI systems [ , , ].The distinction between AI ethics and a framework concerning AI and EDI is important, emphasizing different facets of responsible AI development. AI ethics broadly encompasses moral values, guiding the development and application of AI technologies in a manner that respects human dignity and societal norms. It encompasses principles like transparency, accountability, and fairness, aiming to prevent AI systems from perpetuating harm or injustice [
]. In contrast, the framework focuses specifically on integrating these considerations as well as EDI principles throughout the AI system’s lifecycle—from design and development through deployment and monitoring. This lifecycle approach ensures that EDI principles are central, actively promoting inclusivity and mitigating biases [ ]. This clarifies how AI ethics provides the foundational values, while the framework offers a structured approach to operationalize these values concretely and continuously across all stages of AI implementation.Although studies have identified limitations, to the best of our knowledge, no studies to date have explored the feasibility and challenges of incorporating EDI variables, concepts, practices, and outcomes within AI technologies in the field of health and oral health care. There is a need to explore how EDI principles can be integrated into the lifecycle of AI technologies and gain an understanding of the relevant stakeholders’ perspectives on societal challenges and implementation of EDI in AI technologies for health care, oral health care, and related research. This study aimed to co-design the EDAI framework for integrating EDI throughout the lifecycle of AI in partnership with representatives of socio- or ethnocultural communities and end users, as well as with an international or intersectoral or interprofessional team of policy makers, researchers, clinicians, and industry representatives.
Methods
Study Design
Our research project began with a gap analysis and a comprehensive systematic scoping review, which explored the integration of EDI principles into the life cycle of AI within health and oral health care settings. The results of this initial systematic scoping review have been published separately [
]. In this paper, we focus on the other phases of the project, which is directly related to the framework development stage. This study followed a qualitative exploratory study design with a constructive grounded theory methodology approach and was presented according to the SRQR (Standards for Reporting Qualitative Research) guideline ( ). Grounded theory enables “the generation of theories of process, sequence, and change pertaining to organizations, positions, and social interaction” [ ]. This theory is a systematic methodology in which the researcher is not bound to select an existing theoretical framework. Instead, the findings may serve as the foundation for the development of new theories [ , ]. Therefore, for developing conceptual frameworks from diverse texts like the one presented in this study, the grounded theory technique is appropriate and insightful [ ]. Constructivist grounded theory is a type of grounded theory that is based on constructivism [ ]. It is centered on the coconstruction of experiences and meanings of the area of inquiry with the participants [ ].Ethical Considerations
This study received ethics approval from the McGill University Institutional Ethics Committee (A03-E10-23A). Informed consent was obtained from all participants. Participants were also informed of their right to opt out at any time. Data collected in this study were deidentified; personal identifiers such as names were not recorded. Participants were not compensated monetarily; however, food was provided during the workshop.
Study Phases
This research was conducted in 3 phases, and each phase is explained below in detail.
Phase 1: Gap Analysis
A comprehensive scoping review has been performed to explore what and how EDI principles and practices have been integrated into the conception, design, development, and implementation of AI studies conducted in health and oral health care settings [
]. The results of the scoping review indicate the lack of a comprehensive framework in health care, in which the principles of EDI are systematically deliberated and applied to the AI lifecycle [ ]. Therefore, an expert group workshop was planned to develop a comprehensive framework, that is, EDAI, for health and oral health care settings. The review results informed the EDAI framework’s development.Phase 2: Expert Group Workshops
Establishing a Multidisciplinary Team Involving Stakeholders and Collaborators
The framework development process was structured around 2 strategic workshops. The initial workshop in Montreal, Canada, centered on collaborative creation, bringing together diverse stakeholders to codevelop the framework’s foundational elements. Building on this groundwork, the second workshop in Geneva, Switzerland, was dedicated to refining and validating the preliminary version. This 2-phase approach ensured a comprehensive, iterative process that incorporated broad expertise while allowing for critical assessment and enhancement of the framework’s applicability. The purposeful sampling with maximum variation was used to enroll members from an intersectoral or multidisciplinary background (engineering, dentistry, medicine, AI systems, ethics in AI, communication, and social sciences) including researchers, clinicians, knowledge users, patient representatives, community partners, policy makers, public health agencies, and industry representatives. The selection of stakeholders started 1 year before the first workshop to build an expert research group. The team members were consciously selected to represent excellence in research activities or leadership and diversity in socio- or ethnocultural and geographical contexts. The team’s composition is in line with the participatory approach and the concept of codevelopment, which are at the core of this project. Most of the team members had a long-established working relationship and had successfully collaborated on various projects previously.
Data Collection
Overview
The study follows a qualitative research approach and includes 2-day, by-invitation, in-person workshops with focus group discussions to co-design the EDAI framework [
]. According to Ahmed and Mohd Asraf [ ] and Lain [ ], workshops are essential for prolonged engagement and are regarded as being one of the primary ways of establishing the credibility of the results of a qualitative study and upholding the trust between the researchers and participants. The workshops continued for 2 days and involved representatives from diverse backgrounds including government and nongovernment organizations, visible minority community leaders including Black communities, representatives from the World Health Organization, and the participation of health and oral health care providers, academia, patients, industry partners, and policy makers [ ].Workshop Engagement Exercises
The workshops included plenary presentations, round table working groups, and focused group discussions as a research methodology [
, ]. At the preworkshop phase, to facilitate the exchange of ideas, research team members prepared a draft document relevant to the area of research and distributed it to meeting participants 2-3 weeks before the workshops [ ].- Workshop presentations: Individuals were selected from the attendees to provide presentations on their previous experiences in EDI to enhance the discussions. A semistructured focus group discussion guide was developed before the workshop based on a review of the literature and previous work of team members [ ]. A panel presentation linked the presentations to focus groups and included expert opinions on these questions: (1) What are the visions and values that guide the integration of EDI throughout the lifecycle of AI? (2) What are the facilitators and barriers to the integration of EDI throughout the lifecycle of AI within health and oral health care? (3) What should be the indicators of EDI in health and oral health care data and how can we measure the EDI variables throughout the lifecycle of AI? and (4) How a conceptual framework should be framed and what are the main elements?
- Round table working groups and focused group discussions: The presentation session was followed by focused group discussion groups based on qualitative methodology and inductive reasoning [ ]. The discussion groups answered the same questions that had been asked of the panel participants.
Each focus group had a moderator and notetaker with 5-6 purposefully selected participants with maximum variation and lasted for 45-60 minutes. Afterward, there was a large group discussion among the workshop attendees to further explore the subject and reach a consensus together. All proceedings notes were written by the notetakers. All the notes and memos were compiled for data analysis.
Workshop Conclusion
The workshops were concluded with a summary of the workshop by restating the main points and identifying components for preliminary data analysis.
Data Analysis
Data analysis was followed immediately after each workshop. An inductive thematic analysis was done manually (SAR, EE, and RS). As per Braun and Clarke [
], the analysis began with the generation and collation of initial codes, which will then be sorted into multiple potential themes. These potential themes were then combined to form overarching key themes. During this process, working groups were created across team members to ensure data triangulation.We followed the 8 phases of “conceptual framework analysis” within the grounded theory approach to systematically identify and interpret concepts, categories, and their relationships as explained by Jabareen [
]. Phase 1 is the mapping of data sources consisting of a literature review and workshop with focus group discussions [ ]. Since EDI in AI is an unexplored concept, firsthand knowledge and experience using primary data collection was deemed important [ ]. In phase 2, categorizing the selected data, and phase 3, identifying and naming concepts, the first set of concepts was inductively developed using line-by-line coding of focused group discussions [ , ]. After the identification of the patterns within the concepts, focused coding was initiated to establish initial categories [ , ]. Phase 4 is deconstructing and categorizing concepts, and phase 5 is integrating concepts and refining the coding by iterative assessment [ , ]. It included deconstructing and organizing the concepts and then integrating similar concepts together [ ]. Constant comparative technique at each level of coding was used to assess similarities and differences for refining the concepts [ ]. Phase 6, synthesis and resynthesis, included organizing the EDAI conceptual framework by inductively sorting the categories and grouping them under 3 main dimensions [ , ]. Phase 7, validation, and phase 8, rethinking the conceptual framework, were performed via triangulation, reflexivity, and participant feedback during the second workshop and via emails and digital meetings during data analysis [ , ].Phase 3: Framework Development and Stakeholder Reviews
Based on the workshops’ key impressions and group discussions, the EDAI framework was developed and discussed with all participants electronically and in the second workshop. Their feedback was sought and iteratively incorporated to develop the final framework.
illustrates the key steps and the details of the process across the different phases of the study.Phase and step | Details | |
Phase 1: gap analysis | ||
1. Scoping review | A comprehensive scoping review was performed to explore how EDIa principles are integrated into AIb studies in health and oral health care settings. | |
Phase 2: expert group workshops | ||
2. Establishing a multidisciplinary team | A multidisciplinary team was built through purposeful sampling with maximum variation, including stakeholders from diverse backgrounds (eg, engineering, dentistry, medicine, AI, ethics, and social sciences). The selection began 1 year before the first workshop. The team composition emphasized codevelopment, participatory approaches, and diversity in socio- or ethnocultural and geographical contexts. | |
3. Data collection | Both 2-day, in-person workshops involved focus group discussions to co-design the EDAI framework. Representatives from diverse backgrounds, including government, nongovernment organizations, minority communities, academia, and industry, participated. The workshop included plenary presentations, round table working groups, and focused group discussions, with preworkshop preparations such as draft documents distributed to participants. Focus groups addressed key questions on integrating EDI in the AI lifecycle, identifying facilitators and barriers, and defining EDI indicators and conceptual framework elements. Discussions were moderated, and notetakers documented all proceedings for data analysis. | |
4. Workshop conclusions | The workshops concluded with a summary and identification of components for preliminary data analysis. | |
5. Data analysis | Inductive thematic analysis was conducted manually. Initial codes were generated and sorted into potential themes, which were then refined and organized into key themes through collaborative efforts. The 8 phases of “conceptual framework analysis” (eg, mapping data sources, categorizing data, identifying concepts, and integrating concepts) were used. Constant comparative techniques were used to refine the coding, and the EDAI conceptual framework was organized into 3 main dimensions. | |
6. Validation | Validation and refinement of the conceptual framework were achieved through triangulation, reflexivity, and participant feedback via emails and digital meetings as well as during the second workshop. | |
Phase 3: framework development and stakeholder reviews | ||
7. Framework development | Based on workshop findings, the EDAI framework was developed and discussed with participants electronically. Feedback was iteratively incorporated to finalize the framework. |
aEDI: equity, diversity, and inclusion.
bAI: artificial intelligence.
Results
Overview
We identified the knowledge gaps through a past systematic scoping review, the results of which were published in another paper (phase 1) [
]. The participants involved in phase 2 and phase 3 of the study included over 60 representatives from diverse backgrounds and expertise from government and nongovernment organizations, visible minority community leaders including Black communities, public health agencies, health and oral health care providers, AI and health services researchers, patients, industry partners, and policy makers. This diversity ensured balanced representation not only in terms of sex, gender, and expertise but also included other aspects such as race, ethnicity, socioeconomic background, age, and disability status.We also included international participants and people from minority groups and marginalized populations. By actively recruiting from varied demographics, we aimed to address historical disparities and support equitable participation, thereby enriching the quality and impact of our research findings (phase 2 and phase 3). Five themes emerged from the data collected in phases 2 and 3.
illustrates the coding data structure of the collected data. This table illustrates initial, focused, and theoretical codes to classify the factors affecting EDI throughout the lifecycle of AI at micro (individual), meso (organization), and macro (system) levels to reach the final outcome of quadruple aims of health care by ensuring equitable, diversified, and inclusive care.Initial codes | Focused codes | Theoretical codes | Core category | Final outcome |
|
|
| Based on theoretical codes, classifying the factors affecting EDI throughout the lifecycle of AI (planning, data collection, designing, training, deployment, and decision-making) at micro (individual), meso (organization), and macro (system) levels | To achieve quadruple aims of health care by ensuring equitable, diversified, and inclusive care |
|
|
| Based on theoretical codes, classifying the factors affecting EDI throughout the lifecycle of AI (planning, data collection, designing, training, deployment, and decision-making) at micro (individual), meso (organization), and macro (system) levels | To achieve quadruple aims of health care by ensuring equitable, diversified, and inclusive care |
|
|
| Based on theoretical codes, classifying the factors affecting EDI throughout the lifecycle of AI (planning, data collection, designing, training, deployment, and decision-making) at micro (individual), meso (organization), and macro (system) levels | To achieve quadruple aims of health care by ensuring equitable, diversified, and inclusive care |
aEDI: equity, diversity, and inclusion.
bAI: artificial intelligence.
The current iteration of the EDAI framework is recognized as a preliminary version, designed to set the foundational elements necessary for integrating EDI principles within the AI lifecycle. Given the iterative nature of grounded theory, a second iteration of framework development is done during the second workshop. This session aimed to engage a broader sample and incorporate extensive data analysis and stakeholder input. The objective was to further refine and enhance the framework, thus improving its robustness and applicability in diverse health care settings.
Theme 1: The Role of EDI and AI in Health and Oral Health Care Settings
Participants proposed leveraging AI to enhance access to health and oral health care, particularly in remote areas, by improving referral pathways. They emphasized that AI could address obstacles like cost and distance by offering alternative options such as telehealth. The discussions also highlighted AI’s potential in studying human behavior within health care and predicting health outcomes for diverse populations and the diverse use of AI in decision-making. They recommended organizing education, awareness, and training workshops to improve AI literacy among health care providers and the public, including marginalized communities. Furthermore, participants stressed the significance of incorporating EDI in patient data collection and analysis to reduce health and oral health disparities.
Theme 2: Vision and Values That Will Guide the Integration of EDI in the Life Cycle of AI
During the discussions, participants stressed the importance of addressing biases and discrimination in AI algorithms and ensuring fair representation in the AI algorithms including the decision-making process. They emphasized the need to achieve this by including diverse perspectives and voices, empowering marginalized communities, and addressing cultural sensitivities while respecting individual identities. Additionally, the participants highlighted the significance of incorporating principles of social justice and decolonization at various stages of the AI life cycle, including problem definition, data collection, designing, training, deployment, and decision-making. Responsible and ethical use of AI was a key aspect that participants stressed throughout the conversations.
Theme 3: The Facilitators and Barriers to the Integration of EDI Throughout the Lifecycle of AI Within Health and Oral Health Care
depicts the facilitators and barriers to integrating the EDI within AI that were discussed within the workshop at the levels of patients, health care providers, AI developers, policy makers, and stakeholders. Participants stressed the importance of establishing an EDI committee for each stage of the AI lifecycle with a mandate to define fairness from an EDI perspective and ensure context-specific implementation.
Level and facilitators | Barriers | |
At patients’ level | ||
|
| |
At health care providers’ level | ||
|
| |
At AI developers’ level | ||
|
| |
At policy makers’ level | ||
|
| |
At all stakeholders’ level | ||
|
|
aEDI: equity, diversity, and inclusion.
bAI: artificial intelligence.
Theme 4: The Indicators of EDI in Health and Oral Health Care Data and Their Measurement
We identified the EDI indicators through a past scoping review, the results of which were published in another paper [
]. During the workshop, participants categorized the EDI indicators in health and oral health care into 3 levels: individual (micro level), community (meso level), and society (macro level). details the identified EDI indicators under categorized levels.Levels and indicators | Details | |
Micro level (individual level) | ||
Sociodemographic | Age, gender, residence, education level, occupation, income, marital status, socioeconomic status, race or ethnicity, religion, literacy, health literacy (including for artificial intelligence), cultural or traditional practices, immigration or acculturation, and family or peer or community-based influences | |
Health and functional factors | Physical health, mental health, mobility, and cognitive function | |
Psychobiological factors | Health beliefs, attitude, emotions, self-esteem, perceived health status, and genetics | |
Behavioral | Lifestyle factors and habits (eg, tobacco and alcohol) | |
Meso level (organization level) | ||
Health care system | Existing health system at the place of inquiry | |
Education system | Exiting education system at the place of inquiry | |
Community infrastructure | Physical infrastructure and rurality or remoteness | |
Network and community support | Teamwork and shared ownership | |
Macro level (system level) | ||
Colonialism | Political, socioeconomic, and cultural domination by foreign power | |
Intersectionality | Racism, intersectional discrimination, and social exclusion | |
Governance | Decision-making, accountability, and leadership (eg, health care structure and funding agency) | |
Social belief | Social values and beliefs |
To measure these EDI indicators and variables throughout the lifecycle of AI, participants stated that various tools and methods can be used including quantitative methodological analysis, surveys, interviews, anonymous self-reported questionnaires, image-based data collection, tracking location, and soliciting user feedback. Additionally, qualitative evaluation methods, such as reflective texts or live feedback from users, were mentioned as another source for providing valuable insights into EDI and the user experience with AI-enabled technologies and tools.
Theme 5: Framing the EDAI Framework
Integration of EDI in AI follows a multifactorial approach. This approach is shown in
. AI-based health and oral health practices should underpin all relevant factors at micro, meso, and macro levels. Therefore, the final framework is structured over these 3 levels: micro (individual level), meso (organization level), and macro levels (system level) with the indicators underpinning the whole lifecycle of AI. The micro level comprises smaller entities like individuals or families and their interactions. The meso level involves groups or communities or organizations and their interactions. The macro level encompasses broader systems such as policies or government structures at institutional, national, or international levels and their interactions. These 3 levels are interconnected and interrelated and significantly influence each other.At the micro level, EDI encourages diverse representation and involvement of people of all sociodemographic factors (ie, age, gender, residence, socioeconomic status, and marital status) who may have diverse capabilities, skills, expertise, and values. It inspires people from various sociocultural factors such as race or ethnicity, religion, health literacy, cultural or traditional practices, immigration or acculturation, and family or peer or community-based influences. Moreover, respecting cultural diversities can widen the scope of AI-based health care. Integration of EDI within AI can positively impact an individual’s psychobiological factors including their health beliefs, attitude, emotions, self-esteem, and perceived health status. Furthermore, EDI-informed approaches are helpful in inculcating healthy lifestyle behaviors and habits by fostering an inclusive environment that respects and values diverse cultural perspectives.
At the meso level, health systems comprising both public and private health organizations and institutes should foster the integration of EDI within the lifecycle of AI in health and oral health care by emphasizing on the provision of a culturally safe environment for patients and care providers and ensuring the presence of proficient professionals. The education system should ensure that all in the school community—students, their families, teachers, staff, and administrators—feel valued and heard. Community infrastructure needs to be considered while working on AI-EDI to have better representation of all community members irrespective of physical location, rurality or remoteness, representativity, and accessibility. Furthermore, networks and community support such as social networks, community groups, and peer support should also focus on diverse representation with teamwork and shared ownership.
At the macro level, focusing on intersectionality can aid in understanding the complex interactions of various factors at micro and meso levels, thereby striving to achieve social justice. Integration of EDI within the lifecycle of AI should emphasize decolonization by minimizing the impact of settler colonialism, racism, and discrimination and acknowledging the historical injustices and systemic inequalities that have impacted marginalized communities. Furthermore, social beliefs play a significant role in shaping attitudes and actions toward EDI since positive social beliefs can foster environments where individuals from all backgrounds feel valued and empowered. In the context of EDI in the AI lifecycle, governance plays a critical role in setting policies, implementing practices, and enforcing regulations that promote EDI. Finally, in our workshops, participants also emphasized the importance of responsible AI including reliability and safety, privacy and security, transparency, and accountability [
], which should be considered during development and implementation.Discussion
Principal Findings
Our findings reveal significant gaps in EDI considerations at various stages of the AI lifecycle, from problem definition and data collection to implementation. In response, this study developed a guiding framework—the co-designed EDAI framework—that integrates EDI principles and indicators comprehensively throughout the AI lifecycle. The field of AI ethics, which emerged alongside the evolution of AI technologies, has grown in importance, as AI systems have become more advanced. Focused initially on issues like algorithmic bias, transparency, privacy, and fairness, AI ethics seeks to align AI technologies with moral principles and values. However, it is distinct from EDI in AI, which extends beyond ethical considerations to emphasize the integration of EDI throughout the AI lifecycle. Consideration of EDI not only aims to prevent biases but also ensures diverse representation and targets social equity and inclusivity.
Despite the advantages of AI, there are some concerns that underscore the necessity of consciously integrating EDI principles into the lifecycle of AI systems to ensure they are socially responsible and equitable. These concerns include discrimination, unfair treatment, and risks of developing or intensifying health-related inequalities [
]. Recent systematic reviews of EDI in AI found a lack of EDI considerations in the dataset and AI-based academic literature and a lack of EDI principles and indicators to be integrated into the AI lifecycle [ , ]. The studies also highlighted the need to integrate EDI and racial justice principles and practices in AI; however, there is no guideline or framework to promote this integration. This study is addressing this knowledge gap by identifying EDI principles and indicators to be integrated into the AI lifecycle and developing a guiding framework to guide the next-generation AI system development and implementation.Our results highlight the importance of an EDI-sensitive approach in the life cycle of AI, reducing inequities and inequalities in health and oral health. The EDI concepts should be integrated not only during AI system development but also throughout its lifecycle, including AI planning, data collection, designing, training, deployment, and decision-making. Moreover, the study results emphasize the fair representation of all stakeholders including patients, health care providers, AI developers, and policy makers. Therefore, we developed an EDAI framework in this study encompassing several EDI indicators in health and oral health care data at the individual (micro), community (meso), and society (macro) levels underpinning the whole lifecycle of AI.
In other studies, the AI or Machine Learning Consortium to Advance Health Equity and the Researcher Diversity Ethics and Equity Workgroup has established ethics and equity principles. These principles focus on fostering trust with communities, intentionally designing and implementing AI systems, creating solutions, building capacity, and re-evaluating institutional mechanisms to reset the rules [
]. The group has also developed a glossary containing 12 terms, with particular emphasis on AI development in health equity research [ ]. Furthermore, with the help of the Delphi method, Rokhshad et al [ ] developed a checklist of 11 ethical principles specific to AI-based dentistry including equity and diversity as major principles. However, these studies are providing worthy groundwork by focusing on ethical principles and health equity, the current studies lack identifying and integrating EDI principles and indicators in the AI lifecycle. The proposed EDAI framework offers a structured approach to exploring EDI concepts and the factors influencing them throughout the AI lifecycle. It addresses the need to identify and integrate EDI principles and indicators within AI systems.A recent scoping review conducted by Singhal et al [
] emphasized the crucial integration of fairness, accountability, transparency, and ethics across all phases of AI system development from conceptualization to deployment. The review also suggests using diverse datasets to ensure a more accurate reflection of the world’s population [ ]. Additionally, they advocate the active involvement of all stakeholders at every stage. Likewise, the EDAI framework outlined in this paper aims to address these recommendations, with particular attention to the factors influencing AI-driven health care technology across all stakeholder groups and throughout the lifecycle of AI system. In this line, Dankwa-Mullan et al [ ] have also proposed a framework centered on integrating health equity and racial justice principles within each stage of the AI lifecycle to ensure ethical and moral behavior in AI system development. Similar to the proposed EDAI framework, various AI ethics frameworks have emphasized interdisciplinary stakeholders’ collaboration as one of the foundations of ethical and responsible AI [ , ]. This multifaceted interdisciplinary stakeholders’ collaboration can involve active participation from policy makers, health care providers, AI experts, and patients in AI-based ethical decision-making in health and oral health care [ , ].A recent review of AI guidelines related to EDI by Cachat-Rosset and Klarsfeld [
] analyzed 46 published guidelines pinpointing 14 relevant principles and 18 practices. They reported fairness, justice, and nondiscrimination in a limited compliance approach within these guidelines. Their findings revealed a partial adherence to fairness, justice, and nondiscrimination within these guidelines, suggesting a constrained approach to compliance [ ]. The authors recommended the need for holistic guidelines over the more technical ones [ ]. Considering these recommendations, the EDAI framework endeavors to create a holistic structure that takes into account multiple factors at the individual, organizational, and societal levels.EDI can be challenging to measure because definitions of variables, for example, gender may evolve, reflecting societal changes. Additionally, the measurement of EDI may vary depending on the specific context in which AI models are applied, making it challenging to establish a standardized set of indicators. Furthermore, fairness is context-dependent and relies on data collection methods, which can vary based on factors like self-identification of the data versus government documents. However, having a ready-to-use tool can help all the stakeholders assess the integration of EDI in health or oral health-based AI systems. Therefore, based on the findings of this study, the authors plan to create and validate an evaluation-based checklist and guidelines for the integration of EDAI in health and oral health care.
The developed framework can help AI developers, who are integral to the model, by increasing their awareness of EDI principles, thereby reducing inherent biases and ensuring that development teams are diverse and inclusive, leading to more equitable and effective AI solutions in health care. For instance, the EDAI framework could facilitate the development of AI-driven diagnostic tools that better account for demographic variations, ensuring that diagnoses are accurate across diverse populations. This not only improves patient outcomes but also addresses historical biases in medical research and treatment. Another example is health care management in which AI systems designed and implemented under the EDAI framework can optimize resource allocation and patient scheduling to prioritize accessibility and minimize disparities in care delivery. Furthermore, this framework can guide the development of AI tools for patient education and engagement, ensuring that materials are culturally relevant and accessible to all, thereby improving health literacy and patient autonomy across diverse communities.
Strengths and Limitations
One of the strengths of this research work is its systematic approach and the intersectoral and multidisciplinary team. The team members were consciously selected to represent excellence in research activities and leadership and diversity in socio- or ethnocultural and geographical contexts. The team’s composition is in line with the participatory approach and the concept of codevelopment, which are at the core of this project. Moreover, the EDAI framework also attempts to understand the linkages among various indicators at individual, organizational, and system levels. However, this study has a few limitations. First, while we were unable to record the workshop sessions due to logistical constraints, detailed notes were diligently taken by designated notetakers to ensure the accuracy and richness of the data. Second, despite our efforts, we were not able to engage First Nations and Aboriginal people in this initial phase. However, we did include a diverse population from different communities, ensuring a wide range of perspectives, and we are committed to incorporating First Nations and Aboriginal voices in future iterations of the framework.
Conclusions
Recent studies have found that there are significant gaps in EDI considerations within the AI lifecycle, from the problem definition and data collection stage to implementation. However, there is currently no existing guideline or framework to support this integration. This study aimed to address this gap by identifying EDI principles and indicators to be integrated into the AI lifecycle and developing a guiding framework to steer the development and implementation of future AI systems. The co-designed EDAI framework is comprehensive, encompassing a detailed compilation of EDI indicators across the AI lifecycle, spanning from individual to organizational and system levels for all stakeholders. These insights have the potential to reshape perspectives on integrating EDI into AI and offering valuable guidance to policy makers, service providers, and AI developers to implement change in AI practices.
Acknowledgments
This study was funded by the Canadian Institutes of Health Research and the Network for Oral and Bone Health Research (Réseau de Recherche en Santé Buccodentaire et Osseuse). The authors acknowledge support from these institutions. The authors also acknowledge the support from the Fondation de L’ordre des Dentistes du Quebec. SAR is Canada Research Chair (tier II) in Artificial Intelligence and Advanced Digital Primary Health Care, received salary support from a Research Scholar Junior 1 Career Development Award from the Fonds de Recherche du Québec-Santé during part of this project, and her research program is supported by the Natural Sciences Research Council Discovery (grant 2020-05246).
Data Availability
The datasets generated and analyzed during this study are available from the corresponding author on reasonable request.
Authors' Contributions
Conceptualization was led by SAR and EE, who established the study’s goals, framework design, and research questions. The methodology was developed by SAR, EE, and RS, who crafted the qualitative approach and data collection strategies. Data curation was managed by SAR, EE, and RS. Formal analysis was conducted by RS. The original draft was written by SAR, EE, and RS. Reviewing and editing were a collaborative effort with all authors. Supervision and overall project leadership were provided by SAR and EE.
Conflicts of Interest
None declared.
SRQR (Standards for Reporting Qualitative Research) checklist.
PDF File (Adobe PDF File), 163 KBReferences
- Artificial intelligence and gender equality: key findings of UNESCO's Global Dialogue. UNESCO. 2020. URL: https://unesdoc.unesco.org/ark:/48223/pf0000374174 [accessed 2022-09-17]
- The Topol review: preparing the healthcare workforce to deliver the digital future. NHS. Feb 2019. URL: https://topol.hee.nhs.uk/wp-content/uploads/HEE-Topol-Review-2019.pdf [accessed 2022-09-16]
- Szolovits P. Artificial Intelligence and Medicine. 1st Edition. New York, NY. Routledge; 1982.
- Panda P, Bhatia V. Role of artificial intelligence (AI) in public health. Indian J Community Fam Med. 2018;4(2):60. [CrossRef]
- Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med. 2022;28(1):31-38. [CrossRef] [Medline]
- Lin SY, Mahoney MR, Sinsky CA. Ten ways artificial intelligence will transform primary care. J Gen Intern Med. 2019;34(8):1626-1630. [FREE Full text] [CrossRef] [Medline]
- Tvermosegaard M, Rønn PF, Pedersen ML, Bjerregaard P, Dahl Pedersen ID, Larsen CL, et al. Validation of cardiovascular diagnoses in the Greenlandic Hospital Discharge Register for epidemiological use. Int J Circumpolar Health. 2018;77(1):1422668. [FREE Full text] [CrossRef] [Medline]
- Abbasgholizadeh Rahimi S, Légaré F, Sharma G, Archambault P, Zomahoun HTV, Chandavong S, et al. Application of artificial intelligence in community-based primary health care: systematic scoping review and critical appraisal. J Med Internet Res. 2021;23(9):e29839. [FREE Full text] [CrossRef] [Medline]
- What is the quadruple aim and what role does it play in healthcare? Privis Health. 2019. URL: https://www.privishealth.com/what-is-the-quadruple-aim-and-what-role-does-it-play-in-healthcare/ [accessed 2022-09-15]
- Aung YM, Wong CS, Ting SW. The promise of artificial intelligence: a review of the opportunities and challenges of artificial intelligence in healthcare. Br Med Bull. 2021;139(1):4-15. [CrossRef] [Medline]
- Chen YW, Stanley K, Att W. Artificial intelligence in dentistry: current applications and future perspectives. Quintessence Int. 2020;51(3):248-257. [CrossRef] [Medline]
- Chamberlain C, McNamara B, Williams ED, Yore D, Oldenburg B, Oats J, et al. Diabetes in pregnancy among indigenous women in Australia, Canada, New Zealand and the United States. Diabetes Metab Res Rev. 2013;29(4):241-256. [FREE Full text] [CrossRef] [Medline]
- McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577(7788):89-94. [CrossRef] [Medline]
- Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism. 2017;69S:S36-S40. [CrossRef] [Medline]
- Budhwar P, Malik A, de Silva MTT, Thevisuthan P. Artificial intelligence—challenges and opportunities for international HRM: a review and research agenda. Int J Hum Resour Manag. 2022;33(6):1065-1097. [CrossRef]
- Charlwood A, Guenole N. Can HR adapt to the paradoxes of artificial intelligence? Hum Resour Manag J. 07, 2022;32(4):729-742. [FREE Full text] [CrossRef]
- Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. Oct 25, 2019;366(6464):447-453. [CrossRef] [Medline]
- Kossow N, Windwehr S, Jenkins M. Algorithmic transparency and accountability. Transparency International. 2022. URL: https://knowledgehub.transparency.org/assets/uploads/kproducts/Algorithmic-Transparency_2021.pdf [accessed 2024-11-06]
- Yu S, Carroll F, Bentley BL. Insights into privacy protection research in AI. IEEE Access. 2024;12:41704-41726. [CrossRef]
- Nyariro M, Emami E, Abbasgholizadeh Rahimi S. Integrating equity, diversity, and inclusion throughout the lifecycle of artificial intelligence in health. 2022. Presented at: AH2022: 13th Augmented Human International Conference; May 26-27, 2022; Winnipeg, MB. URL: https://doi.org/10.1145/3532530.3539565 [CrossRef]
- Emami E, Rahimi SA, Nyariro M. Integrating equity, diversity, and inclusion throughout the lifecycle of artificial intelligence for better health and oral health care: a workshop summary. J Can Dent Assoc. Nov 2023;89:n10. [FREE Full text] [Medline]
- Nyariro M, Emami E, Caidor P, Abbasgholizadeh Rahimi S. Integrating equity, diversity and inclusion throughout the lifecycle of AI within healthcare: a scoping review protocol. BMJ Open. 2023;13(9):e072069. [FREE Full text] [CrossRef] [Medline]
- Ojikutu BO, Bogart LM, Dong L. Mistrust, empowerment, and structural change: lessons we should be learning from COVID-19. Am J Public Health. 2022;112(3):401-404. [CrossRef] [Medline]
- Rajkomar A, Hardt M, Howell MD, Corrado G, Chin MH. Ensuring fairness in machine learning to advance health equity. Ann Intern Med. 2018;169(12):866-872. [FREE Full text] [CrossRef] [Medline]
- Clark CR, Wilkins CH, Rodriguez JA, Preininger AM, Harris J, DesAutels S, et al. Health care equity in the use of advanced analytics and artificial intelligence technologies in primary care. J Gen Intern Med. 2021;36(10):3188-3193. [FREE Full text] [CrossRef] [Medline]
- Dankwa-Mullan I, Scheufele EL, Matheny M, Quintana Y, Chapman WW, Jackson G, et al. A proposed framework on integrating health equity and racial justice into the artificial intelligence development lifecycle. J Health Care Poor Underserved. 2021;32(2S):300-317. [CrossRef]
- Floridi L, Cowls J. A unified framework of five principles for AI in society. In: Machine Learning and the City: Applications in Architecture and Urban Design. New York, NY. Wiley; 2022:535-545.
- Glaser BG, Strauss AL, Strutzel E. The discovery of grounded theory; strategies for qualitative research. Nurs Res. 1968;17(4):364. [CrossRef]
- Charmaz K. The genesis, grounds, and growth of constructivist grounded theory. In: Developing Grounded Theory: The Second Generation Revisited. 2nd Edition. New York, NY. Routledge/Taylor & Francis Group; 2021.
- Ralph N, Birks M, Chapman Y. The methodological dynamism of grounded theory. Int J Qual Meth. 2015;14(4):160940691561157. [CrossRef]
- Jabareen Y. Building a conceptual framework: philosophy, definitions, and procedure. Int J Qual Meth. 2009;8(4):49-62. [CrossRef]
- Chun Tie Y, Birks M, Francis K. Grounded theory research: a design framework for novice researchers. SAGE Open Med. 2019;7:2050312118822927. [FREE Full text] [CrossRef] [Medline]
- Ahmed S, Mohd Asraf R. The workshop as a qualitative research approach: lessons learnt from a "critical thinking through writing" workshop. 2018. Presented at: 4th International Postgraduate Conference on Arts, Education, and Business (IPCAEB 2018); July 3-5, 2018; Langkawi, Malaysia. URL: https://www.researchgate.net/publication/330526938_The_Workshop_as_a_Qualitative_Research_Approach_Lessons_Learnt_from_a_critical_Thinking_Through_Writing'_Workshop
- Lain S. Show, don't tell: reading workshop fosters engagement and success. Texas J Literacy Educ. 2017;5(2):160-167. [FREE Full text]
- Ørngreen R, Levinsen K. Workshops as a research methodology. Electron J e-Learn. 2017;15:70-81. [FREE Full text]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101. [CrossRef]
- Graser K, Walzer A, Hunhevicz J, Jähne R, Seiler F, Wüst R, et al. Qualitative technology evaluation of digital fabrication with concrete: conceptual framework and scoreboard. Autom Constr. Oct 2023;154:104964. [FREE Full text] [CrossRef]
- Charmaz K, Thornberg R. The pursuit of quality in grounded theory. Qual Res Psychol. 2020;18(3):305-327. [CrossRef]
- Gioia DA, Corley KG, Hamilton AL. Seeking qualitative rigor in inductive research: notes on the Gioia methodology. Organ Res Methods. 2012;16(1):15-31. [CrossRef]
- Putting principles into practice: how we approach responsible AI at Microsoft. Microsoft. URL: https://www.microsoft.com/cms/api/am/binary/RE4pKH5 [accessed 2024-10-17]
- Cachat-Rosset G, Klarsfeld A. Diversity, equity, and inclusion in artificial intelligence: an evaluation of guidelines. Appl Artif Intell. 2023;37(1):2176618. [CrossRef]
- Hendricks-Sturrup R, Simmons M, Anders S, Aneni K, Wright Clayton E, Coco J, et al. Developing ethics and equity principles, terms, and engagement tools to advance health equity and researcher diversity in AI and machine learning: modified Delphi approach. JMIR AI. 2023;2:e52888. [FREE Full text] [CrossRef] [Medline]
- Rokhshad R, Ducret M, Chaurasia A, Karteva T, Radenkovic M, Roganovic J, et al. Ethical considerations on artificial intelligence in dentistry: a framework and checklist. J Dent. 2023;135:104593. [CrossRef]
- Singhal A, Neveditsin N, Tanveer H, Mago V. Toward fairness, accountability, transparency, and ethics in AI for social media and health care: scoping review. JMIR Med Inform. 2024;12:e50048. [CrossRef]
- Tahri Sqalli M, Aslonov B, Gafurov M, Nurmatov S. Humanizing AI in medical training: ethical framework for responsible design. Front Artif Intell. 2023;6:1189914. [FREE Full text] [CrossRef] [Medline]
- Prakash S, Balaji JN, Joshi A, Surapaneni KM. Ethical conundrums in the application of artificial intelligence (AI) in healthcare—a scoping review of reviews. JPM. 2022;12(11):1914. [CrossRef]
Abbreviations
AI: artificial intelligence |
EDI: equity, diversity, and inclusion |
SRQR: Standards for Reporting Qualitative Research |
Edited by G Eysenbach, T de Azevedo Cardoso; submitted 17.06.24; peer-reviewed by F Iniesto, E Ray Chaudhuri; comments to author 09.07.24; revised version received 16.08.24; accepted 05.09.24; published 15.11.24.
Copyright©Samira Abbasgholizadeh Rahimi, Richa Shrivastava, Anita Brown-Johnson, Pascale Caidor, Claire Davies, Amal Idrissi Janati, Pascaline Kengne Talla, Sreenath Madathil, Bettina M Willie, Elham Emami. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 15.11.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.