SLM Model: Technical Documentation

HeyDonto AI SLM Model: Technical Documentation

1. Overview

The HeyDonto AI SLM (Small Language Model) has been designed, developed, and deployed to specialize in dental industry applications, focusing on the analysis of dental x-rays and integration with Electronic Health Records (EHRs). This document outlines the technical architecture, training process, the use of university-provided datasets, and the model’s capabilities within the dental healthcare domain.

2. Model Architecture

The core technology of the HeyDonto AI SLM is based on Meta’s LLaMA 3.2 Vision Model, a multimodal model with the ability to handle both image and text data simultaneously. The model has been fine-tuned extensively using real-world dental x-rays and associated FHIR-compliant EHR records, specifically tailored for dental clinics.

2.1. Multimodal Design

Image Data: The image component processes dental x-rays, using a convolutional neural network (CNN) that has been pretrained on general medical images and fine-tuned on dental-specific datasets.
Text Data: The text-processing component leverages transformers to interpret structured EHR data in FHIR format, correlating medical history and patient data with visual findings in the x-rays.

2.2. Key Model Features

CNN Backbone for Dental Images: The model’s CNN was adapted specifically for analyzing dental radiographs, building on knowledge acquired from broader medical images.
Multimodal Fusion Layer: This layer integrates the outputs from both the image and text data processing streams, enabling the model to produce unified insights based on dental x-rays and patient medical history.

3. Training Process

3.1. Pretraining on General Medical Datasets

The initial phase of model development involved pretraining on general medical imaging datasets to ensure a solid foundation in medical image analysis.

MURA Dataset: This musculoskeletal dataset provided early training for the model, helping it learn key image recognition tasks(Collective Minds Radiology).
NIH Chest X-ray Dataset: While not specific to dentistry, this dataset helped train the model on basic radiological features(BioMed Central).

3.2. Fine-tuning with Dental X-ray and EHR Data

The model was fine-tuned on specialized dental x-ray datasets and structured EHR records obtained from onboarded clinics.

Dental X-rays: We used a combination of publicly available dental x-ray datasets (like Dental Panorama Radiographs) as well as real-world x-rays sourced from the dental clinics that partnered with HeyDonto(GitHub).
EHR Data: The model was trained to integrate structured FHIR-based EHR data from the clinics. By correlating patient history with x-ray data, the model can offer predictive insights and detailed diagnoses.

3.3. Training with University-Provided Data

Through collaborations with universities in Germany, HeyDonto obtained access to extensive and specialized dental datasets. These datasets, which include annotated dental x-rays from academic research, allowed for the further refinement of the model. This data included:

Panoramic X-rays and Intraoral Radiographs: These images were labeled with dental conditions, treatments, and diagnoses, providing high-quality training data for the model.
Advanced Annotations: University researchers provided expert annotations that helped improve the model’s ability to detect subtle dental pathologies and suggest precise treatments.

This collaboration also allowed for access to large, anonymized datasets from ongoing dental research projects, significantly increasing the amount of available training data.

3.4. Data Augmentation

To increase the robustness of the model, data augmentation techniques were applied to the x-rays:

Rotation, Scaling, and Contrast Adjustments: These augmentations were applied to simulate variations in image quality and capture real-world conditions under which x-rays are taken.
Synthetic Data Generation: GANs (Generative Adversarial Networks) were used to generate synthetic x-ray images when real-world data was sparse(Collective Minds Radiology).

4. Model Capabilities

4.1. Dental X-ray Analysis

The HeyDonto AI SLM model specializes in analyzing dental x-rays, offering the following capabilities:

Pathology Detection: The model detects dental issues such as cavities, root fractures, impacted teeth, and periodontal disease.
Procedural Suggestions: Based on the detected conditions, the model suggests appropriate treatments such as fillings, extractions, or root canals, aiding clinical decision-making.

4.2. EHR Correlation and Predictive Insights

The integration of FHIR-based EHR data allows the model to:

Correlate Findings with Medical History: The model analyzes x-ray results in the context of the patient’s medical and dental history, improving diagnostic accuracy.
Predict Treatment Needs: Based on past treatments and current x-ray data, the model provides predictions for future treatment requirements.

4.3. Operational Support for Clinics

Optimized Inventory Management: The model tracks procedure trends and predicts upcoming treatment needs, helping clinics optimize inventory and manage operational workflows.
Personalized Care Recommendations: By analyzing EHR data and medical history, the model generates personalized treatment recommendations for individual patients.

5. Data Resources and Inventory

5.1. Dental X-ray Inventory

The x-ray data used for model fine-tuning came from multiple sources:

University Collaborations: Dental schools provided high-quality datasets containing annotated dental radiographs, allowing the model to learn from expert-labeled data.
Clinic Data: Onboarded clinics provided anonymized real-world dental x-rays, covering a wide range of conditions and demographics.

5.2. FHIR-Based EHR Data

The FHIR-based EHR records were collected from the dental clinics using HeyDonto’s platform, which provided a standardized structure for patient data, enabling the model to seamlessly integrate medical history with diagnostic image analysis.

6. Ethical Considerations and Data Privacy

6.1. Data Anonymization

All data sourced from clinics and universities was anonymized to comply with GDPR and HIPAA standards. Patient-identifying information was removed, ensuring privacy throughout the training and deployment phases.

6.2. Data Security

Data was encrypted during storage and transfer to ensure secure handling of sensitive medical information. Only authorized personnel had access to the dataset, ensuring full compliance with healthcare regulations.

7. Model Deployment

The HeyDonto AI SLM model has been deployed through the HeyDonto AI API, allowing dental professionals to use it in real-time within their clinical workflows:

X-ray Diagnostics: Dentists can upload x-rays through the API and receive diagnostic results within seconds, along with procedural recommendations.
FHIR Integration: The API interfaces directly with clinic EHR systems, pulling relevant patient history for more accurate and personalized insights.

The model is hosted on Google Cloud Platform (GCP), ensuring scalability, high availability, and robust security for both data processing and inference tasks.

8. Conclusion

The HeyDonto AI SLM model has been meticulously trained, leveraging both clinic-sourced EHRs and university-provided dental x-ray datasets to develop a highly specialized tool for the dental industry. By integrating multimodal data from both images and textual medical records, the model enhances diagnostic workflows, predicts treatment needs, and optimizes operational efficiency for dental clinics. This documentation details the full technical journey, from pretraining to fine-tuning with specialized data, and the deployment of the model in real-world clinical environments.