Purpose Built Llms For Dental Note Taking

Purpose-built LLMs for dental note-taking Announcing our Series F . Learn more

AI engineering

Purpose-built LLMs for dental note-taking

Frontier thinking model performance at a fraction of the latency.

Authors

Harry Partridge

Charles O'Neill

Last updated November 5, 2025

Share

Dentists spend a surprising amount of time writing notes. It's one of those unsexy problems that affects thousands of practitioners daily—converting rambling patient conversations into structured clinical documentation. Baseten has worked with a leader in the dental note-taking industry to create a purpose built, low latency model for dentists to use as an ambient scribe. This customer was initially interested in whether a smaller, more efficient model could be used to perform a single task: producing structured dental notes from a raw transcript. These transcripts were up to 25000 tokens long, and involved complex examination and treatment procedures. Subsequently, they were also interested in whether the same model could be used for another two tasks: real time note taking and note alteration. This resulted in three distinct tasks for our model to perform Task 1 : Convert ambient transcripts into structured clinical notes

Task 2 : In real time, add new findings to existing notes as procedures unfold

Task 3 : Alter existing notes and enhance their quality

Remarkably, our model learned all three tasks simultaneously within a single LoRA adapter (rank 256), whilst still handling specialized dental terminology, multiple tooth notation systems and long, complex transcripts. We eventually deployed a fine-tuned model that matches gemini-2.5-pro 's accuracy while responding 5-6x faster, on three different tasks with the one model. For real-time updates, we achieved performance that surpasses all other models by significant margins, all while handling domain-specific complexity that breaks general-purpose LLMs. Building Evaluators That Know What Good Looks Like Rather than guessing what good looks like, we built comprehensive evaluation frameworks using Lumina , our adaptive evaluation engine that learns to judge like domain experts. Using Lumina, we analyzed thousands of outputs from models across the capability spectrum—from Llama-3-8B to GPT-5. This allows us to catalogue exactly why each model failed, and partition all the errors into semantically related checks. The patterns that emerged were interesting. In addition to the usual LLM errors involving hallucination and omission of key information, we identified a number of errors unique to this customer’s specific task. We observed that models would identify clinical findings correctly but place them in the wrong section. They'd use dental terminology that demonstrated a misunderstanding of the details of a given situation. They'd confuse tooth notation systems in predictable ways. We turned these failure modes into six binary evaluators: Clinical terminology : Does the note use all appropriate terminology, as if an expert dentist wrote it?

Contradictions : Any conflicting statements?

Information completeness : Did we omit any key details from the transcript?

Source fidelity : Did we add anything that wasn't there? Were there any hallucinations?

Structural compliance : Right format, right sections?

Tooth notation : Did we get the teeth right?

Training with Iterative SFT: From Good to Perfect With robust evaluators in place, we employed iterative SFT (iSFT) to train our model. This is a technique that makes use of the full reasoning from LLM-as-a-judge evaluations, leading to faster and cheaper training when compared to RL, which only utilizes the numeric judgements. The iSFT Process for Clinical Notes For each of our 30,000 training examples: Generate : Model produces initial clinical note from transcript

Evaluate : Lumina evaluators identify specific failures (e.g., “incorrect tooth notation in paragraph 3”)

Refine : Model repairs its output based on structured feedback

Iterate : Repeat until all six evaluators pass

Train : Use these perfect outputs for supervised fine-tuning

This process is fundamentally different from standard distillation. Instead of training on "what GPT-5 would say," we train on outputs that achieve a higher evaluation score than even GPT5, raising the ceiling for downstream performance. The information-theoretic advantage of iSFT is quite clear. While standard RL provides O(1) bits of information per example (a scalar reward), iSFT provides O(T) bits—dense, token-level supervision across thousands of tokens. This translates to: 6-7x better sample efficiency than standard SFT

10x fewer LLM evaluator calls than rejection sampling

Monotonic improvement with more data (no plateau from noisy examples)

Solving Domain-Specific Challenges: Synthetic Data Generation In order to give dentists maximal control over their notes, this customer also allowed dentists to specify the type of tooth notation to be used in the output. The dentists were able to choose from one of 4 possible notation systems: Palmer, FDI, UNS, Victor Haderup. The information about these notation systems on the internet is quite rare and inconsistent - many online websites have conflicting recommendations about how to apply these notation systems, particularly for baby teeth and for the Victor Haderup notation system which are less common. This means that by default, only slow reasoning models can consistently use these notation systems in the way desired by the dental scribe - non-thinking frontier models do not understand the notation at an intuitive level. This scribe provided us with a short (<200 word) paragraph, describing exactly how they wanted the model to use all of the four notation systems. We initially tried simply placing these descriptions into the prompt, but found that even with frontier non-thinking models like GPT-4.1 and Claude-Sonnet-4.5, generated notes would often contain errors. In practice, this would waste dentists time and potentially leading to incorrect documentation for patients. It was therefore clear that the only possible way to get the desired performance out of a non-thinking model was to train a better model. However, simply running fine-tuning on this 200 word paragraph is not sufficient to ingrain this knowledge into the model. The only remaining solution was to use synthetic data generation. However, traditional synthetic data generation hits an entropy ceiling quickly. If you...