A clinic that wants years of visit notes sorted into visit types, for reporting or billing, has a classification problem. A common way to solve it is to prompt a chat model and parse the text it writes back. Jev, a new model from TypeSafe AI, is built for the other approach: it reads a text, answers typed questions about it, and returns a probability for every option instead of generated text. Input costs $0.042 per million tokens, and output is not charged.

Before using it on real records, I wanted to know two things: how accurate it is on clinic notes, and whether its confidence can decide which notes a person should check. So I wrote 140 visit notes for a fertility and gynecology practice, in 7 visit types, and ran Jev on all of them. It labeled 138 correctly, took 2.2 seconds for all 140, and cost less than half a cent.

The picture above shows the part I care most about. With a review threshold of 0.8, both mistakes went to a person, and the 132 notes Jev labeled on its own were all correct. The rest of this post explains the setup, the two mistakes, how the confidence is computed, and what the unsure notes say about the visit type descriptions.

The notes and the visit types

Real patient notes cannot go in a blog post, so I wrote the notes for this test, drafted with Claude. There are 20 per visit type, and about 3 per type sit close to the line between two types on purpose, for example a negative pregnancy test, which counts as a results visit and not as early pregnancy. Each note has one correct type, set when the note was written, which plays the role of a clinician-reviewed label. No clinician reviewed these labels.

The notes are short, like real ones:

Bloodwork only today: E2 1,520, LH 1.1, P4 0.6. Nurse called pt - decrease FSH to 150 IU, return tomorrow for scan.

That one is cycle monitoring: a hormone check in the middle of a treatment cycle, with a dose change. The 7 types are:

Visit type What it covers
New patient consult First consultation: history, first tests, treatment options
Cycle monitoring Ultrasound or bloodwork during an active treatment cycle, dose changes, trigger timing
Fertility procedure An egg retrieval, embryo transfer or insemination (IUI) done at this visit
Early pregnancy A positive pregnancy test, early ultrasound, bleeding or miscarriage concerns
Results follow-up Reviewing results or a finished cycle and planning next steps, including a negative pregnancy test
Annual gyn exam A routine yearly exam with no new problem worked up
Gyn problem Pain, bleeding, infection, cysts and other problems in a patient who is not pregnant

One question for Jev

Jev's questions are typed. A Choice question picks one option from a list, and each option has a description. The clinic's rules go into those descriptions, because the model sees nothing else: that a negative pregnancy test counts as a results visit, or that no retrieval, transfer or insemination happens at a monitoring visit.

from typesafe_sdk import AsyncTypeSafeClient, Choice

VISIT_TYPE = Choice(
    instructions="Which type of visit is this note from? Judge by what happened at this visit.",
    criteria={
        "cycle_monitoring": (
            "Monitoring during an active treatment cycle (IVF, IUI, egg freezing or frozen "
            "transfer prep): ultrasound or hormone bloodwork, dose changes, trigger timing. "
            "No retrieval, transfer or insemination is done at this visit."
        ),
        "fertility_procedure": (
            "An egg retrieval, embryo transfer or insemination (IUI) was performed at this visit."
        ),
        # ... five more visit types
    },
)

async with AsyncTypeSafeClient(model="jev-1.13.0") as client:
    response = await client.system_one(note, {"visit_type": VISIT_TYPE})
    answer = response.answers["visit_type"]  # .choice, .probabilities, .confidence

I pinned the model version to jev-1.13.0, so the answers do not change when TypeSafe updates jev-latest, and sent 20 requests at a time.

Accuracy, speed and cost

What Result
Correct 138 of 140 (98.6%)
Time for all 140 notes 2.2 seconds, 20 requests at a time
Time for one request 260 ms (median)
Input per note 648 tokens on average
Cost for all 140 notes $0.0038

Five of the seven types were all correct. Cycle monitoring and fertility procedure each had one mistake, 19 of 20.

The 98.6% is higher than real data would give, for reasons I come back to below. The speed and cost carry over better, because they depend on the length of the notes and not on how hard they are. Most of the 648 tokens are the question and its seven descriptions, which are sent with every note. At that length, 150,000 records would cost about $4 and take about 40 minutes at the speed above. Real notes are longer, and at 1,000 tokens per record the cost would be about $6.30.

The two mistakes

  1. An insemination read as monitoring, at 0.74. The note says: "IUI with frozen donor sperm, natural cycle, LH surge yesterday. Tolerated well." A nurse would read "tolerated well" as a procedure that just happened, but the note never says the IUI was done at this visit, and the monitoring description mentions IUI cycles. Jev gave cycle monitoring 0.78 and fertility procedure 0.22. It judged the note by what it says, and the note only implies the procedure.
  2. A progesterone check read as a results visit, at 0.67. The note says: "Ovulation check on letrozole cycle: CD21 progesterone 14.2, consistent with ovulation. Continue same dose next cycle." I had labeled it monitoring, since it is bloodwork during a treatment cycle. Jev read it as reviewing a result and planning the next cycle, which fits the results description too. This is less a model error than a rule the clinic has to make. Once the clinic decides where a day-21 progesterone check belongs, that rule goes into the description.

Using confidence to decide what a person checks

Every answer comes with a confidence from 0 to 1. Jev does not estimate it separately. TypeSafe's docs describe it as a comparison of the top probability with an even split across the options, and in my results it matches (top probability − 1/7) / (1 − 1/7) for every note, within rounding. With seven options, a confidence of 0.8 means a top probability of about 0.83. With a different number of options, the same confidence means a different top probability, so the threshold has to be chosen again whenever the list of visit types changes.

Here is what different thresholds would do, with every note below the threshold going to a person:

Threshold Labeled by Jev Correct among those Sent to a person Mistakes sent to a person
0.5 140 (100%) 98.6% 0 0 of 2
0.7 134 (96%) 99.3% 6 1 of 2
0.8 132 (94%) 100% 8 2 of 2
0.9 126 (90%) 100% 14 2 of 2

At 0.8, a person checks 8 of the 140 notes and catches both mistakes. Two mistakes are too few to measure an error rate, but they show the property I look for before trusting any threshold: the model is unsure on the cases it gets wrong. In an earlier test, my research agent's confidence label did not have that property, and its low-confidence claims were right as often as its high-confidence ones. That is why I check it first.

What the unsure notes say about the descriptions

The six correct notes below 0.8 are useful too, because they show where the descriptions leave room. Four of them are positive pregnancy blood tests (beta hCG) after an embryo transfer, for example:

Beta hCG 9 days post transfer: 287. Positive. Repeat in 48 hours.

Jev labeled all four as early pregnancy, but gave between 0.25 and 0.34 to results follow-up or cycle monitoring. A beta test is a lab result, and it comes right after a treatment cycle, so the note fits parts of three descriptions. The early pregnancy description could say directly that every positive beta hCG, including the repeat tests, belongs there.

The other two were a visit after a chemical pregnancy (0.53, split 0.6 to 0.4 between results follow-up and early pregnancy) and a follow-up after surgery for endometriosis (0.78, with 0.13 for gyn problem). Both were correct, and both sit between two types.

On real records, this is the loop: read the notes Jev was unsure about, find the rule that is missing from a description, write it in, and run the reviewed examples again.

What this does not show

  1. The notes and the descriptions were written together, so the notes use the words the descriptions use. Real notes have abbreviations, templates, copied text and typos, and some visits cover two types at once. Accuracy on real records will be lower, and it has to be measured on notes a clinician has labeled.
  2. Two mistakes in 140 notes are too few to estimate an error rate or to choose a threshold. On real data, the threshold should come from a larger set of reviewed notes, ideally a random sample and not only hand-picked hard cases.
  3. Every note fits one of the 7 types. TypeSafe's docs recommend an "other" option when the list may not cover every input, and real records will need one.
  4. I did not test whether the order of the options changes the answers, or how Jev handles long notes.
  5. The test ran once, on one model version.
  6. Before sending real patient notes to any API, check the vendor's data terms: HIPAA, a business associate agreement, and how long the data is kept. When I looked in October 2026, TypeSafe's public docs did not mention HIPAA.

What I would do on real records

  1. Run the clinic's reviewed examples first and read every mistake.
  2. Fix the descriptions where Jev goes wrong, and add an "other" type.
  3. Choose the review threshold with the clinic, from the reviewed examples, so only the unsure records go to a person.
  4. Run the full history in batches from the database, and write the visit type, the probabilities and the confidence back next to each record, so the threshold can change later without calling the API again.