Letting Jev Choose the Related Articles

I wanted Jev to choose the related articles beneath a new piece when I published it. The useful part was the judgment: given what someone had just read, which of my other articles would continue that thought?

After seeing the introduction of Jev, I wanted to try that kind of decision in my own site. Related reading gave it a specific responsibility. The articles already existed, their links were known, and I could correct the selection whenever I disagreed.

Asking for a decision

Jev is TypeSafe's System One model. It returns typed decisions with probabilities, which makes it suitable for a feature where the application needs to select existing content. I use it through Vercel AI Gateway's evaluation API, with the model ID typesafe-ai/jev.

Its question types cover different judgments. Boolean, called Noul by TypeSafe, estimates the probability that a condition is true. Choice selects among named options and returns their probability distribution plus confidence. Score places an answer on an ordered rubric and returns the score, distribution, and confidence.

For related articles, Score lets each candidate stand on its own. Several articles can be relevant, and sometimes none should be selected. Each candidate receives a judgment against the same rubric:

Two pieces mentioning React can still serve very different readers. Someone reading about cache invalidation may benefit from an explanation of publishing webhooks. An unrelated animation article needs more than a shared framework to earn that place.

The model receives titles, excerpts, tags, and bounded passages from the articles. The question asks whether reading the candidate immediately afterward would be useful. Instructions also distinguish article text from commands, so a sentence inside an article cannot redefine the selection rule.

Confidence is separate from relevance

A confidently unrelated answer is still unrelated. That distinction is why the implementation checks both relevance and uncertainty.

A candidate must score at least 2, put at least 80% probability on the relevant levels, and have at least 80% confidence. The top 3 passing candidates are selected. If fewer qualify, the section stays shorter.

const passes =
  score >= 2 &&
  relevantProbability >= 0.8 &&
  confidence >= 0.8;

TypeSafe describes confidence as a measure derived from the distribution of its answers. A concentrated distribution means a more certain decision. It is not a guarantee that the decision is correct.

In the Gateway response, score and probabilities arrive in the answer, while the separate confidence value is in providerMetadata.typesafe.confidence, keyed by question ID. Missing confidence stops automatic selection.

An initial live check compared a cache-invalidation example with an article about publishing webhooks and an unrelated example about sourdough. The companion article scored 2.96 with 96% confidence and 99% probability on the relevant levels. The bread example scored 0.24, with only 9% probability on those levels. It was rejected.

The demo below uses those relevance values. Adjusting confidence is a local simulation that runs the same selection policy as the application. It does not call Jev.

Try the unrelated example at 100% confidence. It remains unselected because confidence cannot compensate for low relevance. Then bring the related example below 80% confidence to see the other condition take effect.

Making the judgment reusable

Jev runs when an eligible new article is published. A production build also handles new MDX articles. Its decisions are stored, so opening the page does not ask the model to repeat the comparison.

Candidates are evaluated in bounded groups, with each candidate appearing once. For the current archive, a new publication needs 4 requests. The model compares every eligible candidate; a keyword filter does not remove possible matches beforehand.

Repeated publication events reuse the existing job. Manual selections take priority over the stored automatic result. The next demo illustrates those rules locally, including what happens when a publication event arrives again.

I kept this limited to new articles. Older articles remain available as candidates, but enabling the feature does not rewrite their related sections. That gives the model a useful archive to work with while leaving existing editorial choices intact.

Where I still want control

The evidence sent to Jev is bounded. A connection explained only late in a long article can be missed. The rubric can also produce a defensible suggestion that does not match the continuation I intended for the reader. I can edit, reorder, or remove the saved links.

The implementation also has a daily request limit, a $1 monthly Gateway-key quota, and no automatic retries after an inference failure. Those limits make the scope of each decision explicit. A failed or uncertain selection can wait for editorial review.

The live example verifies that the response fields and threshold rule work together. It does not measure accuracy across all my writing. That will require reviewing actual suggestions over time, especially borderline pairs where the subjects overlap but the reader's intent differs.

For now, Jev has a small, inspectable job: compare the articles, report its uncertainty, and let the application select only the results that pass the rule.

No related articles appear below this piece because none of the eight candidates evaluated passed the relevance and 80% confidence requirements. I have left the section empty rather than added links that did not qualify.