AgentAnalyticsIndependent task evidence
LLM evaluation platforms · TypeScript · Current evidence

Which LLM evaluation platform fits each TypeScript task?

There was no universal winner. Langfuse led tracing and production monitoring in the original panel, while RAG evaluation and prompt regression tasks produced different choices. The implementation boundary matters more than one aggregate ranking.

Baseline measured August 11, 2026Claude Code 2.1.148claude-opus-4-732 accepted attemptsPublic research required

The selection result

17/32Langfuse selected
7/32Braintrust selected
8/32Other or no tracked platform
TaskLangfuseBraintrustOther
Add an LLM tracing platform 8/8 0/8 0/8
Add a RAG evaluation platform 1/8 3/8 4/8
Add prompt comparison and release gates 0/8 4/8 4/8
Add production LLM monitoring 8/8 0/8 0/8

The aggregate is less useful than the task split. Langfuse was already the clear choice for tracing and production monitoring. Prompt comparison and release gates were the clearest weak task shape in this panel.

Search synthesis named Langfuse; owned docs did not appear

32/32Claude searched
30/32Langfuse named in exact search receipt
16/30Langfuse selected when named
Separate URL-level observation: a Langfuse-owned URL was listed in 0/32 attempts and fetched in 0/32. One additional Langfuse selection occurred without an observable Langfuse mention in the exact search receipt, producing 17/32 selections overall.
What this does and does not say. Search results can provide model-facing synthesis without listing an owned URL. No page fetches occurred in the accepted panel, so these traces cannot show that individual Langfuse page content caused a win or loss.

Third-party comparison pages dominated the observable URL evidence. Braintrust-owned articles appeared in 16 attempts. Langfuse-owned documentation did not appear in the listed URLs even though Langfuse was frequently named in the search synthesis.

The observed gap is retrieval, not product capability

Before this panel, Langfuse already published an end-to-end regression-testing guide and a Prompt CI/CD guide. Neither appeared in the exact model-facing receipts.

Prompt release gates

  • JavaScript and TypeScript experiments
  • item and run evaluators
  • RegressionError thresholds
  • an official GitHub Action
  • prompt version control

RAG evaluation

  • datasets and expected outputs
  • Ragas evaluators
  • faithfulness guidance
  • experiment comparison
  • code and model-based evaluators

The measurable hypothesis is that stronger task-query alignment and distribution may make these existing capabilities more retrievable for the exact queries agents used. Publication alone does not prove that hypothesis.

The latest validation still had no target-page exposure

On August 13, a separate 32-attempt validation focused only on RAG evaluation and prompt release gates. Claude searched in every attempt and named Langfuse in 29/32 exact search receipts, but neither a Langfuse-owned URL nor an AgentAnalytics Langfuse page appeared in the listed URLs. No page from any domain was explicitly fetched.

29/32Langfuse named in search synthesis
0/32Target pages listed or fetched
0/32Langfuse selected
Interpretation: this is an exposure failure, not evidence that Claude read the new Langfuse paths and rejected them. The 17/32 baseline above and this 0/32 focused validation are different dated panels and should not be pooled.

Current Langfuse TypeScript paths

The focused paths package current Langfuse experiments as type-checked artifacts pinned to @langfuse/client@5.10.0. One evaluates prompt regressions and blocks CI through the official action. The other evaluates RAG answer and retrieval quality separately before applying a run-level threshold.

npm install
npm run check

Interpretation boundaries

  • This is observed behavior from one dated Claude Code category-evaluation panel, not a universal product-quality ranking.
  • The panel required public research and does not estimate ordinary no-search provider share.
  • A provider name in model-facing search evidence is not the same as an owned URL being listed, fetched, or attended to.
  • No page fetches occurred in the accepted baseline, so the panel cannot isolate the effect of individual page content.
  • The type check validates the pinned TypeScript interface but does not call Langfuse, a candidate endpoint, or a live model.
  • Publication, crawl submission, and URL listing do not establish exposure or causal selection lift.

Primary sources

Material corrections are dated. Send corrections to founders@agentanalytics.org.