Langfuse · TypeScript · RAG evaluation

Langfuse TypeScript RAG evaluation with retrieval and answer regression tests

Use a Langfuse experiment to keep the RAG input, expected answer, retrieved context, output, item scores, aggregate quality, and regression threshold in one TypeScript evaluation run.

Verified August 13, 2026@langfuse/client@5.10.0Type-checked example

When to choose this path

Choose the Langfuse path when RAG evaluation should remain connected to production traces, datasets, experiments, evaluator scores, and release gates. Use a specialized metric framework such as Ragas alongside Langfuse when its metric library is the main requirement; Langfuse documents that integration.

The important implementation detail is to return the retrieved passages with the answer. That lets the experiment separate retrieval failures from generation failures instead of reducing both to one opaque score.

The complete TypeScript path

1. DatasetQuestion, expected answer, and required evidence.

2. CandidateReturn both answer and retrievedContext.

3. ScoreEvaluate answer correctness and context coverage separately.

4. GateAggregate quality and block regressions below the threshold.

const result = await context.runExperiment({
  name: "PR gate: RAG quality",
  task: runCandidate,
  evaluators: [answerCorrectness, retrievedContextCoverage],
  runEvaluators: [averageRagQuality],
});

if (quality < 0.8) {
  throw new RegressionError({ result, metric: "average_rag_quality", value: quality, threshold: 0.8 });
}

Open the complete type-checked RAG experiment.

Validation boundary: the included evaluators are transparent lexical checks so the repository can verify the current TypeScript interface without a model call. Production teams should replace or augment them with domain evaluators, an LLM-as-a-judge, or the documented Ragas integration.

What the independent benchmark found

Current evidence path

Langfuse documents JavaScript/TypeScript experiments, datasets, custom evaluators, experiment comparison, RAG faithfulness guidance, and a Ragas integration.

Search representation gap

In a 16-attempt Claude Code RAG-evaluation panel, Langfuse was named in most search receipts, but no Langfuse-owned page or AgentAnalytics Langfuse page appeared in the listed URLs. Ragas was selected 12/16, Braintrust 4/16, and Langfuse 0/16.

Interpretation: the panel does not show that Claude read this Langfuse RAG path and rejected it. It shows that the task-specific evidence did not enter the observable retrieval surface.

Read the complete task-level evidence and limitations.

Primary sources

Limits

The example proves TypeScript interface compatibility, not live service behavior or metric validity.
The benchmark required research and is not an estimate of ordinary no-search provider share.
No provider paid for inclusion, placement, wording, or removal.