from laminar import Laminar
import pandas as pd

# Initialize Laminar with your project
laminar = Laminar(project_name="my_ai_project")

# 1. Log LLM traces automatically
@laminar.trace()
def call_llm(prompt: str, model: str = "gpt-4"):
    # Your LLM call logic here
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

# 2. Run evaluations on your traces
eval_results = laminar.evaluate(
    traces=laminar.get_traces(),
    metrics=["accuracy", "latency", "cost"],
    evaluation_pipeline="my_eval_pipeline"
)

# 3. Create dataset from successful traces
good_traces = eval_results[eval_results["accuracy"] > 0.9]
dataset = laminar.create_dataset(
    name="high_quality_responses",
    data=good_traces,
    description="Traces with >90% accuracy"
)

# 4. Label data for fine-tuning
labeled_data = laminar.label(
    dataset=dataset,
    labeling_instructions="Label as helpful/unhelpful",
    num_labelers=3
)

print(f"Automated {len(laminar.get_traces())} traces, {len(dataset)} dataset entries, and {len(labeled_data)} labeled examples")

# Export for model training
training_data = labeled_data.to_pandas()
training_data.to_csv("training_data.csv", index=False)

For every hour spent fine-tuning a model or crafting a prompt, AI teams spend two more wrestling with logging, evaluation pipelines, and dataset management. This infrastructure tax is the silent killer of productivity in modern AI product engineering. Emerging from Y Combinator's S24 batch, Laminar (lmnr-ai/lmnr) is an open-source platform built on a radical premise: to win in AI, you must first master your data operations, not just your models.

The $100 Billion Infrastructure Tax

The journey from a prototype AI feature to a reliable, improving product is fraught with operational complexity. Developers typically stitch together a patchwork of tools: one for tracing LLM calls (like LangSmith or Phoenix), another for running evaluations (perhaps using OpenAI's Evals framework), a separate system for managing datasets (like Hugging Face Datasets), and often a manual process for collecting human feedback and labels. This fragmentation creates massive overhead. Context is lost between systems, data becomes siloed, and the feedback loop necessary for improvement—the vaunted "data flywheel"—grinds to a halt.

Laminar's founders identified this as the central bottleneck. "We kept seeing brilliant ML engineers and product teams stuck in infrastructure quicksand," explains a project spokesperson. "They had the vision for a self-improving AI application, but they were spending their cycles building plumbing, not product. The data flywheel remained a theoretical concept because the tools to build it were too disparate."

Laminar: The All-in-One Data Flywheel Engine

Laminar's solution is audaciously simple: a single, cohesive open-source platform that integrates the four core pillars of AI product operations.

1. Traces: The Complete System Narrative

Instead of just logging LLM inputs and outputs, Laminar captures full execution traces. This includes the chain of reasoning, tool calls, retrieved context from vector databases, code execution, and user interactions. It provides a holistic view of how your AI application behaves in the wild, turning black-box interactions into debuggable narratives.

2. Evals: Continuous, Automated Assessment

The platform allows teams to define and run evaluations directly against traced data. These can be model-graded (using another LLM as a judge), code-based, or involve human review. Crucially, evals are not a separate process; they are integrated into the trace lifecycle, enabling automatic scoring of production runs against key performance indicators.

3. Datasets: Curated Fuel for Improvement

Laminar automatically surfaces the most valuable data from traces. Failed runs, edge cases, and low-confidence outputs can be programmatically identified and exported as datasets. This transforms noisy production data into structured, high-signal training and testing material.

4. Labels: Closing the Human Feedback Loop

The platform includes tools for collecting human feedback—corrections, preferences, and ratings—directly on traced outputs. These labels are automatically associated with the original trace data, creating a rich, annotated corpus that can be used to fine-tune models, adjust prompts, or retrain classifiers.

The Integrated Flywheel in Motion

The magic is in the connection. Here’s how the flywheel spins: An AI customer support agent handles a conversation (Trace). An evaluation rule flags a response as unhelpful (Eval). This problematic interaction, with its full context, is automatically added to a "Needs Improvement" dataset (Dataset). A human reviewer corrects the agent's response within the platform (Label). This new gold-standard example is then used to fine-tune the underlying model or adjust the prompt chain. The improved agent is deployed, and its future traces are automatically evaluated against the same criteria, measuring the impact of the change.

This closed-loop system is what turns a static AI feature into a learning, adapting product. By unifying these steps, Laminar eliminates the friction that prevents most teams from ever getting their flywheel off the ground.

Open Source as a Strategic Advantage

Laminar's decision to be open-source (written in TypeScript) is critical. It addresses two major concerns for engineering teams: vendor lock-in and data sovereignty. Companies can self-host the entire platform, keeping their sensitive trace data, evaluation results, and proprietary datasets completely in-house. The architecture also allows for extensibility; teams can write custom evaluators, data exporters, and labeling workflows that fit their specific domain.

This positions Laminar not as a black-box SaaS service, but as foundational infrastructure—akin to what Kubernetes is for deployment or what PyTorch is for model development. Its rapid ascent on GitHub Trending, garnering nearly 2,500 stars shortly after its YC demo day, signals strong developer interest in this approach.

Implications for the AI Product Landscape

The rise of platforms like Laminar signifies a maturation in the AI industry. The initial wave focused on model access and basic orchestration (the "how to call an API" problem). The next wave, now underway, is about productization and operational excellence (the "how to build a reliable, improving product" problem).

For startups, this lowers the barrier to building robust AI features. They no longer need a dedicated ML ops team to establish a basic improvement cycle. For larger enterprises, it offers a standardized, auditable framework for managing dozens or hundreds of AI use cases across different teams.

It also shifts competitive advantage. When every company can access similar foundation models from OpenAI, Anthropic, or Google, the winner will be the one who can learn fastest from user interactions. The company with the tightest, fastest data flywheel will consistently have the better-tuned, more context-aware, and more reliable AI. Laminar aims to be the engine for that flywheel.

The Road Ahead: From Infrastructure to Autonomy

Laminar's current release tackles the foundational data problem. The logical next steps are clear: deeper automation. Future iterations could automatically suggest new evaluation criteria based on observed failure modes, recommend dataset splits for training, or even trigger retraining pipelines when performance drifts below a threshold.

The long-term vision is a platform where the AI product truly manages its own improvement cycle, with humans in the loop for high-stakes decisions and novel edge cases. This is the path from assisted intelligence to autonomous, self-optimizing systems.

The message for AI engineers and product leaders is stark: the era of hacking together your own observability and evaluation stack is ending. The strategic focus must shift from building infrastructure to leveraging it. Tools like Laminar won't build your AI product for you, but they will ensure that every line of code you write, every prompt you engineer, and every model you train is informed by real-world data and drives measurable improvement. The teams that embrace this integrated, data-centric approach will be the ones whose AI products don't just launch, but learn, adapt, and ultimately dominate.