Skip to content

TraceForge

TraceForge extracts insights from LLM traces to create fine-tuned models.

Code Video Cerebral Valley

Core trick

Receives OpenTelemetry traces over OTLP/HTTP from coding CLIs and scores them with an LLM judge to build fine-tuning datasets.

Tags

dev toolsagents

Description

TraceForge extracts insights from LLM traces to create fine-tuned models. It analyzes conversation patterns from Claude Code and Gemini CLI, generating custom models that incorporate your interaction data. - Captures OpenTelemetry traces via OTLP/HTTP and integrates with gemini cli and claude code - introspect and evaluate model interactions - Crafts system prompts incorporating conversation nuances, guiding models around core failure points identified in your traces - OpenAI-compatible endpoint for custom models on the spectrum of prompt to lora to full fine-tuning of models based on llm interaction metadata (traces, hard rules) - Browse and share quality traces, evaluating the quality of traces by utilizing llm-as-a-judge
natask/traceforge★ 00 forksTypeScriptCreated Nov 9, 2025 · last push Nov 9, 2025

README excerpt

# TraceForge

A modern platform for managing, organizing, and fine-tuning LLM conversation traces. Built with Next.js 15, TypeScript, Tailwind CSS, and shadcn/ui.

## Features

- **Authentication**: JSON-based auth with NextAuth.js (no database required)
- **Projects**: Organize your LLM traces into fine-tuning projects
- **Marketplace**: Browse and discover high-quality conversation traces
- **Linear-inspired UI**: Clean, modern interface inspired by Linear's design
- **Local JSON Storage**: All data stored in local JSON files for easy debugging

## Getting Started

### Prerequisites

- Node.js 20+
- npm

### Installation

1. Clone the repository
2. Install dependencies:
```bash
npm install
```

3. Seed demo data:
```bash
npm run seed
```

4. Start the development server:
```bash
npm run dev
```

5. Open [http://localhost:3000](http://localhost:3000) in your browser

### Demo Credentials

```
Email: demo@traceforge.com
Password: demo123
```

## Project Structure

```
traceforge/
├── app/                    # Next.js App Router pages
│   ├── api/               # API routes
│   ├── login/             # Login page
│   ├── signup/            # Signup page
│   ├── projects/          # Projects pages
│   ├── marketplace/       # Marketplace page
│   └── your-space/        # User space page
├── components/            # React components
│   ├── ui/               # shadcn/ui components
│   ├── navigation.tsx    # Top navigation
│   └── app-layout.tsx    # Layout wrapper
├── lib/

Team