Category
Category : Testing & Debugging Code Training LLMs is expensive.
Core trick
Parses a Python training file to derive transformer dimensions, then plots per-op FLOPs and bytes on a live roofline chart.
Description
Category : Testing & Debugging Code
Training LLMs is expensive. Burning hours on an H100 only to discover your model was memory-bound the entire time is an expensive mistake and it happens constantly. Live Roofline Analyzer fixes that.
It's a VS Code extension that shows you exactly what's bottlenecking your transformer — compute or memory — in real time, right inside your editor. Open any Python training file, and it auto-detects your model architecture, plots every transformer operation on a live roofline chart, and tells you precisely which ops are wasting your GPU. Drop one Python file next to your training script and the panel updates live as training runs, showing step count, tokens per second, and a continuously updated roofline.
No more guessing. No more profiling after the fact. No more wasted compute.
Supports TPU v4/v5e/v5p, A100, H100, and H200 out of the box, with one-click presets for LLaMA 3 8B/70B/405B, Mistral, Gemma, and GPT-2. Hit one button and GPT-4o streams back concrete optimization suggestions — batch size, precision, parallelism strategy — based on your actual numbers.
README excerpt
# Live Roofline Analyzer A VS Code extension for real-time roofline analysis of transformer models on TPUs and GPUs, with OpenAI Codex-powered optimization suggestions. ## What It Does - Visualizes the **roofline model** for your transformer — which ops are memory-bound vs compute-bound - Breaks down per-layer FLOP and memory costs (QKV, attention, FFN, layer norm) - Detects model config by parsing your open Python file - Streams optimization suggestions from OpenAI Codex (gpt-4o) ## Prerequisites - [Node.js](https://nodejs.org/) v18+ - [VS Code](https://code.visualstudio.com/) v1.85+ - An [OpenAI API key](https://platform.openai.com/api-keys) (optional — only needed for Codex analysis) ## Setup ```bash # 1. Install dependencies npm install # 2. Compile TypeScript npm run compile ``` ## Running the Extension (Development) 1. Open the project folder in VS Code 2. Press **F5** (or run **Run → Start Debugging**) 3. A new VS Code window opens with the extension loaded Inside the new window: | Action | How | |---|---| | Open the roofline panel | `Cmd/Ctrl+Shift+P` → **Roofline: Open Live Roofline Analyzer** | | Analyze a Python file | Open a `.py` file → click the graph icon in the editor title bar, or run **Roofline: Analyze Current File for Model Config** | ## Configuration Set your OpenAI API key in VS Code settings (`rooflineAnalyzer.openaiApiKey`) or via the environment variable: ```bash export OPENAI_API_KEY=sk-proj-... ``` You can also change the default har