openME
A Go-based RAG orchestrator for local LLMs, integrating SearXNG and Qdrant via MCP.
Core trick
A DAG planner splits queries into tasks for a small local model, which calls SearXNG search and Qdrant storage through MCP tools.
Description
A Go-based RAG orchestrator for local LLMs, integrating SearXNG and Qdrant via MCP.
README excerpt
# LLM Compiler: Low-Memory Agentic Framework An efficient, DAG-based agentic framework designed to run complex task plans using small, local LLMs via Ollama. It orchestrates external tools through the Model Context Protocol (MCP) to perform research, data processing, and vector storage. ## 🚀 Overview The **LLM Compiler** handles complex user queries by: 1. **Planning**: Decomposing the query into a Directed Acyclic Graph (DAG) of discrete tasks. 2. **Execution**: Running tasks in parallel where possible, respecting dependencies. 3. **Tooling**: Leveraging MCP servers for search (SearXNG) and storage (Qdrant), plus built-in tools for URL fetching and embeddings. 4. **Optimization**: Specifically tuned for ultra-small models like **Qwen 3.5 0.8B**, making it suitable for low-memory environments. ## 🏗️ Architecture - **Planner**: Compiles user queries into a JSON-based DAG. - **Executor/Worker Pool**: Managing the lifecycle of tasks, dependency resolution, and concurrent execution. - **MCP Client**: Dynamic integration with any MCP-compliant server. - **Repair Layer**: Intelligent post-processing of LLM-generated tool arguments to fix hallucinations or schema mismatches. - **Task Writer**: Real-time progress tracking in a human-readable `task.md` file. ## 🛠️ Components & Tools | Component | Description | | :--- | :--- | | **Search** | `searxng_search` via MCP | | **Storage** | `create_collection`, `upsert_points` via Qdrant MCP | | **Embeddings** | `embed` t