#local-llm

InferBench's MCP server lets coding agents run, serve and benchmark local LLMs (text + image, llama.cpp + Stable Diffusion) on your own hardware on demand. Measures real tokens/sec, picks the optimal quant for your GPU, and exposes a 124-model catalog. Local-first, no cloud required.

Check whether a task can run on a local model instead of cloud before every inference call. Saves money on every call that does not need cloud.

CP Log Analyzer automatically triages your log files using a local AI model. It scans for errors, deduplicates repeated events, analyses root causes using Ollama (deepseek-r1:14b), and files structured GitHub Issues — all running on your own machine with zero data sent to external servers.

A powerful, intelligent shell assistant powered by local AI models through Ollama. Shell AI brings advanced AI capabilities directly to your terminal, enabling natural language interaction with your file system, shell commands, and development workflow.

Spring AI Playground is a self-hosted web UI that simplifies AI experimentation and testing. It provides Java developers with an intuitive interface for working with large language models (LLMs), vector databases, prompt engineering, and Model Context Protocol (MCP) integrations. Built on Spring AI, it supports leading model providers and includes comprehensive tools for testing retrieval-augmented generation (RAG) workflows and MCP integrations. The goal is to make AI more accessible to developers, helping them quickly prototype Spring AI-based applications with enhanced contextual awareness and external tool capabilities.

Minimalist examples to provide your own MCP servers to your local llm models.

Lightweight MCP client that uses a local Ollama LLM to query multiple MCP servers defined in config.json

Self-hosted MCP Japanese text indexing & search—chunking+embeddings with BM25×vector rerank

API service to search vectorized Cursor IDE chat history using LanceDB and Ollama

A Python-based client for interacting with Model Context Protocol (MCP) servers using Ollama. Features include multi-server support, dynamic model switching, tool management, and a rich terminal interface.

a magical desktop app that puts the power of LLMs and MCP in the hands of everyone

This Streamlit application provides a user interface for connecting to MCP (Model Context Protocol) servers and interacting with them using different LLM providers (OpenAI, Anthropic, Google...).

A template repository that includes a dev container for running a local LLM and included knowledge base. Add a git repo using `git subtree` or `git submodule` and update it using an MCP Client/Server relationship i.e. `VS Code` extension like `Cline` and the `Filesystem`/`Obsidian-MCP` MCP server.