FEATURE GUIDE · LOCAL MCP
Free Local AI Models and Privacy Controls
Run the default semantic embedding baseline on your CPU, optionally request a supported GPU backend, or connect an OpenAI-compatible local or remote endpoint.
There is no account, central inference service, or product telemetry. The default model file downloads once from Hugging Face, then the Transformers.js embedding call runs on the local CPU. Set AGENT_DECISION_DEVICE to request an installed backend such as DirectML on Windows, CUDA on Linux x64, or CoreML on macOS. The first cold call is slower than a cached warm call, and a GPU backend can be slower for this small model.
HOW IT WORKS
From current state to a bounded action
- Use the local semantic provider without adding API credentials; it defaults to CPU.
- If the ONNX Runtime build and drivers support it, set AGENT_DECISION_DEVICE=dml on Windows, cuda on Linux x64, coreml on macOS, or auto to request a platform-supported device.
- Benchmark the same task set on your machine before changing the default. One 30-example Ryzen 5 5600H / GeForce GTX 1650 run had warm p95 of 17 ms on CPU and 132 ms with DirectML requested; these small, non-held-out results are not a speed guarantee.
- For an existing Ollama server, set AGENT_DECISION_PROVIDER=openai-compatible and leave the default base URL on loopback. Loopback OpenAI-compatible endpoints may receive browser context without the remote-sharing opt-in; hosted endpoints are blocked by default.
- For a remote OpenAI-compatible API, set its URL and key explicitly; request data will leave your machine if you enable browser context sharing.
- Jev is a separate optional TypeSafe adapter and can incur that provider's charges.
LIMITS TO KEEP IN VIEW
Weights are third-party model artifacts governed by their own model license. The repository license does not replace their terms. A requested ONNX device does not prove which provider ran every node, and unsupported operations may fall back to CPU. Remote provider privacy and billing are controlled by that provider; hosted browser page context is blocked unless you explicitly opt in.