Live demo

Try wasm-llm directly in your browser. Enter your API key and start a conversation. The entire runtime runs locally in WebAssembly — your key is sent directly to the provider, not to any intermediary.

Enter your API key above and click Initialize to begin.

WASM runtime not loaded

About

wasm-llm brings the llm.rb AI runtime to browsers and edge runtimes by compiling mruby-llm to WebAssembly via Emscripten. It exposes the same provider support, tools, MCP, A2A, and streaming capabilities through a JavaScript API — no server required.

Quick start

LLM.Context

The LLM.Context object is the core of the runtime. It is a low-level interface to a model that requires tool execution to be managed manually. Almost all other features build on top of it. Here a context is created with a DeepSeek provider and a prompt is sent to the model through ctx.talk:


const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET});
const ctx = new LLM.Context(llm, {
  model: "deepseek-chat",
  stream: process.stdout
});
await ctx.talk("Hello world");

LLM.Agent

The LLM.Agent object is implemented on top of LLM.Context. It provides the same interface but manages tool execution for you. It also has built-in features such as a loop guard that detects repeated tool call patterns and advises the model to change course rather than raise an error:


const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET});
const agent = new LLM.Agent(llm, {
  model: "deepseek-chat",
  stream: process.stdout
});
await agent.talk("Write one short sentence about WebAssembly.");

Tools

The LLM.Tool function creates tools that extend the abilities of a model. JavaScript tools are defined as host callbacks and run through the same tool loop as Ruby tools. Parameters can use a Zod schema directly. Host tools are synchronous — call() must return a plain JSON-like value:


import * as z from "zod";
export const Echo = LLM.Tool({
  name: "echo",
  description: "Echo text back to the model",
  parameters: z.object({
    text: z.string().describe("The text to echo")
  }),
  call({text}) {
    return {text};
  }
});

Supported Providers

OpenAI and OpenAI-compatible endpoints
Anthropic
Google Gemini
DeepSeek
xAI
Z.ai
Ollama
llama.cpp

Key Features

LLM.Context — Stateful execution environment with message history and streaming
LLM.Agent — Automatic tool-loop execution with loop guards
Tools — JavaScript host callbacks with Zod or JSON Schema parameters
MCP HTTP — Model Context Protocol client over HTTP for remote tools
A2A — Agent-to-Agent protocol client for remote skills
Streaming — Content and tool event callbacks
Serialization — Context persistence to JSON

Build Integration

Add to an mruby build config:

MRuby::CrossBuild.new("wasm-llm") do |conf|
  conf.toolchain :emscripten
  conf.gembox "stdlib"
  conf.gembox "stdlib-ext"
  conf.gem core: "mruby-compiler"
  conf.gem File.expand_path("/path/to/wasm-llm")
end

Dependencies

mruby-json
mruby-stringio
mruby-time
mruby-struct
mruby-regexp

wasm-llm