rails-llm

About

rails-llm integrates the llm.rb AI runtime into Ruby on Rails with generators for quick setup and an engine that provides a stream-capable chat interface. Extend it with your own tools and skills.

Built on llm.rb's core runtime, rails-llm gives Rails applications access to multi-provider LLM support, MCP, A2A, structured outputs, context persistence, and all the tools and skills available in the llm.rb ecosystem.

Quick Start

bundle add rails-llm
rails generate rails_llm:install
rails db:migrate

Set your API key and visit http://localhost:3000/ai/agents.

Features

• Generators — Model, migration, initializer, and routes in one command
• Chat Engine — Stream-capable web interface at /ai/agents
• acts_as_agent — Add LLM capabilities to any ActiveRecord model
• File Attachments — Pass documents directly to supported models
• Streaming — Real-time response streaming to the browser
• Full llm.rb Access — All providers, tools, MCP, and A2A available

Example

class Agent < ApplicationRecord
  acts_as_agent provider: :set_provider, context: :set_context

  private

  def set_provider
    LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
  end

  def set_context
    {model: "deepseek-v4-flash"}
  end
end

agent = Agent.create!
agent.ask("What is the capital of France?").content
agent.ask("Summarize this", with: "report.pdf").content

Resources

• GitHub Repository
• RubyGems
• llm.rb Runtime