CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Mimi is an LLM-driven, multi-source RAG (Retrieval-Augmented Generation) assistant that aggregates knowledge from Telegram, Logseq, and GitHub. The system is model-agnostic, supporting OpenRouter, OpenAI, and Gemini via Genkit.

Tech Stack: Go 1.24, PostgreSQL (with pgvector), CozoDB (graph/semantic queries), Genkit (LLM orchestration), Telegram Bot API, GitHub API

Common Commands

Development workflow uses the Makefile extensively. The Makefile auto-populates .env from example.env, so manual .env management is typically unnecessary.

Setup & Development

• make install - Install all dev tools (sleek, geni, staticcheck, air, sqlc) and download CozoDB libs
• make dev-db - Start local PostgreSQL container with pgvector
• make migrate-up - Run database migrations (uses geni)
• make run - Run the application (includes vet check)
• make psql - Connect to local database

Code Quality

• make format - Format Go code and SQL files (uses sleek for SQL)
• make vet - Run go vet and staticcheck (includes format)
• make test - Run all tests (includes vet)

Database

• make sqlc - Generate Go code from SQL queries (outputs to internal/persist/)
• make migrate-down - Rollback last migration
• Both migration commands clean up auto-generated schema.sql files

Deployment

• make -C deploy production - Deploy to production

Architecture Overview

Multi-Service Architecture

The main application (main.go) orchestrates three concurrent services:

1. Telegram Bot (internal/bot/) - Context-aware chat with LLM routing
2. GitHub Scraper (internal/provider/github/scraper/) - Watches repos for events/issues/boards
3. Telegram Scraper (internal/provider/telegram/scraper/) - Ingests messages using Telegram Client API

All services share a PostgreSQL connection pool and communicate via the database.

LLM Agent System

The bot uses a router-based agent architecture (internal/bot/llm/):

• Router (prompts/router.prompt) - Genkit prompt that analyzes user queries and selects appropriate agent
• Agents (internal/bot/llm/agent/) - Implement Agent interface with GetInfo() and Run() methods
  • logseq - Queries Logseq graph via CozoDB Datalog
  • logseqquery - Uses custom regex-based Logseq query parser
  • github - Retrieves GitHub project data
  • telegram - Searches Telegram message history
  • summary - Generates periodic reports (daily/weekly/topic-based)
  • fallback - Handles general queries without RAG context

Agents can return text (DataText) or files (DataFile) as responses.

Data Flow & Storage

PostgreSQL (via pgx/sqlc):

• Chat history stored as JSONB (llm_chat.messages)
• Telegram message metadata (telegram_peer, telegram_topic, telegram_message)
• GitHub repository tracking (github_repository)
• All queries defined in sql/queries/*.sql, auto-generated to internal/persist/

CozoDB (graph database):

• Logseq knowledge graph with native Datalog queries
• Relations created in internal/provider/logseq/db/db.go
• Supports semantic/graph traversal queries

Embeddings: Stored in PostgreSQL with pgvector for RAG retrieval

Prompts & Genkit Integration

Prompts are defined in prompts/*.prompt files using Genkit's dotprompt format:

• YAML frontmatter defines input/output schemas
• Templates use {{variable}} syntax
• Loaded via genkit.LookupPrompt(g, "router")

Critical prompts:

• router.prompt - Agent selection
• *-retrieve.prompt - RAG document retrieval for each source
• *-eval.prompt - Evaluation/filtering logic
• fallback.prompt - General LLM responses

Logseq Integration

Mimi implements a custom Logseq parser and graph sync system:

• internal/provider/logseq/logseq.go - Regex-based markdown parser
• internal/provider/logseq/sync.go - Git-based synchronization (implements PushEventHook)
• internal/provider/logseq/query/ - Custom Datalog-like query language parser
• Triggered automatically on GitHub push events to configured repos

GitHub Integration

GitHub scraper acts as a webhook listener and polls for:

• Repository events
• Issues/PRs
• Project boards
• Synchronized to PostgreSQL for RAG queries
• Uses PushEventHook pattern for extensibility (e.g., triggering Logseq sync)

Telegram Integration

Dual Telegram integration:

1. Bot API (internal/bot/) - Responds to user messages
2. Client API (internal/provider/telegram/scraper/) - Scrapes configured channels/groups
   • Requires session file (SESSION_FILE env var)
   • Use cmd/telegram-session/main.go to create session interactively

Key Environment Variables

See example.env for complete list. Critical ones:

LLM Providers:

• OPENROUTER_API_KEY, OPENROUTER_API_URL
• OPENAI_API_KEY or GEMINI_API_KEY

Telegram:

• TELEGRAM_BOT_API_TOKEN - Bot API token
• APP_ID, APP_HASH, TG_PHONE - Telegram Client API credentials (from my.telegram.org)
• SESSION_FILE - Telegram session storage path

GitHub:

• GITHUB_TOKEN - Personal access token
• GITHUB_REPOSITORY_BASE_PATH - Local clone directory

Database:

• DATABASE_URL - PostgreSQL connection string (auto-constructed from DB_* vars)

Paths:

• LOGSEQ_GRAPH_PATH - Path to Logseq graph repository (typically ${GITHUB_REPOSITORY_BASE_PATH}/org/repo)

Development Notes

Database Schema Management

• Migrations in sql/migrations/ managed by geni
• Queries in sql/queries/*.sql use sqlc annotations
• Run make sqlc after modifying queries to regenerate Go code
• Generated code appears in internal/persist/ with .sql.go suffix

Adding New Agents

1. Create package in internal/bot/llm/agent/yourname/
2. Implement Agent interface with GetInfo() and Run()
3. Add prompt file in prompts/yourname*.prompt
4. Register in llm.New() (internal/bot/llm/llm.go)

Native Library Dependencies

• CozoDB requires native library (libcozo_c.a)
• Downloaded via pull_libs.sh during make install
• CGO_LDFLAGS must point to libs/ directory (set in Makefile)

Organization References

Hardcoded organization references exist:

• GitHub org: "cyberia-to" (previously "cyber-valley")
• Logseq repo hook: cyberia-to/cvland Update these in cmd/app/main.go and internal/bot/llm/llm.go for different deployments.

Testing

Scraper services can be tested independently:

• cmd/scraper/github/main.go
• cmd/scraper/logseq/main.go
• cmd/scraper/telegram/main.go

Use TELEGRAM_TEST_CHAT_ID env var for testing bot interactions.