Historia-chanPython · local AI VTuber / chat companion · development since 2025
still in development
The second biggest thing I have built. Historia-chan connects a local llama.cpp chat model to a custom VRM avatar, voice in and out, screenshot vision and a long-term memory system. Nothing is uploaded anywhere: the model, the voice, the memory and the eyes all run on the user's machine.
- Chat — a local llama.cpp model answers through a PySide6 desktop app.
- Voice — microphone input transcribed with faster-whisper; replies spoken with Kokoro TTS.
- Eyes — a screenshot-vision pipeline: capture an area, and the multimodal model reads math problems, documents, code, UI captures, photos through one generic path.
- Web search — searches the internet when it needs information it doesn't know (Bing scraping by default, Tavily with an API key).
- Avatar — a custom “Historia” VRM model with 20+ animations and expressions; her face reacts to the mood of the conversation, and emotions are paired with animations.
- Memory — LanceDB long-term memory. A separate small memory model reads each new turn and keeps durable facts in categories: USER_FACT, USER_PREFERENCE, PROJECT, RELATIONSHIP, IMPORTANT_EVENT, GOAL, NOSTALGIA. Short-term memory summarizes every 20 messages; pressing Reset Chat folds the last session into the next one.
- Learning — planned: when a task is given (like playing a game with the user), Historia trains itself at it.
How it is built
- Python —
ui/PySide6 app + VTuber viewer,llm/llama.cpp and web tools,speech/whisper + Kokoro,memory/LanceDB databases,vision/screenshot pipeline,avatar/character definition files. - The main chat model and the memory model are intentionally separate roles — a strong model for natural conversation, a small deterministic one for memory extraction, so the memory work never blocks the chat UI.