Robots you can talk to, drive and become. Carpathron connects a small ESP32 camera robot to Meta's Muse models, a Windows AI agent, WhatsApp (text + voice), the browser, and a native Meta Quest 3S avatar app. One brain (Muse), one open robot protocol, every way you want to reach your robot.

Built for the Meta Global AI Developer Hackathon 2026 — Startup track.

All files of this version: Release v1.7.0 (all releases: Releases) · alternative split installer (5 parts + Install.bat) in downloads/windows · install notes in downloads/README.md.

• New name: Muse Carpathron is now Carpathron Robotics & Coding by Qubitpage LIMITED. Updating keeps all chats, settings, servers and keys.
• Production security review (independent audit, all findings fixed and covered by 10 new tests):
  • phones and Quest connect only when you switch on network access (off by default); pairing PIN is valid 2 minutes, works once and is cancelled after 10 wrong tries;
  • every paired phone/headset gets its own revocable key (Settings → Phone & Quest → Paired devices); turning network access off disconnects them at once; settings that start programs or hold keys can only be changed on the PC;
  • the WhatsApp Cloud webhook requires Meta's signature; the Cloudflare tunnel exposes only the webhook; allowed numbers must match exactly; /mode from WhatsApp changes only that chat; the Quest app always asks on the PC;
  • API key, app key and WhatsApp tokens are encrypted at rest (DPAPI) like SSH and GitHub secrets; Quest tokens are stored as fingerprints;
  • Ask mode asks before reading credential folders (.ssh, browser profiles, the app's data), fetching URLs that could carry data out or reach local/cloud-metadata addresses, and taking desktop screenshots;
  • strict Content-Security-Policy on the app page; web pages in the agent's browser can't open local files; render windows use their own session;
  • the GitHub CLI download is verified against GitHub's checksums; tokens are kept out of command lines; safer PowerShell quoting; DevTools off in release builds; damaged settings are restored from the last good copy.
• Dependencies: ws 8.22.0 (security fixes); spreadsheets are read with exceljs (replaces xlsx 0.18.5, which has unfixed advisories on npm).
• Microsoft Store ready: MSIX package config, Store tiles, firewall rule in the package manifest, and a Windows build workflow (.github/workflows/desktop-windows.yml). The Store version uses the official WhatsApp Business Cloud API only.

• Computer use, tested on Windows: after opening an app it waits for the window and brings it to the front before typing; held Windows/Ctrl/Alt/Shift keys are released before typing and clicking (no accidental shortcuts); screenshots in the chat are small thumbnails (click to enlarge).

• Write while it works: the message box stays open while Muse works. ⚡ Now adds your message to the running job (it keeps every earlier task and finishes them all unless you say stop); ⏳ Queue runs it afterwards. Queued messages can be sent now, edited or removed. Works with both engines (Muse Code uses its native steer).
• Task list: multi-step jobs show a live task list above the message box; Muse keeps working until every task is done (stops only when blocked or making no progress).
• Runs until the job is done: in Auto and Full mode the step limit is only a checkpoint — no more "Reached the step limit (40)" in the middle of a build. A safety limit (1000 steps) and a no-progress check prevent runaway loops; Ask and Plan still pause.
• Fewer mistakes: honesty rules (no invented facts, files, outputs or test results; sources for current facts), a self-check pass before finishing (every claim verified with tools), automatic "change approach" when the same call keeps failing, test-before-delivery and clean-work rules, a per-chat scratch folder.
• Computer use: Muse sees the screen and controls mouse and keyboard on Windows (any app), plus a visible Muse Browser it drives element by element — to use desktop apps without an API, test the apps it builds and fix defects. An on-screen bar shows when it is in control, with a Stop button.
• Raspberry Pi and NVIDIA: dev environment inventory (IDEs, SDKs, GPU/CUDA, boards on USB and on the network), installs (mpremote, Thonny, Raspberry Pi Imager, CUDA…), open IDEs, MicroPython on Pico/ESP32, Pico UF2 flashing, Raspberry Pi SD-card imaging (always confirmed), Pi and Jetson over SSH. New skills /raspberry-pi, /nvidia-jetson, /test-app, /security-audit.
• Security checks: security_check for servers (ports, firewall, SSH, updates, fail2ban, Docker, exposed files) and websites (certificate, redirects, headers, version leaks, exposed .git / .env / backups), run automatically after public deployments.
• Other models from chat: ask_model gets a second opinion from another Muse model (e.g. Muse Wonder for hard reasoning or reviews).

• GitHub "Sign in with browser" fixed: newer GitHub CLI versions show the one-time code as "One-time code (XXXX-XXXX) copied to clipboard"; the app now reads both formats, shows the code, copies it and opens GitHub.
• Carpathron Avatar 1.1.5 (Quest): fixes a crash at start on headsets without a text-to-speech engine (Godot's voice list call); the microphone permission is asked only the first time you hold the talk button; the demo robot shows at once while the app looks for your real robot.
• Store art follows Meta's VRC.Quest.Asset.2: the title "Carpathron Avatar" is the most prominent text, no tagline (assets/store).

• GitHub sign-in with the browser, reworked: the app downloads the official GitHub CLI by itself (into its own folder — no admin rights, no winget, no extra window), shows the one-time code in the app (already copied) and opens the GitHub page. Authorize and it connects. A GH_TOKEN set on the PC no longer blocks the login.
• MCP servers that work out of the box: filesystem, memory (knowledge graph) and sequential-thinking ship with the app and run on its built-in runtime — no Node.js needed. They are set up automatically and work with both engines (Meta API key and Muse Code). Your own servers (npx, uvx, remote URL) work as before; errors now say clearly what is missing.

• 📖 Guide & help (bottom left): the complete documentation inside the app — install from scratch, connecting Meta (API key or Muse Code), every menu and button, all agent tools, remote servers and SSH keys, GitHub, MCP servers, Arduino, the robot, WhatsApp, phone and Quest, troubleshooting. Searchable, with a contents list. Same text in docs/GUIDE.md.
• No duplicate servers: a server is one entry per user + host + port. Adding it again (by you or by the agent) updates the saved entry instead of making a copy; duplicates saved by older versions are merged at start, keeping the credentials, and their chats move to the kept entry.

• Sign in to GitHub with the browser (Plugins → GitHub): uses the GitHub CLI web login when it is installed, or the GitHub device code page. No token to copy.
• Server chats are saved. A chat started on a remote server keeps that server, so you can come back another day and continue there. Chats on a server show 🌐 in the list.
• Terminal sessions are saved as chats (⌨ in the list). Open one later to read what was run, ask the agent to continue, or press Continue in terminal.

• Muse Code sessions now get the app tools (SSH, GitHub, PDF, research, Arduino, robot…) — tested on Windows with Muse Code 1.4.14, also in Full mode.
• Muse Code can no longer get stuck waiting for a question dialog the app cannot show.

• Remote page (left menu): all your servers with live status, terminal in the app, Test / Disconnect, Chat here (new chat working on that server), edit and delete.
• SSH keys: generate ed25519 keys (stored encrypted), import existing keys, copy the public key, export to .ssh, and Install key on a server — it adds the key to authorized_keys, switches the server to key login and can remove the saved password.
• Plugins page with GitHub: connect a token (encrypted), then the agent can create repositories, publish a folder (commit + push, author = you), clone, make releases with installer/APK uploads, and manage issues and pull requests. New skill /github-publish (checks for secrets and a .gitignore before pushing).
• New project and other dialogs work in the desktop app (they used the browser prompt, which Electron does not support).

• Muse Code commands work on Windows: Muse Code's app host reported the Windows sandbox as unavailable even after muse sandbox windows setup, so every command failed. The app now detects this and runs Muse Code without the sandbox (Settings → Tools & models → Muse Code Windows sandbox: Automatic / Always / Off); approvals still apply. Set up sandbox now runs the real muse sandbox windows setup (the old button failed with "Windows cannot find 'sandbox'").
• Settings page no longer shows duplicated sections.

• Remote servers over SSH: save a VPS or dedicated server once (password or key, encrypted with your Windows account; host key checked). Switch any chat to a server with 🖥 This PC → my-vps: files and commands then run on the server. Tools: ssh_exec, interactive ssh_shell (sudo prompts, REPLs, tail -f), ssh_upload / ssh_download, background jobs on the server.
• Full access mode: Ask · Auto · Full · Plan. Full runs everything without approvals and picks Muse Code's full-access permission profile when it offers one (no sandbox).
• Documents: reads PDF (by page), Word, Excel (all sheets + formulas), PowerPoint. Designed PDFs from HTML/CSS or Markdown with the built-in Chromium (create_pdf), page screenshots at desktop and phone width so the agent can check and fix websites and layouts it builds (screenshot_page, view_image).
• Web search + deep research: Meta search grounding with sources (DuckDuckGo fallback); deep_research plans searches, reads the sources and writes a cited Markdown report.
• Firmware like a developer: arduino_setup (finds Arduino IDE 2's arduino-cli or downloads it, installs ESP32/AVR/RP2040 cores and libraries), arduino_boards, arduino_compile (errors with file:line), arduino_upload, serial_monitor, open_arduino_ide.
• Every Muse model, used for what it can do: the model menu shows each model's capabilities (images, video, audio, PDF, tools, reasoning, web search). Muse Wonder and reasoning effort (minimal … max). Pick Muse Image to draw or edit pictures, SAM 3.1 to find objects (boxes + overlay, works on robot photos), Muse Voice Transcribe for audio files. Chat models can call them as tools too (generate_image, segment_image, transcribe_audio). Attached images go to vision models as images.
• Muse Code gets the same tools: an MCP bridge gives Meta-account sessions the app's SSH, PDF, research, Arduino, image, SAM and robot tools.
• Stop works every time: Stop (or Esc) ends the turn within seconds even if the model or a command hangs; long commands are killed.
• New skills: /design-website, /design-pdf, /research, /firmware, /server-setup, /carpathron-build (the whole robot → app → Avatar → server process).
• 57 backend tests (35 + 22 new, incl. a real SSH/SFTP test server and the MCP bridge) + Electron tests for PDF, screenshots and overlays.

              ┌──────────────── Meta Model API ────────────────┐
              │  Muse Spark (chat · tools · vision)            │
              │  Muse Voice Transcribe (speech → text)          │
              └───────────────▲────────────────────────────────┘
                              │ HTTPS
 WhatsApp (text/voice) ──► ┌──┴──────────────────────────┐ ◄── Browser / phone / Quest browser (pairing PIN)
                           │  Muse Carpathron (Windows)   │
 Carpathron Avatar ──────► │  agent · memory · tools · MCP│ ◄── Muse Code (Meta account)
 (Quest, linked once)      └──┬───────────────────────────┘
                              │ Carpathron Robot Protocol (HTTP + MJPEG, LAN)
                       ┌──────▼─────────┐ ◄── Carpathron Avatar can also drive it directly
                       │ Baby Carpathron │
                       │ ESP32-CAM car   │
                       └─────────────────┘

• Talk to your robot anywhere: "drive forward 1 metre and take a photo" in the app, on WhatsApp, or by voice in the headset.
• Be the robot: in the Quest app the robot's camera fills your view; turn your head and the car turns with you.
• Robot head: the camera is fixed, so "look left / behind" turns the car's body, takes the photo and turns back.
• Room mapping: the car turns a full circle, Muse Spark names what it sees and how far, and you get a top-down map (also on WhatsApp and saved to memory).
• Real agent: full PowerShell + file tools, SSH servers, documents and PDFs, web research, Arduino firmware, MCP servers, parallel sub-agents, approvals (Ask / Auto / Full / Plan), live activity showing what ran locally vs in the cloud.
• Memory that grows: BM25 RAG over chats, projects with knowledge files, auto-learned memories and skills.
• Production details: single standalone Windows installer (Win 10/11/Server), safety auto-stop on the robot, token-protected LAN access, PIN pairing, 57 backend tests + Electron/UI tests + a headless end-to-end Quest test.

1. Car: assemble the kit, flash firmware/carpathron_car with Arduino IDE (esp32 core 2.0.12), put it on your Wi-Fi (CAR-SETUP).
2. PC: install Carpathron Robotics & Coding (Carpathron-Robotics-Coding-Setup.exe, or build: cd apps/desktop && npm ci && npm run dist). Add your Meta Model API key or sign in with Muse Code. Robot page → Find my car.
3. Phone: Settings → WhatsApp → scan the QR. Send /help or a voice note.
4. Quest 3S: sideload CarpathronAvatar-1.1.5.apk (Developer Mode + Meta Quest Developer Hub), open it from Library → Unknown Sources, click Allow on the PC once.

The full walk-through is in the A-to-Z guide.

# Desktop app (Node 20+)
cd apps/desktop && npm ci
npm test                      # 82 tests: end-to-end backend (fake Meta API, fake car, fake MCP) + security
node test/ui-eyes.js          # simulated Quest 3S in browser VR (Playwright)
npm run dist                  # Windows installer (on Linux needs wine)
npm run dist:store            # Microsoft Store package (MSIX/appx; Windows only — or run the "Desktop app (Windows)" GitHub workflow)

# Quest app (Godot 4.4.1 + Android templates, JDK 17, apktool 2.11, zipalign, apksigner, Khronos OpenXR loader AAR)
cd apps/quest-avatar
DESKTOP=../desktop GODOT=godot node test_e2e.js      # headless end-to-end: robot + Muse link + voice
KS=my.keystore KS_PASS=... KS_ALIAS=... ./build_quest.sh

# Car firmware: open firmware/carpathron_car/carpathron_car.ino in Arduino IDE (board: AI Thinker ESP32-CAM)

apps/desktop/        Electron app: backend (agent, engines, tools, robot, mapper, voice, WhatsApp) + UI + tests
apps/quest-avatar/   Godot 4 OpenXR app: robot_link.gd (protocol + MJPEG), muse_link.gd (Muse via PC), main.gd (VR)
firmware/            ESP32-CAM car firmware + assembly / Wi-Fi setup notes
protocol/            Carpathron Robot Protocol v1
docs/                A-to-Z guide
assets/brand/        Logo, icons, social preview
tools/               Release helpers

MIT — see LICENSE.