Koe is the first AI agent that sees your screen, understands context, and acts on your voice commands. Completely local. Zero cloud. Free.
Koe understands what you want and makes it happen. No menus. No clicking. Just speak.
The first voice assistant that actually sees what you see. Koe doesn't just transcribe -- it understands your screen and takes action.
Real-time OCR captures every element on your display -- text, buttons, error messages, form fields.
A local LLM processes your voice command alongside the screen context to determine the best action.
Types text, clicks buttons, controls system settings. All executed locally on your Mac.
Use your iPhone as a wireless voice controller, trackpad, and shortcut launcher for your Mac.
Speak into your iPhone from across the room. Text appears on your Mac instantly.
Koe analyzes your Mac screen and suggests next actions. Tap to execute.
Use your iPhone as a trackpad. Quick access to keyboard shortcuts like Cmd+Tab, Cmd+C/V.
4-digit PIN on first connect. Auto-reconnects on the same Wi-Fi network after that.
Sees your screen via OCR, understands context with LLM, and takes action -- clicks, types, controls. All locally.
Whisper + Metal GPU acceleration. Hybrid engine: Apple Speech for instant preview, Whisper for final accuracy.
100% local processing. Your voice, your screen, your data -- nothing ever leaves your Mac. Period.
Control your Mac from your iPhone. Voice input, trackpad, shortcuts, and AI-powered action suggestions.
AI analyzes your screen and suggests the next action. "Reply to email", "Fix this error", "Close this dialog".
"Volume up", "Lock screen", "Play music" -- 10+ natural language commands for macOS system control.
Say "Hey Koe" to activate hands-free. Custom wake word engine built with MFCC+DTW. Zero external deps.
Text appears the moment you start speaking. Apple Speech for instant preview, auto-replaced by Whisper's final result.
Auto-records speech with timestamps. Saves as a text file when done. Perfect for meeting minutes.
Start meeting mode with Option+Cmd+M. Koe captures Zoom/Teams audio, transcribes in real-time with speaker labels, and generates AI summaries with action items.
ScreenCaptureKit grabs remote participant audio. No bot needed, no meeting link sharing required.
Automatic speaker labeling with tinydiarize. Know who said what, without any setup.
Live transcription window with searchable text, speaker colors, and "mark important" voice commands.
6 templates: general, 1-on-1, standup, sales, brainstorm, interview. Auto-extracts action items to Apple Reminders.
Ask questions about your meeting. "What did we decide about the budget?" AI answers from the transcript.
Auto-post summaries to Slack. Create Notion pages. Auto-start recording when calendar meetings begin.
Side-by-side with the alternatives. Koe is the only voice tool that can see and act on your screen.
| Feature | Koe | macOS Dictation | Siri | ChatGPT Voice |
|---|---|---|---|---|
| Screen awareness | ✓ | ✕ | ✕ | ✕ |
| Clicks buttons | ✓ | ✕ | ✕ | ✕ |
| Local processing | ✓ | ✕ | ✕ | ✕ |
| Free | ✓ | ✓ | ✓ | ✕ |
| iPhone remote | ✓ | ✕ | ✕ | ✕ |
| Open source | ✓ | ✕ | ✕ | ✕ |
Download the PKG and double-click. One click install to /Applications. Allow microphone access.
Same Wi-Fi network. Koe auto-detects your Mac. Enter the 4-digit PIN once -- done.
Press Cmd+Opt+V or say "Hey Koe". Speak naturally. Koe understands and acts.
Every line of code is public. Fork it, modify it, use it commercially. Contributions welcome.
Free. Private. Open source. No account needed.