Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 36 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,50 @@ A native macOS push-to-talk dictation app. Hold a key, speak, release — your
speech is transcribed and (optionally) cleaned up by ChatGPT, then pasted at your
cursor in any app.

Unlike [ZeroType](https://github.com/nick1ee/ZeroType) (which needs a paid API
key), LizardType uses **your existing ChatGPT web session** (`cookies.json`) via
a hidden, logged-in `WKWebView` — so transcription and cleanup run on your
ChatGPT subscription. No API key, no extra cost.
LizardType supports two interchangeable **API providers** (pick one in Settings):

- **ChatGPT Web** (default) — uses **your existing ChatGPT web session**
(`cookies.json`) via a hidden, logged-in `WKWebView`, so transcription and
cleanup run on your ChatGPT subscription. No API key, no extra cost. Unlike
[ZeroType](https://github.com/nick1ee/ZeroType), which needs a paid API key.
- **Groq API** — uses [Groq](https://console.groq.com/docs/quickstart)'s
OpenAI-compatible endpoints (Whisper for transcription, Llama/etc. for
cleanup). Fast and key-driven; no ChatGPT session needed. Just paste your
`GROQ_API_KEY`.

## How it works

```
hold trigger key ─▶ record m4a ─▶ WKWebView (logged in via cookies.json)
├─ POST /backend-api/transcribe → raw text
└─ POST /backend-api/conversation → cleaned text
┌─ ChatGPT Web (cookies.json) ─ WKWebView
hold trigger key ─▶ record m4a ─▶─┤ ├─ POST /backend-api/transcribe → raw text
│ └─ POST /backend-api/conversation → cleaned text
└─ Groq API (GROQ_API_KEY) ─ URLSession
├─ POST /openai/v1/audio/transcriptions → raw text
└─ POST /openai/v1/chat/completions → cleaned text
paste at cursor (⌘V) ◀────────┘
```

The WebView runs real WebKit on your Mac, so it passes Cloudflare like Safari and
inherits your session. `/api/auth/session` provides the bearer token the
`/backend-api/*` calls need.
In **ChatGPT Web** mode the WebView runs real WebKit on your Mac, so it passes
Cloudflare like Safari and inherits your session; `/api/auth/session` provides
the bearer token the `/backend-api/*` calls need.

### Using the Groq provider

1. Get an API key at [console.groq.com](https://console.groq.com).
2. **Settings → General → Provider → Groq API**, then paste the key (stored in
your macOS Keychain). Alternatively, leave it blank and set `GROQ_API_KEY` in
a `.env` file (current directory or `$HOME/.env`) — handy during `make run`.
3. Optionally tweak the transcribe model (`whisper-large-v3-turbo`) and cleanup
model (`llama-3.3-70b-versatile`).

Verify the whole pipeline from the terminal:

```bash
GROQ_API_KEY=gsk_… build/LizardType.app/Contents/MacOS/LizardType \
--selftest --groq /path/to/clip.m4a
```

## Download

Expand Down
4 changes: 2 additions & 2 deletions Resources/Info.plist
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
<key>CFBundleIdentifier</key>
<string>com.lizardtype.app</string>
<key>CFBundleVersion</key>
<string>1</string>
<string>2</string>
<key>CFBundleShortVersionString</key>
<string>0.1.0</string>
<string>0.2.0</string>
<key>CFBundleExecutable</key>
<string>LizardType</string>
<key>CFBundleIconFile</key>
Expand Down
54 changes: 37 additions & 17 deletions Sources/AppState.swift
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ final class AppState: ObservableObject {
let settings = AppSettings.shared
let recorder = AudioRecorder()
private let bridge = ChatGPTBridge()
private let groq = GroqClient()
private let hotkey = HotkeyManager()
private let overlay = OverlayController()
private var pipeline: Task<Void, Never>?
Expand Down Expand Up @@ -137,21 +138,38 @@ final class AppState: ObservableObject {
}
}

/// The backend selected in Settings.
private var activeProvider: SpeechProvider {
settings.provider == .groq ? groq : bridge
}

func warmBridge() async {
status = .warming
guard !settings.cookiesPath.isEmpty else {
status = .error("Set cookies.json in Settings"); return
}
do {
NSLog("[LizardType] warming bridge with %@", settings.cookiesPath)
try await bridge.start(cookiesPath: settings.cookiesPath)
await bridge.waitUntilReady()
_ = try await bridge.accessToken(forceRefresh: true) // verify login
status = .ready
NSLog("[LizardType] bridge ready — logged in")
} catch {
status = .error(error.localizedDescription)
NSLog("[LizardType] warm failed: %@", error.localizedDescription)
switch settings.provider {
case .groq:
do {
try await groq.validate() // ensure a key is resolvable
status = .ready
NSLog("[LizardType] Groq provider ready")
} catch {
status = .error(error.localizedDescription)
NSLog("[LizardType] Groq warm failed: %@", error.localizedDescription)
}
case .chatgpt:
guard !settings.cookiesPath.isEmpty else {
status = .error("Set cookies.json in Settings"); return
}
do {
NSLog("[LizardType] warming bridge with %@", settings.cookiesPath)
try await bridge.start(cookiesPath: settings.cookiesPath)
await bridge.waitUntilReady()
_ = try await bridge.accessToken(forceRefresh: true) // verify login
status = .ready
NSLog("[LizardType] bridge ready — logged in")
} catch {
status = .error(error.localizedDescription)
NSLog("[LizardType] warm failed: %@", error.localizedDescription)
}
}
}

Expand Down Expand Up @@ -265,20 +283,22 @@ final class AppState: ObservableObject {
busy = true
defer { busy = false; recorder.cleanup(url) }
do {
await bridge.waitUntilReady()
let provider = activeProvider
if settings.provider == .chatgpt { await bridge.waitUntilReady() }
status = .transcribing
overlay.show(.transcribing)
let raw = try await bridge.transcribe(audioURL: url, language: settings.transcribeLanguage)
let raw = try await provider.transcribe(audioURL: url, language: settings.transcribeLanguage)
guard !raw.isEmpty else { status = .ready; overlay.hide(); return }
lastTranscript = raw

var final = raw
if settings.cleanupEnabled {
status = .cleaning
overlay.show(.cleaning)
let cleanupModel = settings.provider == .groq ? settings.groqCleanupModel : settings.model
do {
final = try await bridge.cleanup(raw: raw, prompt: settings.cleanupPrompt,
model: settings.model, language: settings.oaiLanguage)
final = try await provider.cleanup(raw: raw, prompt: settings.cleanupPrompt,
model: cleanupModel, language: settings.oaiLanguage)
} catch {
// Cleanup failed (e.g. sentinel) — fall back to raw so text is never lost.
final = raw
Expand Down
125 changes: 125 additions & 0 deletions Sources/Bridge/GroqClient.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
import Foundation

/// Groq backend (OpenAI-compatible REST). Key-driven, no WebView.
/// transcribe → POST /openai/v1/audio/transcriptions (whisper-large-v3-turbo)
/// cleanup → POST /openai/v1/chat/completions (llama-3.3-70b-versatile)
/// Auth: `Authorization: Bearer <GROQ_API_KEY>` (see GroqSecrets).
@MainActor
final class GroqClient: SpeechProvider {

enum GroqError: LocalizedError {
case noKey
case http(Int, String)
case badResponse(String)
var errorDescription: String? {
switch self {
case .noKey: return "No Groq API key — paste one in Settings or set GROQ_API_KEY in .env"
case .http(let c, let b): return "Groq HTTP \(c): \(b)"
case .badResponse(let s): return "Unexpected Groq response: \(s)"
}
}
}

private let base = URL(string: "https://api.groq.com/openai/v1")!
private let session: URLSession

init(session: URLSession = .shared) { self.session = session }

private func key() throws -> String {
guard let k = GroqSecrets.apiKey() else { throw GroqError.noKey }
return k
}

/// Cheap auth/connectivity check used at warm-up. Throws on a missing key or
/// an auth failure; tolerates other transient errors (returns without throwing).
func validate() async throws {
_ = try key() // surface missing-key immediately
}

// MARK: - Transcribe

func transcribe(audioURL: URL, language: String) async throws -> String {
let apiKey = try key()
let model = AppSettings.shared.groqTranscribeModel
let fileData = try Data(contentsOf: audioURL)
let filename = audioURL.lastPathComponent
let mime = audioURL.pathExtension == "wav" ? "audio/wav" : "audio/mp4"

let boundary = "LizardType-\(UUID().uuidString)"
var body = Data()
func field(_ name: String, _ value: String) {
body.append("--\(boundary)\r\n")
body.append("Content-Disposition: form-data; name=\"\(name)\"\r\n\r\n")
body.append("\(value)\r\n")
}
field("model", model)
if !language.isEmpty { field("language", language) }
field("response_format", "json")
body.append("--\(boundary)\r\n")
body.append("Content-Disposition: form-data; name=\"file\"; filename=\"\(filename)\"\r\n")
body.append("Content-Type: \(mime)\r\n\r\n")
body.append(fileData)
body.append("\r\n--\(boundary)--\r\n")

var req = URLRequest(url: base.appendingPathComponent("audio/transcriptions"))
req.httpMethod = "POST"
req.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
req.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
req.httpBody = body

let obj = try await sendJSON(req)
guard let text = obj["text"] as? String else {
throw GroqError.badResponse(String(describing: obj))
}
return text.trimmingCharacters(in: .whitespacesAndNewlines)
}

// MARK: - Cleanup

func cleanup(raw: String, prompt: String, model: String, language: String) async throws -> String {
let apiKey = try key()
let message = Prompts.cleanupMessage(prompt: prompt, raw: raw)
let payload: [String: Any] = [
"model": model,
"temperature": 0.2,
"messages": [["role": "user", "content": message]],
]
var req = URLRequest(url: base.appendingPathComponent("chat/completions"))
req.httpMethod = "POST"
req.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
req.setValue("application/json", forHTTPHeaderField: "Content-Type")
req.httpBody = try JSONSerialization.data(withJSONObject: payload)

let obj = try await sendJSON(req)
guard let choices = obj["choices"] as? [[String: Any]],
let first = choices.first,
let msg = first["message"] as? [String: Any],
let content = msg["content"] as? String else {
throw GroqError.badResponse(String(describing: obj))
}
let text = content.trimmingCharacters(in: .whitespacesAndNewlines)
guard !text.isEmpty else { throw GroqError.badResponse("empty cleanup result") }
return text
}

// MARK: - HTTP helper

private func sendJSON(_ req: URLRequest) async throws -> [String: Any] {
let (data, resp) = try await session.data(for: req)
let status = (resp as? HTTPURLResponse)?.statusCode ?? 0
guard status == 200 else {
let snippet = String(data: data.prefix(800), encoding: .utf8) ?? ""
throw GroqError.http(status, snippet)
}
guard let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else {
throw GroqError.badResponse(String(data: data.prefix(800), encoding: .utf8) ?? "")
}
return obj
}
}

private extension Data {
mutating func append(_ string: String) {
if let d = string.data(using: .utf8) { append(d) }
}
}
97 changes: 97 additions & 0 deletions Sources/Bridge/GroqSecrets.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import Foundation
import Security

/// Resolves and persists the Groq API key.
///
/// Lookup order (first hit wins):
/// 1. Keychain — written from the Settings SecureField (survives reinstalls,
/// works for shipped `.dmg` users).
/// 2. `GROQ_API_KEY` process environment variable.
/// 3. `.env` file (`KEY=VALUE`) in the current working directory, then `$HOME/.env`.
/// Accepts `GROQ_API_KEY` or `GROQ_API`. Convenient for `make run` in dev.
enum GroqSecrets {
private static let service = "com.lizardtype.groq"
private static let account = "api-key"

/// The effective key from any source, or nil if none is available.
static func apiKey() -> String? {
if let k = keychainKey(), !k.isEmpty { return k }
if let k = environmentKey(), !k.isEmpty { return k }
return nil
}

/// True if a key exists outside the Keychain (env / .env) — used by the UI to
/// tell the user a key was auto-detected even though the field is empty.
static func hasEnvKey() -> Bool {
(environmentKey()?.isEmpty == false)
}

/// The key currently stored in the Keychain (the editable Settings value).
static func keychainKey() -> String? {
let query: [String: Any] = [
kSecClass as String: kSecClassGenericPassword,
kSecAttrService as String: service,
kSecAttrAccount as String: account,
kSecReturnData as String: true,
kSecMatchLimit as String: kSecMatchLimitOne,
]
var item: CFTypeRef?
guard SecItemCopyMatching(query as CFDictionary, &item) == errSecSuccess,
let data = item as? Data,
let s = String(data: data, encoding: .utf8) else { return nil }
return s
}

/// Write (or clear, when empty) the key in the Keychain.
static func setKeychainKey(_ key: String) {
let base: [String: Any] = [
kSecClass as String: kSecClassGenericPassword,
kSecAttrService as String: service,
kSecAttrAccount as String: account,
]
SecItemDelete(base as CFDictionary)
let trimmed = key.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty, let data = trimmed.data(using: .utf8) else { return }
var add = base
add[kSecValueData as String] = data
add[kSecAttrAccessible as String] = kSecAttrAccessibleAfterFirstUnlock
SecItemAdd(add as CFDictionary, nil)
}

// MARK: - Environment / .env

private static func environmentKey() -> String? {
let env = ProcessInfo.processInfo.environment
if let k = env["GROQ_API_KEY"], !k.isEmpty { return k }
if let k = env["GROQ_API"], !k.isEmpty { return k }
let fm = FileManager.default
let candidates = [
fm.currentDirectoryPath + "/.env",
(fm.homeDirectoryForCurrentUser.path) + "/.env",
]
for path in candidates {
if let k = parseEnvFile(path) { return k }
}
return nil
}

/// Minimal `.env` parser: returns the value of GROQ_API_KEY / GROQ_API.
private static func parseEnvFile(_ path: String) -> String? {
guard let contents = try? String(contentsOfFile: path, encoding: .utf8) else { return nil }
for rawLine in contents.split(whereSeparator: \.isNewline) {
var line = rawLine.trimmingCharacters(in: .whitespaces)
if line.isEmpty || line.hasPrefix("#") { continue }
if line.hasPrefix("export ") { line = String(line.dropFirst("export ".count)) }
guard let eq = line.firstIndex(of: "=") else { continue }
let name = line[..<eq].trimmingCharacters(in: .whitespaces)
guard name == "GROQ_API_KEY" || name == "GROQ_API" else { continue }
var value = line[line.index(after: eq)...].trimmingCharacters(in: .whitespaces)
// Strip surrounding quotes.
if value.count >= 2, let f = value.first, (f == "\"" || f == "'"), value.last == f {
value = String(value.dropFirst().dropLast())
}
if !value.isEmpty { return value }
}
return nil
}
}
14 changes: 14 additions & 0 deletions Sources/Bridge/SpeechProvider.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import Foundation

/// A backend that can transcribe recorded audio and (optionally) clean it up
/// with an LLM. Implemented by `ChatGPTBridge` (WebView) and `GroqClient` (REST).
@MainActor
protocol SpeechProvider: AnyObject {
/// Transcribe the audio file. Returns the raw transcript text.
func transcribe(audioURL: URL, language: String) async throws -> String

/// Run the cleanup LLM pass over `raw` using `prompt`. Returns cleaned text.
func cleanup(raw: String, prompt: String, model: String, language: String) async throws -> String
}

extension ChatGPTBridge: SpeechProvider {}
Loading