I don't use the openai capabilities. I use talon voice recognition. I can control my computer well enough to code by voice, and {edit: in a prerelease patreon version,] the dictation mode is now based on whisper. It's pretty amazing.

My favorite part from the Getting Things Done book is the idea to capture 100% of your ideas, and to only process them after the fact. Rather than, say, trying to only write down good ideas. On LW this philosophy is known as Babble and Prune.

So for years, I've wished for the ability to record voice notes anytime I want, and to then get an accurate transcript automatically. Almost exactly one year ago, I bought a Pixel 7 phone for this very reason, hoping that their advertised AI chip and Recorder app could provide just that. They couldn't; the Recorder app prioritizes live transcription over accuracy, and the transcript is not usable without listening to the recording, which defeats the point.

However, due to Whisper, I can now indeed record voice memos via my phone or, newly, my smartwatch; then upload the file to cloud storage (e.g. Google Drive); and then immediately and automatically receive a Whisper AI transcript (awesome in its accuracy) and ChatGPT summary etc. (so far irrelevant for me) in my Notion workspace.

This is implemented by following this step-by-step automation guide by Thomas Frank, and only requires an OpenAI account incl. API key (costing $0.40 per hour of audio), a free Pipedream account (which is like Zapier but allows arbitrary code blocks), free cloud storage account, and free Notion account.

In principle the Notion part is unnecessary, and someone who wanted to take the time to manually adjust the automation could have the transcript output instead be an email or text file or whatever.

I liked this extension (https://chrome.google.com/webstore/detail/whispering/oilbfihknpdbpfkcncojikmooipnlglo), which I use for long messages. I press a shortcut, it starts recording with Whisper, then repress and it puts the transcript in my clipboard.

I can now get real-time transcripts of my zoom meetings (via a python wrapper of the openai api) which makes it much easier to track the important parts of a long conversation. I tend to zone out sometimes and miss little pieces otherwise, as well as forget stuff.

ChatGPT is using Whisper for speech to text, which is open source and available through OpenAIs APIs.

I personally tried to use more text to speech on my phone, but was annoyed by it and went back to typing.

I've heard Whisper is a definite step-up, especially when mixing English and German.

https://openai.com/research/whisper
https://github.com/openai/whisper
https://platform.openai.com/docs/models/whisper

€: This used to say text to speech.