Automatically OCR screenshots

If great artists steal, I hope great designers take a lot of screenshots, because I take a lot of screenshots. When auditing an existing problem space, it’s incredibly useful to explore existing design conventions. At this point I have an embarrassingly large volume of screenshots, some going as far back as the 90s.

Though they do in Photos, Apple doesn’t automatically OCR screenshots in the Finder or attach any recognized text as metadata, but thanks to a little script and some Automator, we can process incoming screenshots in the background.

What you’ll need

Select a new home for screenshots (e.g. ~/Pictures/Screenshots) using the Screenshot application located in Applications/Utilities

Install Homebrew…

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

…so you can then install the OCR library Tesseract

brew install tesseract

Then open Apple’s Automator and make a new Workflow. Drop in Ask for Finder Items, Filter Finder Items, and Run Shell Script with the following configuration.

Paste in the shell script produced for me courtesy my stellar former colleagues Edouard and Vlad.

for f in "$@"
do
  /opt/homebrew/bin/tesseract "$f" stdout | xargs -0 
osascript -e 'on run {f, c}' -e 'tell app "Finder" to set comment of (POSIX file f as alias) to c' -e end "$f"
done

And voila! Run the script on a few images (the limit is around 1,000 a batch) and you should see the recognized text appear in the Finder comments.

One you feel comfortable you can augment the workflow with other scripts. For example, to reduce the file size you can prepend ImageOptim.

When you are ready to automatically OCR screenshots in the background, from the File menu select Convert To… and Folder Action. Then select your Screenshots folder, remove Ask for Finder Items section and the next time you take a screenshot you should see the ⚙️ icon spinning in the system menu bar.

Any questions or suggestions? Ping @ryanfeeley on Twitter.