Open Interface

Self-Operate Computers using LLMs

Imagine a future where you could just ask your computer to figure out how to use Garage Band to add a backing track to your latest voice memo in the style of Dr. Dre. Or edit the videos from your latest Tahoe trip to make a Godfather 2-esque trailer in iMovie.
While we’re not there yet, watching Open Interface run for the first time thrilled me about this future.

Open Interface, in one line, is self-driving software for your computer. It runs on MacOS, Windows, and Linux, is open-sourced, and available to download from this Github repository.

Below is a brief demo of asking it to create a meal plan in Google Docs.

Using an LLM backend (OpenAI’s GPT-4V), Open Interface figures out how to convert user-requests to a series of keyboard-strokes and mouse-clicks that it can then execute automatically to open applications and do tasks.

---

Check out Open Interface on Github - github.com/AmberSahdev/Open-Interface