There's a new Apple image editor, if you know where to look. The iPhone king teamed up with researchers at the University of California at Santa Barbara to create a tool that lets you edit photos and images with text-based instructions. There's no official release, but researchers are hosting a demo you can try yourself, which has been seen for the first time Extreme Tech,
This project is called Multimodal Large Language Model Guided Image Editing (MGIE). There are a lot of AI image editors on the market right now. Photoshop now comes with built-in AI tools, and other tools like OpenAI's DALL-E let you create images from entire content in addition to editing them. However, if you've ever tried to use them, you know that it can be a little frustrating. In many cases, AI has difficulty understanding what you are actually looking for.
The innovation with MGIE is adding another layer of AI interpretation. When you tell the AI what you want to see, MGIE first uses text-based AI to make your instructions more clear and descriptive. “Experimental results demonstrate that expressive instructions are important for instruction-based image editing,” the researchers said in a study. paper Published on arXiv. “Our MGIE can bring significant improvements.”
Apple published an open-source version of the software GitHub, If you're savvy you can run a version of MGIE on your own, but researchers have set up this tool hugging face, It runs a little slow when there are a lot of people using it, but it's a fun experiment.
Giant tech companies like Apple spend billions of dollars on projects that no one ever sees, so it's entirely possible that this so-called MGIE tool will never get an official release. Apple did not immediately respond to a request for comment.
We tested it ourselves here at the Gizmodo office. I uploaded a photo of my co-worker and closest advisor Kyle Barr wearing a weird pair of sunglasses that he took on Netflix. This year's consumer electronics show, I said to the AI ”Man standing in the desert.” Before creating the image, the MGIE tool extrapolated:
“The man is wearing a metal helmet and standing in the desert. The environment around him is dry and barren, with sand dunes stretching as far as the eye can see.”
After playing with the tool for much longer than we should have, it's clearly subject to many of the same limitations as any other AI image generator. Sometimes the results are strange and nothing like you asked for. But in some cases, it worked impressively, and in the program's defense, the AI performs better with familiar subjects. “Familiar” isn't something you'd call Kyle's sunglasses.