MGIE demonstration
Image courtesy: Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, Zhe Gan, UC Santa Barbara, and Apple

Apple and University of California, Santa Barbara researchers have come up with a way to crop, resize and add filters to photos — without the help of traditional photo editing software. This and more are possible thanks to the new MGIE AI image editing model.

What is MGIE?

MGIE stands for MLLM-Guided Image Editing. The nested initialism, MLLM, refers to Multimodal Large Language Model. Put these pieces together and you get an image editing tool guided by the same plain-language instructions used to direct other AI models like ChatGPT and Midjourney.

That means you can simply type out the changes you want to see in the image, and MIGE will adjust the image accordingly. This include basic edits like cropping, flipping and resizing. However, MGIE can also apply more advanced edits, like adding filters or adjusting the brightness in specific areas of the frame. You can even do things like modify the shapes of objects in an image with simple, plain-language instructions.

In sum, MGIE promises to add flexibility and control to the image editing process. Though Apple is one tech giant that hasn’t yet invested heavily in AI yet, that may soon change. In addition to MGIE, Apple last year released an open-source AI framework called MLX, which aimed to make AI training on Apple Silicon chips easier.

How does it work?

When you give MGIE a prompt, two things happen. First, the model reinterprets your simple instruction into a more elaborate “expressive instruction.” For example, let’s say you give MGIE a photo of the Eiffel Tower along with the prompt, “turn the day into night.” The model will look at the source image and create more elaborate instructions based on the image’s context and subject matter: “If the day were to be turned into night in this image, the Eiffel Tower would be illuminated by artificial lights, creating a contrast against the dark sky.”

Next, the model “imagines” and applies the changes identified in the expressive instructions to the source image. The result is a more nuanced form of AI image generation and editing that can be used to perform complicated image edits without extensive knowledge of image editing software.

How to access MGIE

For the tech savvy and AI adventurous, MGIE is available for download from GitHub at mllm-ie.github.io. For the AI curious, you can test out the demo on Hugging Face, the AI community development hub.

Nicole LaJeunesse is a professional writer and a curious person who loves to unpack stories on anything from music, to movies, to gaming and beyond.