Getting Around in GIMP - G'MIC Inpainting (Content Aware Fill)

One of the advantages of hanging around people smarter than me is that I get to see some really neat things as they are being developed. Lucky for me, +David Tschumperlé let’s me see interesting things that they are hacking on over at G’MIC.

Of course, if you’re reading me occasionally you’ll remember that last time I got a chance to work on something with G’MIC, we ended up making some fun film emulation presets. This time I’m not involved other than to play, and it’s really neat!

It is the latest work on the patch-based inpainting algorithm.

I had previously written about using the Resynthesizer plug-in (with the Heal Selection script) for GIMP to produce results similar to Photoshop’s “Content Aware Fill”, but I think at this point I’m going to switch over to using G’MIC full-time for this type of work.

What the algorithms basically do is take a given region in an image that needs to be replaced, search around that region for textures to fill it with, determine what works well, then fills it in (see the previous post for a visual explanation of this).

Well, the folks over at G’MIC have been working hard on improving their algorithm for doing this type of inpainting, focusing on results and speed. Let’s have a look…

G’MIC Inpainting

I’m using some examples from my previous post on Resynthesizer so we can compare the results. To access the command, it will be in G’MIC:

Filters → G'MIC

then,

Repair → Inpaint [patch-based]

A major difference between Inpaint and Resynthesizer is the ability to fine-tune the algorithm for your image. Resynthesizer only really has a couple of parameters - how big of a search area to use around your selection, and to restrict the texture search to all-around, sides, or top/bottom.

By contrast, here are the parameters for the Inpainting [patch-based] algorithm:

In speaking with David, the primary parameters that one would use to adjust the results would mainly be the Patch size, Lookup size, and Blend size.

To experiment, I just created a duplicate of my base layer, made a freehand selection around my object to remove, sharpened the selection, and filled it with bright red (to match the mask color indicated in the G’MIC window):

So, let’s have a quick run with the same selection as above, but with the default settings to see where things stand:

Not too bad with the defaults. Not too good either, though.

As I said earlier, what’s really nice about the G’MIC Inpainting vs. Resynthesizer is that I can tune the parameters to my liking…

Tweaking

As I understand it from David, the Patch size parameter adjusts the size of the area that will be copied from the surrounding texture to place into the region. The Lookup size adjusts how far away from the inpaint region to look for new textures (similar to “sampling width” in Resynthesizer), and the Blend size adjusts how big of a blend to allow when patching.

I left the Blend size alone, but did increase the patch size to 25 (from the default of 7). I also increased the Lookup factor a bit to allow a bigger search radius from a given patch. Here are the results:

This is a really nice result and I prefer the G’MIC version over the Resynthesizer output here. Being able to modify parameters to adjust how the algorithm approaches the problem really opens up many more possibilities.

Of course, here are some requisite samples using G’MIC, pulled from the previous samples I had done with Resynthesizer.

I think with just a small bit of experimentation we can get some really fantastic results. Particularly in places where it might have been harder for Resynthesizer to work well.

The best part is that this is already a part of G’MIC, and the team has been actively working to make it faster and better (I believe they are even presenting a paper on this topic later this year).

Personally, this is my future go-to for inpainting/content-aware-fill where I need it (not often, but when I do, boy am I glad we have options like this). If you haven’t already, you may want to give G’MIC a try (don’t forget there’s a ton of fun film emulation filters as well!).