DeOldify: a free/open photo-retoucher based on machine learning

Originally published at: https://boingboing.net/2018/11/02/reverse-entropy.html

1 Like

I notice it did nothing to remove the gecko from her shoulder, which was the style of the time but today serves as a dead giveaway, dating the photo to a period of about 6 months (during which the gecko population decreased dramatically!)

2 Likes

I’m quite interested in AI and photo retouching. There have been some amazing AI based recoloring demos. But if the photo in the article is representative, this isn’t one of them. The image hasn’t been “deOldified” it’s just been very poorly colorized. The colorizing isn’t remotely photorealistic, isn’t accurate to the subject boundaries and, at best, emulates really bad amateur colorization.

That’s very complicated, compared to oil pastels.

1 Like

Note that if you click through and read the article, the images posted here are examples of early stages of a harder problem that he’s working on - getting faded photos to look good. Some of the examples he has on the page do look quite good (though the algorithm has an affinity for putting people into purple clothes for some reason).

However I’m stuck with the question of “why” - why do this? I can understand wanting to restore old photos but why colorize them? The black and white images that he has posted as “before” images look quite striking in many cases - colorizing them doesn’t seem to bring anything to the table to me. (But I’d love to see more work done on restoring old photos - anything that can help us with the preservation of history is great as far as I’m concerned).

1 Like

It is a little baffling to me. There’s a photo restoration group on Facebook that does free work. People are constantly submitting old photos and asking them to be colorized, and I’m a bit baffled. Especially when the submitter has no idea what the colors should be, such as eye color, or the uniform a person is wearing, or hair color. I’d much prefer a nice black and white restoration. But lots of people want color - something that came up years ago when film colorizing became a thing… :-/

My favorite future AI functions for photography are in creating perfect cutouts, including fly away hair, with perfect background color tint removal (thin or translucent objects always get some of the background color in them, and it’s hard to manually subtract that contamination, especially if the background is varied), and in color changing (manual color changing in Photoshop tends to give un-realistic monochrome looks). Demos in those areas have been stunning. Hoping for real world tools in not too long.

I do find this realm fascinating, though I have to admit I also have a deep passion for the earliest Kodachrome film, and early colour photography as well. I’ve gone to great lengths to digitize early Kodachrome film, and would LOVE to see the advances of modern colourization applied to film. This particular example of Halloween, 1932 that I digitized would be fascinating to see in colour: https://youtu.be/Fwqwv1LbSx8

I am not averse to colourization at all, and I both Kickstarted the book The Paper Time Machine at Unbound https://unbound.com/books/paper-time-machine/ and have also invited the man behind http://www.canadiancolour.ca/ to speak at the Vancouver Postcard Club.

I also follow Imbued with Hues on Facebook https://www.facebook.com/imbuedwithhues/ and am fascinated by the results, with the occasional critique of a colour choice or hue. But the criticisms are IMHO outweighed by the impact that colourized images can have to connect us to early 20th century history. The more accurately that they can be determined, the better. I will concede, respect for Black and White is an equally valid point of view.

Color images, from the past before color photography (especially those 19th century Russian RGB color-photos that have recently been recomposited), has the effect of making me more aware of the subjects as people I could probably relate to naturally if I had met them face-to-face, instead of just being “old-timey.”

The real killer app is automatically colorizing line drawings. I’ve used some software-assisted cel animation coloring software which was just basic (fill originating in the same clicked x-y coordinate), but saved a ton of time not having to color every frame. I can imagine Disney or some Japanese or Korean outfit hard at work adapting this software to do allow just one guy or gal to color ALL the output of a feature-length film. It just might bring 2D cel-animation costs down to 3D CG character-animation and cause another cel-animation renaissance.

3 Likes

This is what I do for a living, so I must say that I really wish this AI the least of luck!
As for the comment about us not knowing what colors to use, all uniforms - military and sports - can be researched. As can locations and plants and objects. That’s part of the fun.
https://www.mancavepictures.com/

2 Likes

Nice looking work :slight_smile:

This, though, is one of those areas where AI can excel, and will eventually do an excellent job of creating a realistic range of skin tones, which currently is hard to do in Photoshop, which still has relatively crap tools for color changing/colorizing.

The degeckofication filter is planned for version 2.0.

I can’t say the tools are crap, since I can do all that restoration and colorizing with Photoshop.
And I have a real hard time imagining that it can do a restoration like this any time soon, or automatically coloring the buildings without referencing old paintings. (Hong Kong 1869)

I think AI that allows input would be ideal. You still get to manually enter what the colors should be and the AI fills correctly and gives the photo realistic color range.

I’m not a colorizer (or an expert in color in any fashion), but my limited attempts in coloring images in Photoshop required different techniques and blending modes and futzing depending on what tones I was trying to color and what my target colors were. Plus lots of manual masking. AI can simplify a lot of that. To put it bluntly, in a limited fashion it can help me do easily what you do with skill and experience. It won’t do research, though. And can clearly do many abominations with the same ease. So I don’t think it will put you out of business, so much as change where your effort goes. Your artistry isn’t just technical skill, it is also your artistic taste.

Lots of video and photography functions have been democratized by technology. Just taking photos at night and getting the right exposure used to be a big deal. Film has a non-linear response at longer exposures, resulting in “reciprocity failure”, and crude exposure meters in film cameras cannot expose for the highlights of night lights so you had to know what you were doing. But these days, anybody with a digital camera can get very good auto exposed images, including at night since the exposure algorithms work off of the actual imaging chips not off a photo diode or resistor. Photographers can see their composition in real time off the chip, and their final image immediately thereafter. Having technical skills is still a step up above amateurs and dilettantes, but it is a much smaller step than it used to be. So now photographers have to find new ways to add value other than just knowing technical tricks that cameras and photo processing software can increasingly do for us with a reasonable level of quality.

1 Like

This topic was automatically closed after 5 days. New replies are no longer allowed.