Originally published at: https://boingboing.net/2018/05/09/enhance-enhance.html
…
3 Likes
2 candles one neural pipe.
3 Likes
They threw away the algorithm that learned how to open Photoshop and adjust the levels.
3 Likes
The system deals with the issues they point out (white balance and noise) extremely well, but the result it still a pretty fuzzy photo, which makes sense. There’s no substitute for catching more photons, however you go about that.
Which is not to dump on this at all. I guess I just find it comforting that there’s still some distance separating state-of-the-art image processing from straight-up magic.
7 Likes
I wonder if it will learn to read lips.
2 Likes
Just imagine what it can do with brighter images (superresolution?), or being able to bring out shadows. Or showing non-HDR content on an HDR-capable monitor. Very exciting stuff!
2 Likes
I wonder if this is temporally coherent. Could they apply it to frames of a video and have it still look good when played back?
4 Likes
This seems like a dangerous path to go down, to me. If this type of technology gets adopted by camera-makers, then they will begin to subtly affect the art of photography by their choice of what’s included in the training data. If one can take a low-light shot, but only if the subject matter is in some way “conventional,” according to how the algorithm was trained, then it makes it very difficult for the artist to prevent those biases from entering into their work.
It’s obviously the kind of thing that’s very useful for regular consumers, and artists don’t have much market power, so I fear there’s no stopping it.
4 Likes
It’s not really seeing, it’s making good guesses from a list of things that it’s been trained to see.
If that ottoman in the dark is really a live alligator, oops, it wasn’t expecting that.
This reminds me of Vinge’s The Children of the Sky, where someone is snooping on a conversation, and finally checks the status of the snoop net, and finds that it has been failing badly over time and the detailed extrapolations are based on real data that’s heading to the noise level.
5 Likes
The folks who make the Light cameras do at least have a superficially plausible way around thermal and shot noise, by taking several simultaneous photos with identical lenses, allowing you to do your denoising by comparing the images.
That said, their demo images on the website aren’t that impressive. I have some lenses much cheaper than that camera that take way better pictures than their examples.
I’m waiting to see if the technology eventually results in something better.
Neat, but describing the results as “super sharp” is … not quite right, put mildly. The pictures still have the tell-tale softness of heavy noise reduction - the impressive part here is the amount of detail, particularly color detail, preserved considering the massively boosted exposure.
Also, for some of those horrendously noisy A7SII photos (indoors stuff), they must have been using ISO 100000+. as that camera has ridiculously low noise.
2 Likes
Well, colour me astounded.
This is fucken sweet. How long till I can install an app on my phone with this tech?
The art of photography will certainly be affected if “pictures” turn into “what the neural network confabulated based on a cheap sensor driven to the extremes”(though, given that human vision is…not exactly…an objective datafeed sent to the homonculus for transcription to the tabula rasa, it will probably be quite popular and deemed fairly impressive for Instagram happy-snapping).
The “dangerous path to go down” side will really heat up when Team DoD(or one of their foreign counterparts) expresses the interest they would obviously have in getting imaged out of very, very low light or otherwise troublesome images. Unless someone insists on a training set so conservative that the customer gets deeply upset about false negatives you’ll probably end up with imaging systems that paints hostiles into the scene with enthusiasm.
More broadly, this sort of issue seems like it is likely to crop up in a number of areas as the appropriate neural networks become more widely available. Aside from all the awful sensor input cases there is the entire genre of lossy compression:
The whole point of that exercise is to throw away as much data as you can without irking the viewer. With the relatively crude techniques (chroma subsampling, discarding audio outside human frequency range, etc.) the results can be kind of nasty; but generally not too misleading. You might lose detail you need(awful audio compression, say, might ruin your ability to distinguish two speakers or pick someone out of the background noise); but most of the time you can tell when compression artifacts are messing with you. The one major counterexample I can think of was the frankly astonishing Xerox JBIG2 issue; which did involve a compression mechanism designed around having the computer stitch together that it ‘thought’ it saw (essentially it determined how many distinct characters a document had and saved space by OCRing and deduplicating; with characters that are ‘the same’ all being represented by references to a single image of that character. As a compression technique OCR is undoubtedly massively better for text scans than any image compression option; but using it this way also means that any OCR errors, and their usually are some, are impossible to detect in the output, barring outside knowledge of what the document was supposed to say; and that, unlike the “compression to scrunchy to read” case, the erroneous bits look perfect).
With more advanced neural network witchcraft we will presumably be able to bring this style of compression to all sorts of areas; but will face the same problem. Crude lossy compression can, if used in a way unsuited to your purpose, destroy details you need; but the destruction is generally evident(and, even if you use tricks like the old “a bit of unsharp a day keeps the jaggies away!” the result is less overtly offensive to the typical viewer; but the missing detail is still clearly missing.
‘Confabulation’ based compression will likely turn in some amazingly good compression ratios(at the expense of relatively massive compression and decompression programs and a lot of computational expense; but they will also promote ‘error specifically engineered for apparent accuracy’ to being, essentially, an intended feature.
The results will likely look spooky good for their size; but that will be the problem.
2 Likes
Respectfully, that sounds like capturing more photons.
The Light camera looks really interesting, though.
This topic was automatically closed after 5 days. New replies are no longer allowed.
