Glass supercharges smartphone cameras with AI — minus the hallucinations
Your phone’s camera is as much software as it is hardware, and Glass is hoping to improve both. But while its wild anamorphic lens creeps to market, the company (running on $9.3 million in new money) has released an AI-powered camera upgrade that it says vastly improves image quality — without any weird AI upscaling artifacts.
GlassAI is a purely software approach to improving images, what they call a neural image signal processor (ISP). ISPs are basically what take the raw sensor output — often flat, noisy and distorted — and turn that into the sharp, colorful images we see.
The ISP is also increasingly complex, as phone makers like Apple and Google like to show, synthesizing multiple exposures, quickly detecting and sharpening faces, adjusting for tiny movements, and so on. And while many include some form of machine learning or AI, they have to be careful: Using AI to generate detail can produce hallucinations or artifacts as the system tries to create visual information where none exists. Such “super-resolution” models are useful in their place, but they have to be carefully monitored.
Glass makes both a full camera system based on an unusual lozenge-shaped front element, and an ISP to back it up. And while the former is working toward market presence with some upcoming devices, the latter is, it turns out, a product worth selling in its own right.
“Our restoration networks correct optical aberrations and sensor issues while efficiently removing noise, and outperform traditional Image Signal Processing pipelines at fine texture recovery,” explained CTO and co-founder Tom Bishop in their news release.
The word “recovery” is key, because details are not simply created but extracted from raw imagery. Depending on how your camera stack already works, you may know that certain artifacts or angles or noise patterns can be reliably resolved or even taken advantage of. Learning how to turn these implied details into real ones — or combining details from multiple exposures — is a big part of any computational photography stack. Co-founder and CEO Ziv Attar says their neural ISP is better than any in the industry.
Even Apple, he pointed out, doesn’t have a full neural image stack, only using it in specific circumstances where it’s needed, and their results (in his opinion) aren’t great. He provided an example of Apple’s neural ISP failing to interpret text correctly, with Glass faring much better
GlassAI is a purely software approach to improving images, what they call a neural image signal processor (ISP). ISPs are basically what take the raw sensor output — often flat, noisy and distorted — and turn that into the sharp, colorful images we see.
The ISP is also increasingly complex, as phone makers like Apple and Google like to show, synthesizing multiple exposures, quickly detecting and sharpening faces, adjusting for tiny movements, and so on. And while many include some form of machine learning or AI, they have to be careful: Using AI to generate detail can produce hallucinations or artifacts as the system tries to create visual information where none exists. Such “super-resolution” models are useful in their place, but they have to be carefully monitored.
Glass makes both a full camera system based on an unusual lozenge-shaped front element, and an ISP to back it up. And while the former is working toward market presence with some upcoming devices, the latter is, it turns out, a product worth selling in its own right.
“Our restoration networks correct optical aberrations and sensor issues while efficiently removing noise, and outperform traditional Image Signal Processing pipelines at fine texture recovery,” explained CTO and co-founder Tom Bishop in their news release.
The word “recovery” is key, because details are not simply created but extracted from raw imagery. Depending on how your camera stack already works, you may know that certain artifacts or angles or noise patterns can be reliably resolved or even taken advantage of. Learning how to turn these implied details into real ones — or combining details from multiple exposures — is a big part of any computational photography stack. Co-founder and CEO Ziv Attar says their neural ISP is better than any in the industry.
Even Apple, he pointed out, doesn’t have a full neural image stack, only using it in specific circumstances where it’s needed, and their results (in his opinion) aren’t great. He provided an example of Apple’s neural ISP failing to interpret text correctly, with Glass faring much better
Comments
Post a Comment