Artificial intelligence is changing the world of image editing and manipulation, and Adobe doesnt want to be left behind. Today, the company is releasing an update to Photoshop version 22.0 that comes with a host of AI-powered features, some new, some already shared with the public. These include a sky replacement tool, improved AI edge selection, and the star of the show a suite of image-editing tools that Adobe calls neural filters.
These filters include a number of simple overlays and effects but also tools that allow for deeper edits, particularly to portraits. With neural filters, Photoshop can adjust a subjects age and facial expression, amplifying or reducing feelings like joy, surprise, or anger with simple sliders. You can remove someones glasses or smooth out their spots. One of the weirder filters even lets you transfer makeup from one person to another. And its all done in just a few clicks, with the output easily tweaked or reversed entirely.
This is where I feel we can now say that Photoshop is the worlds most advanced AI application, Maria Yap, Adobes vice president of digital imaging told The Verge. Were creating things in images that werent there before.
To achieve these effects, Adobe is harnessing the power of generative adversarial networks or GANs a type of machine learning technique thats proved particularly adept at generating visual imagery. Some of the processing is done locally and some in the cloud, depending on the computational demands of each individual tool, but each filter takes just seconds to apply. (The demo we saw was done on an old Mac Book Pro and was perfectly fast enough.)
Many of these filters are familiar to those who follow AI image editing. Theyre the sort of tools that have been turning up in papers and demos for years. But its always significant when techniques like these go from bleeding-edge experiments, shared on Twitter among those in the know, to headline features in consumer juggernauts like Photoshop.
As always with these sorts of features, the proof will be in the editing, and the actual utility of neural filters will depend on how Photoshops many users react to them. But in a virtual demo The Verge saw, the new tools delivered fast and good quality results (though we didnt see the facial expression adjustment tool). These AI-powered edits werent flawless, and most professional retouchers would want to step in and make some adjustments of their own afterwards, but they seemed like they would speed up many editing tasks.
AI tools like this work by learning from past examples. So, to create the neural filter thats used to smooth away skin blemishes, for example, Adobe collected thousands of before and after shots of edits made by professional photographers, feeding this data into their algorithms. The GANs operate like a paired student and teacher, with one part trying to copy these examples while the other tries to distinguish between this output and the training data. Eventually, when even the GAN is getting confused trying to tell the difference between the two, the training process is complete.
Basically, were training the GAN to make the same corrections a professional retoucher would do, Alexandru Costin, Adobes vice president of engineering for Creative Cloud, told The Verge.
It sounds straightforward, but there are lots of ways this training can go wrong. A big one is biased data. The algorithms only know the world you show them, so if you only show them images of, say, white faces, they wont be able to make edits for anyone whose complexion doesnt fit within this narrow range. This sort of bias is why facial recognition systems often perform worse on women and people of color. These faces just arent in the training data.
Costin says Adobe is acutely aware of this problem. If it trained its algorithms on too many white faces, he says, its neural filters might end up pushing AI-edited portraits toward whiter complexions (a problem weve seen in the past with other ML applications).
One of the biggest challenges we have is preserving the skin tone, says Costin. This is a very sensitive area. To help root out this bias, Adobe has set up review teams and an AI ethics committee that test the algorithms every time a major update is made. We do a very thorough review of every ML feature, to look at this criteria and try and raise the bar.
But one key advantage Adobe has over other teams building AI image-editing tools is its catalog of stock photography a huge array of images that span different ages, races, genders. This, says Costin, made it easy for Adobes researchers to balance their datasets to try to minimize bias. We complemented our training data with Adobe stock photos, says Costin, and that allowed us to have a good as possible, distributed training set.
Of course, all this is no guarantee that biased results wont appear somewhere, especially when the neural filters get out of beta testing and into the hands of the general public. For that reason, each time a filter is applied, Photoshop will ask users whether theyre happy with the results, and, if theyre not, give them the option of reporting inappropriate content. If users choose, they can also send their before and after images anonymously to Adobe for further study. In that way, the company hopes to not only remove bias, but also expand its training data even further, pushing its neural filters to greater levels of fidelity.
This sort of speedy update based on real-world usage is common in the fast-moving world of AI research. Often, when a new machine learning technique is published (usually on a site named arXiv, an open-access collection of scientific papers that havent yet been published in a journal), other researchers will read it, adopt it, and adapt it within days, sharing results and tips with one another on social media.
Some AI-focused competitors to Photoshop distinguish themselves by embracing this sort of culture. A program like Runway ML, for example, not only allows users to train machine learning filters using their own data (something that Photoshop does not), but it operates a user-generated marketplace that makes it easy for people to share and experiment with the latest tools. If a designer or illustrator sees something cool floating around on Twitter, they want to start playing with it immediately rather than wait for it to trickle into Photoshop.
As a widely used product with customers who value stability, Adobe cant truly compete with this sort of speed, but with neural filters, the company is dipping a toe into these fast-moving waters. While two of the filters are presented as finished features, six are labeled as beta tools, and eight more are only listed as names, with users having to request access. You can see a full list of the different filters and their respective tiers below:
Featured Neural Filters: Skin Smoothing, Style TransferBeta Neural Filters: Smart Portrait, Makeup Transfer, Depth-Aware Haze, Colorize, Super Zoom, JPEG Artifacts RemovalFuture Neural Filters: Photo Restoration, Dust and Scratches, Noise Reduction, Face Cleanup, Photo to Sketch, Sketch to Portrait, Pencil Artwork, Face to Caricature
Yap says this sort of approach is new to Photoshop but will hopefully let Adobe temper users expectations about AI tools, giving them the license to update the tools more quickly. Weve built this framework that allows us to bring models [to users] faster, from research to Photoshop, says Yap. Traditionally when we do features, like sky replacement, theyre really deeply integrated into the product and so take a longer time to mature. With neural filters, that update cycle will ideally be much faster.
Its this pace that were trying to bring into Photoshop, says Costin. And it will come at the cost of the feature not being perfect when we launch, but were counting on our community of users to tell us how good it is [...] and then we will take in that data and refine it and improve it.
In other words: the flywheel of AI progress, wherein more users create more data that creates better tools, is coming to Photoshop. Tweaking someones age is just the start.
- The 12 Coolest Machine-Learning Startups Of 2020 - CRN - November 19th, 2020
- Utilizing machine learning to uncover the right content at KMWorld Connect 2020 - KMWorld Magazine - November 19th, 2020
- The way we train AI is fundamentally flawed - MIT Technology Review - November 19th, 2020
- DIY Camera Uses Machine Learning to Audibly Tell You What it Sees - PetaPixel - November 19th, 2020
- Machine Learning Predicts How Cancer Patients Will Respond to Therapy - HealthITAnalytics.com - November 19th, 2020
- This New Machine Learning Tool Might Stop Misinformation - Digital Information World - November 19th, 2020
- Fujitsu, AIST and RIKEN Achieve Unparalleled Speed on MLPerf HPC Machine Learning Processing Benchmark - HPCwire - November 19th, 2020
- SVG Tech Insight: Increasing Value of Sports Content Machine Learning for Up-Conversion HD to UHD - Sports Video Group - November 19th, 2020
- SiMa.ai Adopts Arm Technology to Deliver a Purpose-built Heterogeneous Machine Learning Compute Platform for the Embedded Edge - Design and Reuse - November 19th, 2020
- Machine learning removes bias from algorithms and the hiring process - PRNewswire - November 6th, 2020
- Using machine learning to track the pandemic's impact on mental health - MIT News - November 6th, 2020
- AI Recognizes COVID-19 in the Sound of a Cough Machine Learning Times - The Predictive Analytics Times - November 6th, 2020
- The consistency of machine learning and statistical models in predicting clinical risks of individual patients - The BMJ - The BMJ - November 6th, 2020
- PathAI and Gilead Report Data from Machine Learning Model Predictions of Liver Disease Progression and Treatment Response at AASLD's The Liver Meeting... - November 6th, 2020
- Google Introduces New Analytics with Machine Learning and Predictive Models - IBL News - November 6th, 2020
- Free Webinar | Machine Learning and Data Analytics in the Pandemic Era - MIT Sloan - November 6th, 2020
- Global Predictive Analytics Market (2020 to 2025) - Advent of Machine Learning and Artificial Intelligence is Driving Growth - PRNewswire - November 6th, 2020
- Machine learning and predictive analytics work better together - TechTarget - October 31st, 2020
- Microsoft Introduces Lobe: A Free Machine Learning Application That Allows You To Create AI Models Without Coding - MarkTechPost - October 31st, 2020
- Amwell CMO: Google partnership will focus on AI, machine learning to expand into new markets - FierceHealthcare - October 31st, 2020
- Microsoft/MITRE group declares war on machine learning vulnerabilities with Adversarial ML Threat Matrix - Diginomica - October 31st, 2020
- 93% of security operations centers employing AI and machine learning tools to detect advanced threats - Security Magazine - October 31st, 2020
- Machine Learning in Insurance Market(COVID-19 Analysis): Indoor Applications Projected to be the Most Attractive Segment during 2020-2027 - Global... - October 31st, 2020
- Leveraging Machine Learning and IDP to Scale Your Automation Program - AiiA - October 31st, 2020
- 5 machine learning skills you need in the cloud - TechTarget - October 31st, 2020
- Machine learning approach could detect drivers of atrial fibrillation - Cardiac Rhythm News - October 31st, 2020
- Vanderbilt trans-institutional team shows how next-gen wearable sensor algorithms powered by machine learning could be key to preventing injuries that... - October 31st, 2020
- Machine Learning & Big Data Analytics Education Market Size And Forecast (2020-2026)| With Post Impact Of Covid-19 By Top Leading Players-... - October 31st, 2020
- The security threat of adversarial machine learning is real - TechTalks - October 31st, 2020
- Bridging the Skills Gap for AI and Machine Learning - Integration Developers - October 23rd, 2020
- Nudges and machine learning triples advanced care conversations - Penn Today - October 23rd, 2020
- Machine Learning and AI Can Now Create Plastics That Easily Degrade - Science Times - October 23rd, 2020
- insitro Strengthens Machine Learning-Based Drug Discovery Capabilities with Acquisition of Haystack Sciences - Business Wire - October 23rd, 2020
- Revolutionizing IoT with Machine Learning at the Edge | Perceive's Steve Teig - IoT For All - October 23rd, 2020
- Mastercard Says its AI and Machine Learning Solutions Aim to Stop Fraudulent Activites which have Increased Significantly due to COVID - Crowdfund... - October 23rd, 2020
- Abstract Perspective: Long-term PM2.5 Exposure and the Clinical Application of Machine Learning for Predicting Incident Atrial Fibrillation - DocWire... - October 23rd, 2020
- Machine-Learning Inference Chip Travels to the Edge - Electronic Design - October 23rd, 2020
- Machine Learning Data Catalog Software Market share forecast to witness considerable growth from 2020 to 2025 | By Top Leading Vendors IBM, Alation,... - October 23rd, 2020
- AI and machine learning: a gift, and a curse, for cybersecurity - Healthcare IT News - October 21st, 2020
- Teaming Up with Arm, NXP Ups Its Place in the Machine Learning Industry - News - All About Circuits - October 21st, 2020
- Machine Learning Capabilities Come to the Majority of Open Source Databases with MindsDB AI-Tables - PR Web - October 21st, 2020
- Soleadify secures seed funding for database that uses machine learning to track 40M businesses - TechCrunch - October 21st, 2020
- NXP Announces Expansion of its Scalable Machine Learning Portfolio and Capabilities - GlobeNewswire - October 21st, 2020
- NXP Invests in Au-Zone to Enhance Machine Learning Capabilities - Mobile ID World - October 21st, 2020
- Factories of The Future Are Using Machine Learning Analytics to Optimize Assets - Embedded Computing Design - October 21st, 2020
- Lantronix Brings Advanced AI and Machine Learning to Smart Cameras With New Open-Q 610 SOM Based on the Powerful Qualcomm QCS610 System on Chip (SOC)... - October 21st, 2020
- EMA Webinar to Uncover How Machine Learning and Predictive Analytics Can Improve Workload Automation Outcomes - PR Web - October 21st, 2020
- How To Choose The Best Machine Learning Algorithm For A Particular Problem? - Analytics India Magazine - October 21st, 2020
- AI and Machine Learning Technologies Expected to Play a Key Role in Expanding Multi Billion Dollar Digital Banking Sector: Report - Crowdfund Insider - October 21st, 2020
- EXCLUSIVE: Amazon AI executive explains three things every business needs to address before using machine lear - Business Insider India - October 21st, 2020
- Futurism Reinforces Its Next-Gen Business Commerce Platform With Advanced Machine Learning and Artificial Intelligence Capabilities - Yahoo Finance - October 15th, 2020
- Purebase Enhances Its Board of Advisors with An Expert on Machine Learning and Cheminformatics - GlobeNewswire - October 15th, 2020
- How to Beat Analysts and the Stock Market with Machine Learning - Knowledge@Wharton - October 15th, 2020
- Synopsys and SiMa.ai Collaborate to Bring Machine Learning Inference at Scale to the Embedded Edge - AiThority - October 15th, 2020
- Robotic Interviews, Machine Learning And the Future Of Workforce Recruitment - Entrepreneur - October 15th, 2020
- Top 8 Books on Machine Learning In Cybersecurity One Must Read - Analytics India Magazine - October 15th, 2020
- AI and Machine Learning Can Help Fintechs if We Focus on Practical Implementation and Move Away from Overhyped Narratives, Researcher Says - Crowdfund... - October 15th, 2020
- AI and Machine Learning Can Help FIs Avoid Riskbut They Have Risk of Their Own - PR Web - October 15th, 2020
- Machine learning for rowdy roadies: Cops and tech to rein in traffic offenders - Bangalore Mirror - October 15th, 2020
- Automated ATOs and cybersecurity - FCW.com - October 15th, 2020
- Experian partners with Standard Chartered to drive Financial Inclusion with Machine Learning, powering the next generation of Decisioning - Yahoo... - October 15th, 2020
- Machine Learning Answers: Facebook Stock Is Down 20% In A Month, What Are The Chances It'll Rebound? - Trefis - September 22nd, 2020
- Machine Learning in Education Market Incredible Possibilities, Growth Analysis and Forecast To 2025 - The Daily Chronicle - September 22nd, 2020
- Proximity matters: Using machine learning and geospatial analytics to reduce COVID-19 exposure risk - Healthcare IT News - September 22nd, 2020
- Global Machine Learning Market Tends To Show Steady Growth Post Pandemic With Regional Overview and Top Key Players - Verdant News - September 22nd, 2020
- PREDICTING THE OPTIMUM PATH - Port Strategy - September 22nd, 2020
- AI/ML Remains The Most In-Demand Tech Skill Post COVID - Analytics India Magazine - September 22nd, 2020
- Panalgo Brings the Power of Machine-Learning to the Healthcare Industry Via Its IHD Software - AiThority - September 15th, 2020
- Microchip Partners with Machine-Learning (ML) Software Leaders to Simplify AI-at-the-Edge Design Using its 32-Bit Microcontrollers (MCUs) - EE Journal - September 15th, 2020
- What is 'custom machine learning' and why is it important for programmatic optimisation? - The Drum - September 15th, 2020
- PODCAST: NVIDIA's Director of Data Science Talks Machine Learning for Airlines and Aerospace - Aviation Today - September 15th, 2020
- The Use of Machine Learning to Forecast Progression to Advanced AMD - DocWire News - September 15th, 2020
- How Can Machine Learning Help the Teaching Profession? - FE News - September 15th, 2020
- Global Machine Learning in Automobile Market: Development History, Current Analysis and Estimated Forecast to 2024 - The Market Correspondent - September 15th, 2020
- Using machine learning to organize the chemical diversity - Tech Explorist - September 15th, 2020
- Dashboard AI Announces Its Technology Vision for the Foodservice and Hospitality Industry - PRNewswire - September 15th, 2020
- Alfa Releases Second Paper on AI, Using Machine Learning in the Wild - Monitor Daily - September 10th, 2020
- Combatting COVID-19 misinformation with machine learning (VB Live) - VentureBeat - September 10th, 2020
- This artist used machine learning to create realistic portraits of Roman emperors - The World - September 10th, 2020
- Domino Data Lab Named a Leader in Notebook-Based Predictive Analytics and Machine Learning Evaluation by Global Research Firm - Business Wire - September 10th, 2020