8 Best Virtual Makeup SDKs (Compared)
See how computer vision is powering the makeup industry.
·
Join the DZone community and get the full member experience.
Join For Free
People have always enjoyed playing with their appearance. With augmented reality, this process has become easier and more accessible than never before.
This post outlines the landscape of beauty AR software, the best Virtual Makeup SDKs to implement beauty AR experiences, and a comparison of their features and performance.
What Are Beauty AR Experiences?
Beauty AR experiences involve the overlay of digital elements on a user’s face or modification for a better visual representation. These experiences can be segmented into three types depending on the program's functionality and purpose.
Makeup Try-Ons
Makeup try-ons help consumers shop for cosmetics by allowing them to try on items in augmented reality using the web or a mobile camera. Therefore, the true-to-life representation of digital elements (makeup) plays a crucial role.
Video Face Beautification
Used in video and photo editors or camera apps like Snapchat or TikTok, beautification includes a range of face retouch tools. It adds skin smoothing, face morphing (cheeks slim down), teeth whitening, and lighting adjustment to name just a few. Its purpose is to fix camera distortion and make the user enjoy the visual image in the camera.
Facial Feature Modification
By detecting the facial parameters and segmenting the face in certain parts, the technology allows users to see themselves with a different eye and hair color or try on different eyebrows. Facial feature modification can also be applied in cosmetic surgery, demoing the desired result from procedures, such as facelifting or plastic surgery.
Where to Implement Beauty AR Experiences
Beauty AR experience can be implemented on mobile, web, and in-store. Each location has its own advantages and pitfalls.
Mobile virtual makeovers are widely used by all leading cosmetics brands like L’Oreal, Sephora, and Maybelline. They provide a more quality user experience compared to web solutions, due to more powerful camera capabilities. Users can easily take photos and share them on their social networks right from the app. As a drawback, they may drain battery and many users feel reluctant to install these apps.
WebAR beauty experiences are integrated right on a website. The user needs only to allow access to the web camera to experience them. However, the loading usually takes a significant amount of time, and the AR experience doesn’t look nice without beautification — the camera distortion and poor lighting spoil all the fun.
In-store beauty AR experiences are enabled either with iPads or AR mirrors displayed in-store; they have the same functionality as mobile makeovers. They improve the in-store shopping experience by allowing visitors to test products without wandering the shelves. As a drawback, the in-store AR experience is limited to its physical location, i.e. a store.
What Are the Virtual Makeup SDK Providers?
The key augmented reality companies that provide Virtual Makeup SDKs for any brand or developer to implement beauty AR experiences include:
- AlgoFace, a division of Algo, Inc., is developing cutting edge technology at the intersection of Artificial Intelligence, computer vision, and Augmented Reality for face-related applications.
Banuba, a computer vision lab developing the Face AR technology for mobile and web apps. Apart from Beauty AR SDK, the company also provides face filters, virtual try-on for face-wear, avatars and face analytics.
Face++, a platform offering computer vision technologies, deep learning-based image analysis recognition technologies for mobile applications.
Modiface, the startup acquired by L’Oreal, the pioneer of AI beauty experience and the creator of the first virtual makeover apps for L’Oreal’s subsidiary brands.
Orbo, a visual intelligence company operating in Computer Vision and Deep Learning fields with various market segments, such as OEM mobile devices, consumer cameras, action cameras, drones, and surveillance.
Perfect Corp, the creator of award-winning YouCam Makeup makeover studio, YouCam Perfect selfie cam, YouCam Fun animated selfie filters, and YouCam Nails manicure salon. Perfect Corp provides SDK for small, medium, and enterprise businesses.
Visage Technologies, a private company making computer vision software for face tracking, analysis, and recognition, as well as computer vision for automotive applications.
VOIR, a technology company founded in 2016 with a focus on virtual makeup try-on.
Virtual Makeup SDK Comparison
To compare Virtual Makeup SDKs, I’ve tested the real-world apps and demos available on the companies’ websites. The comparison results don’t claim to be an expert opinion and include my subjective perception of the experience. The images supporting my observations are screenshots taken during testing on an iPhone 7.
Devices and platforms used for testing:
iPhone 7.
Android Pixel 2.
Web (Chrome, Safari).
What Virtual Makeup SDK Features Were Tested?
Virtual Makeup Component
The virtual makeup section lists the items allowed to try within SDK e.g. highlighter, blush, and foundation to the skin, lipstick to lips, and eye shadows, eyeliner, eyelashes (mascara) to eyes.
Beautification
On the surface, face beautification in virtual makeovers may seem non-essential. Indeed, isn’t it supposed to be about makeup only? In practice, it’s not. With slight skin smoothing and lightings adjustment, the makeup looks and fits organically, which makes the overall experience more enjoyable and AR products — sellable.
Face Modification
The more varied functionality of the Virtual Makeup SDK allows brands and developers to introduce new features quickly while keeping abreast of consumer demand. Face modification features like eye recoloring or virtual accessory try-on can be great add-ons to the core makeup functionality.
Hair Recolor
The hair recolor feature was estimated based on accuracy and realism. Most of the virtual SDKs provide accuracy and a variety of juicy colors to try (and they look amazing), yet only Modiface nails the true-to-life representation of the virtual experience.
Performance
Virtual Makeup SDKs are enabled with face tracking and 3D renderer. The face tracking algorithms extract facial parameters, e.g. the lips or eyebrows location to “fit” the virtual makeup in real-time. With optimized face tracking, the virtual makeup applies accurately, while the low-quality face tracking performance results in lagging, makeup dispositioning and jitter.
The 3D renderer component deals with the realistic graphical representation of AR makeup. The gloss of lipstick reacts to dynamic light or shadows with head-down tilts.
The performance was estimated based on the accuracy of makeup fit, speed, level of jitter, overall realism, and quality of the user experience.
Comparison Results and Demo Links
ModiFace
ModiFace has come as a winner and rightfully justifies its reputation of L’Oreal’s top choice and the best beauty AR company.
The lipsticks are all real items with a good representation of gloss. The hair recolor seems the most realistic, providing detailed hair texture and natural color transfer. The other SDKs provide the rough hair “overlays” that can be great in entertainment apps.
ModiFace demo apps:
Banuba
Banuba’s Virtual Makeup SDK provides the most versatile beauty AR features with fast and accurate performance. Makeup try-on, the most complete beautification, hair recolor, and face modification options are included.
The performance and colors on the iPhone look more appealing when compared to Android. The skin and eye makeup components look fantastic and feel real. Few colors are presented for lipstick (matte pallet). However, it might be fixed on the renderer side.
Banuba demo apps:
Perfect Corp
Perfect Corp is a great showcase of how beautification can support the virtual makeup try-on experience, especially on the web. The makeup fits accurately with a variety of colors and tones. The hair recolor is also amazing — you can play for hours with it.
The company keeps its pricing policy opened with package breaks for small, medium, and enterprise businesses.
Perfect Corp demo apps:
VOIR
For now, VOIR supports only iOS; however, Android is promised to come soon. The iOS experience is amazing with a huge variety of colors and options to try. The makeup is applied accurately and feels realistic.
The hair segmentation needs improvements. Compared with other demos, VOIR was the least accurate on the edges with a high level of jitter and rough texture. All the rest, a nice makeover for iOS.
VOIR demo app:
Makeover iOS
Visage Technologies
The live web demo allows applying mascara, lipstick, eyeshadow, eyeliner, and blush. The overall experience is satisfactory with accurate lips’ segmentation and makeup fitting. However, the high level of jitter is distracting for user experience.
The try-on features seem to have no dynamic lighting support and don’t really look realistic. The beautification component is missing too.
Visage Technologies demo:
AlgoFace
The live web performance requires improvements. The high level of jitter, inaccurate makeup fitting, and frequent lags may be cased with poor face detection and tracking components.
Web demo tested on the mobile browser showed similar performance issues with slightly better visual perception due to the lighting adjustment of the mobile camera.
AlgoFace demo:
Face++
Face ++ provides no makeup virtual try-on features, only video beautification with AR filters. The company keeps its pricing open, listing 35,000 for the beautification component. There is no live demo to estimate the performance and no detailed functionality provided.
Orbo
There’s no information about the device and platform support on the company’s website. The beautification features are guessed from gifs on the site. There is no live demo to estimate the performance and no detailed functionality provided.
Virtual Makeup SDK Comparison Table and Summery
Company |
Virtual Makeup |
Beautification |
Face Modification |
Hair Recolor |
Performance |
Platforms |
AlgoFace |
Foundation Blush Eyeshadow Mascara Lipstick |
NO |
NO |
NO |
High jitter, inaccurate lips, glitches with glasses, no dynamic lighting |
Web |
Banuba |
Foundation Blush Highlighter Eyeshadow Eyeliner Mascara Lipstick |
YES Skin smooth, Blemish removal, Expressive eyes, Teeth whitening, Improve lighting and more |
Eyes, lips, brows reshape and recolor, Face morphing |
YES accurate, smooth edges, no face occlusion, rough hair texture |
Low jitter, accurate makeup fit, glasses support, dynamic lighting, realistic look & feel |
iOS, Android, Windows, Web |
Face++ |
NO |
YES, N/A | NO |
NO |
NO DEMO |
iOS, Android |
ModiFace |
Foundation Blush Eyeshadow Eyeliner Mascara Lipstick |
NO |
NO |
YES accurate, realistic recolor, no face occlusion, detailed hair texture |
Medium jitter, accurate makeup fit, variety of matte & gloss colors, dynamic lightning |
Android, iOS, Windows, Web (photo) |
Orbo |
Foundation Blush Eyeshadow Lipstick Mascara |
YES, N/A | NO |
NO |
NO DEMO |
N/A |
Perfect Corp. |
Foundation Blush Highlighter Bronzer Eyeshadow Eyeliner Mascara Lipstick |
YES Skin smoothing, Improve lighting, Morphing |
Eye recolor and lens |
YES accurate, no overlaps on the face, bright colors, rough hair texture |
Glasses support, accurate makeup fit, dynamic lighting, medium jitter, realistic colors from matte to gloss |
iOS, Android, Web |
Visage Technologies |
Mascara Lipstick Eyeshadow Eyeliner Blush |
NO |
NO |
NO |
High jitter, accurate lips, no dynamic lighting support, not realistic |
iOS, Android, Linux, Windows, HTML5, OS X |
VOIR |
Lipstick Foundation Eyeshadow Blush Highlighter Contour Bronzer |
YES, N/A |
NO |
YES high jitter, not accurate edges, overlaps the face and background, rough hair texture |
Accurate makeup fit, dynamic lighting, medium jitter, realistic colors from matte to gloss |
iOS |
Best Virtual Makeup: ModiFace, Banuba, Perfect Corp, VOIR
Best Beautification: Banuba, Perfect Corp.
Best Face Modification: Banuba, Perfect Corp.
Best Hair Recolor: ModiFace
Other Features
Company |
Other features |
AlgoFace |
20 facial traits detection (age, gender, ethnicity, skin tone, hair color, size and shape of the face, eyes, nose, lips, and more) |
Banuba |
Skin + hair color detection Face filters, Avatars and 3D emojis, Video background removal Virtual try on for glasses, jewelry, hats, etc Age, gender recognition, eye tracking, hand tracking |
Face++ |
Face recognition, human body detection, portrait segmentation, AR Filters |
ModiFace |
68 non-identifying parameter detection (lip and eye edges, iris size and location, head pose, skin features including spots, texture, and wrinkles) |
Orbo |
Image processing, object removal |
Perfect Corp. |
Face analytics for AR skin analysis, AI smart shade finder, AI product recommendation Virtual try-on for glasses on Web |
Visage Technologies |
Face analytics for age, gender recognition, emotion detection Eye gaze tracking, iris radius Ear tracker for virtual earings try on Try on for glasses |
VOIR |
Skin tone detection, eyes, and brows tracking |
Further Reading
Opinions expressed by DZone contributors are their own.
Comments