However, we need to wait till the story feature goes live
YouTube, the video sharing platform is testing an AR feature over its stories that are already limited to some creators. Google shared a blog mentioning the addition of an AR green screen effect to the stories in YouTube app. The effect enables the users to replace their background with an effect by extracting the foreground in real time.
The video segmentation is the technology that allows the video creators to separate a background and foreground from any scene. Then, these two different visual layers can be replaced by interesting effects, mood or content to cook the scene. Google is trying to implement the technology in real-time video sharing feature of the YouTube app. This means the stories in YouTube will be able to use the AR green screen effect that replaces the background to add any effect. However, the YouTube stories are still in the beta version of the app, shared only with a group of creators for testing.
In order to add high-quality visual effect in movie scene, the director uses a green screen in the background to add the intended effect in the scene using softwares and tools. Google used the same technique with its stories to enable the users for replacing their background, but without the need for a green screen. In order to drop the background and separate the front object in real time, Google took help from machine learning and neural networks. In the blog, Google explained how it achieved the addition of the AR effect in the stories using the mobile phones with a specific requirement.
However, the effect isn't 100 percent perfect yet, as you can see some ghosting around the edges. Google is still working on the feature to improve it and plans to integrate it with the company’s other AR products in the future. “Our immediate goal is to use the limited rollout in YouTube stories to test our technology on this first set of effects”, Google wrote in the blog. “As we improve and expand our segmentation technology to more labels, we plan to integrate it into Google's broader Augmented Reality services”.