- September 12, 2018
- Posted by: vmayo
- Categories: Digital Marketing, Technology
There are so many memes on Facebook and Instagram that the company has enlisted its artificial intelligence to help understand them. Facebook developers say they have created a dedicated AI tool, called Rosetta. It read the text that appears in memes (and other images and video frames) that shared to Facebook and Instagram.
At face value, understanding memes might not seem like the most important problem for AI to solve. But Facebook‘s researchers point out that the technology, which designed to recognize depicted text in a wide variety of languages, has many practical uses.
Why Understand Text in Images and Videos
Understanding the text that appears on images is important for improving experiences. Such as a more relevant photo search or the incorporation of text into screen readers that make Facebook more accessible for the visually impaired. Understanding text in images along with the context in which it appears also helps our systems proactively identify inappropriate or harmful content and keep our community safe.
A significant number of the photos shared on Facebook and Instagram contain text in various forms. It might be overlaid on an image in a meme, or inlaid in a photo of a storefront, street sign, or restaurant menu. Rosetta extracts text from more than a billion public Facebook and Instagram images and video frames (In Different languages), daily and in real time.
How Rosetta AI Perform Extraction
There are two independent steps to perform text extraction. In the first step, Rosetta detects rectangular regions that contain text and then uses text recognition to identify what the text actually says. Once the text has been transcribed, the system interprets what the text could mean. By using Rosetta, Facebook will able to improve its image search as well as the systems. That determine the types of images that may appear in your News Feed. It also helps the company automatically detect and remove hate speech. Facebook says it could apply the same technology towards understanding text that appears in the video as well. Though that requires a more complex system.
Whenever we surf on Facebook or Instagram, many time we have to face the problem of understanding memes. There are memes in different languages and understanding them is not easy for a user. So now it would become easier for us to understand texts on images with Rosetta Artificial Intelligence tool. It will be easy for us to understand the texts on the images. It helps us to prevent the harmful or offensive content that it may not have been able to identify previously.
To know more about how Rosetta AI works, please visit https://code.fb.com/ai-research/rosetta-understanding-text-in-images-and-videos-with-machine-learning/