Facebook is developing a new, machine learning powered system that would enable users to re-create the style of text in virtually any photo using just a single word example, which could lead to new developments in translation, editing, replication, etc.
As you can see here, Facebook’s TextStyleBrush project would enable users to replace text in existing images, including background scenes, by using just a single shot, and a single word, as a reference point. The process is a significant advancement on current text replication models, and as noted, could have a range of potential applications.
As explained by Facebook:
“Unlike most AI systems that can do this for well-defined, specialized tasks, TextStyleBrush is the first self-supervised AI model that replaces text in images of both handwriting and scenes – in one shot – using a single example word.”
Facebook specifically notes that the project could help in the translation of text within images, enabling users to even interpret hand-written signs in different languages (example below), while it could also enable users to create personalized messaging and captions, along with other applications.
But it could also be misused. Through this process, people could easily remove watermarks from photos, or they could change elements of actual photos to alter their meaning. What if, for example, someone were to use this to change an old college photo of a political candidate holding a sign, so that he/she was now seemingly promoting an offensive slogan, rather than the actual message?
Facebook is aware of these risks, which is partly why it’s released its research paper on the project, to “spur additional research and dialogue preempting deepfake text attacks”.
“If AI researchers and practitioners can get ahead of adversaries in building this technology, we can learn to better detect this new style of deepfakes and build robust systems to combat them.”
Even so, it seems risky. Handy, for sure, especially if there’s a text style that you might want to re-create, and for evolving translation. But it is possible that the risks outweigh the benefits, at least in terms of a broader public rollout.
Maybe then, Facebook will only ever use it in translation tools, but it does seem like it could have additional value that Facebook will be keen to capitalize on.
Either way, it’s an interesting project – you can read more about the TextStyleBrush project here.
