As the recent rise in Covid-19 threatens once again to shutter advertising agencies, film studios, and similar media “factories” globally, a quiet, desperate shift is taking place in the creation of new media, brought about by increasingly sophisticated AI capabilities. A new spate of actors and models are making their way to people’s screens, such as pink-haired Imma, right, who has developed an extensive following in Japan on Instagram and TikTok, and is appearing increasingly on the covers of Japanese magazines.
She also doesn’t exist.
Imma joins a growing host of digital avatars who are replacing human actors, models, and photographers with computer-generated equivalents. Cloud-based GPUs and sophisticated game and modeling software have increasingly attracted the attention of a new generation of artist/programmers who are taking advantage of this to generate images, video, and audio that are becoming increasingly indistinguishable from reality, especially when that reality is otherwise captured via jump cuts, and matte overlays that have made tools such as TikTok and Reels the primary tools for video production for the typical Instagram celebrity.
The business potential for such virtual models and spokespeople is huge, according to a recent piece by Bloomberg on digital avatars. Such avatars have obvious benefits over their flesh and blood counterparts. They can appear in print or video anywhere – on a far-off beach, on a busy street in a bustling city, or staring out at dirigibles and flying saucers while taking a taxi above the clouds – without ever having to send a crew out for several days to some otherwise uninhabited Caribbean beach, reserving expensive permits for filming or dealing with observers, or spending a great deal of time with specialized green screen effects.
The models don’t age out of roles, don’t have bad hair days, or become prima donnas. The initial cost to develop such models may be fairly high (though seldom more than the cost of sending models and crew to a tropical island) but once created, that model becomes highly malleable, and can be used in a large number of different situations.
Ordinarily, these changes have been on the horizon for a while, and until the pandemic, the use of such models was increasing slowly anyway. Yet as for so many other things, the pandemic shifted the need for virtual models and actors into overdrive, as social distancing requirements and lockdowns put a very real limit on the ability of creative agencies to put together content with live actors.
It is significant, for instance, that The Walt Disney Company, which had already invested heavily in the use of CGI-based actors and props for its wildly popular series The Mandalorian, was able to finish production on the awaited second season of that show so quickly after California’s Governor Newcom started loosening the limits on production. It did so by taking advantage of the hyperrealistic rendering capabilities of graphical processing units (GPUs) which are now capable of extensive texture mapping, shading, ray tracing, subdermal lighting and so forth by dint of the GPU architectures that were optimized for better gameplay.
Moreover, such processing can readily be done in parallel across multiple dedicated cloud processors, which not only increases the render time dramatically but also makes modeling, rigging (the process of positioning objects relative to one another) and lighting take far less time than they would on a single laptop. This also makes procedural shaders, which handle the animation of everything from hair to skin to dirt to water and smoke, feasible to the home-bound graphic artist or designer. Companies such as nVidia have partnered with cloud providers to make racks of GPUs acting in parallel available to anyone with a decent Internet connection even without necessarily needing to have such GPUs available on the artist’s computers directly.
This process is also driven by the rise of generative adversarial networks (a form of neural network that is able to take related types of images and build subtle composites that can then be evaluated to determine the “verisimilitude” of a given image. Those that survive are then used as the basis for other such images. When used as is, such GANs are remarkably effective at creating realistic portraits of people quickly, while at the same time also providing templates for the generation of models underlying such templates that can then be used for simulating action. The StyleGan2 algorithms drive the site This Person Does Not Exist, which generates unique, very realistic facial images. The site also contains detailed information about the StyleGan2 algorithms involved in the process.
The ability to create such realistic avatars has also led to the rise of fake identities in social media sites, as people take advantage of such tools to either disguise themselves on places as diverse as Facebook and Tinder, or simulate fake profiles for spamming and trolling, or, in at least a couple of cases, creating faked dossiers for political mischief, complete with people who existed only within a GPU.
StyleGan2, in conjunction with similar research for capturing facial orientation and lip-synch movements, are also changing the nature of gaming as the ability to wear avatars that are able to speak and make facial gestures. From pre-generated audio recorded live, or text-to-speech interfaces such as are used to power a host of verbal virtual assistants (Siri being the Ur example), such viseme-matching algorithms are increasingly putting a face to what once had been primarily audio only “digital-companions”. Additionally, the use of GANs can be extended into the third dimension, making it possible for a single 2D photograph to be approximated as a surprisingly complete and sophisticated 3D version.
It is very likely that, for good or ill, this particular area of development will become one of the hottest faces of the AI movement. If the Japanese experience is any indication, such “Virtual Ambassadors” may very well become celebrities in their own rights. In many cases, the environments for creating such virtual models are likely to be the same -platforms that currently are used for developing immersive reality games, such as Microsoft’s Unreal Engine or Reallusion’s Character Creator platform. As the pandemic continues to change how we work, it is likely that the next generation of advertising, media creation and immersive gaming will rely upon these virtual avatars, the next general of digital influencers.
Kurt Cagle is the Community Editor for Data Science Central.
Credit: Data Science Central By: Kurt Cagle
