Wednesday, March 3, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Neural Networks

Automatic Generation of AI Imagery Reacting to Music

April 14, 2020
in Neural Networks
Automatic Generation of AI Imagery Reacting to Music
585
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter
Source

At the Barbican Center in London, on October 31st, 2018, London Contemporary Orchestra will play two great pieces of contemporary classical music by Giacinto Scelsi and John Luther Adams. However, there is something new and unique about this event. In unprecedented step, the music will be accompanied by visuals that are automatically generated using an artificial intelligence algorithm in response to live music with aesthetics inspired by nature.

Displaying visuals to accompany music in concerts is typical in live pop-music concerts, and sometimes in classical music concerts too. However, such visuals are typically prepared offline and scripted to play along the music and mostly are not reactive to the music. Making such visuals usually requires lengthy work by computer graphics and animation experts using various software tools and elaborate effort to give the illusion that the visuals are synchronized with the music. In offline scenarios, when making a music video or a movie, art directors and computer animation and graphic designers have the time to generate visuals that synchronize with music, however this is such an intricate and time-demanding process. The 1940 Disney movie Fantasia was groundbreaking in that front, where around 1000 artists and technicians worked in making it.

You might also like

The Symbolic World: Raising A Turing’s Child Machine (1/2) | by Puttatida Mahapattanakul | Feb, 2021

The Ways in Which Big Data can Transform Talent Management and Human Resources | by Amelia Jackson | Feb, 2021

Why small businesses and startups should always use Analytics and AI | by Yogesh Chauhan | Feb, 2021

What is happening at the Barbican event is quite different. The visual art is generated by an artificial intelligence algorithm in real-time in response to live music as it is played. The algorithm generates images with aesthetics from nature and uses the music as inspiration for the generation.

Jobs in ML

At Artrendex we developed Music2Art, a generative model that takes music as inspiration to make art. The algorithm uses a novel generative model that learn the aesthetics from a collection of images or videos offline. At the concert, in real-time, the algorithm takes the music stream, decomposes it to basic frequencies and align the audio frequency contents with a representation of the aesthetics that it learned offline. The generative model then renders new images that is directly reactive and synchronized with the music.

Earlier Experiments:

We first experimented Music2Art with AICAN art. AICAN uses Creative Adversarial Network to learn the aesthetics by looking at art from art history and generate art that follows that aesthetics but doesn’t fit existing styles. Music2Art aligns the music content with the modes of variations in the generated art. In this example we used the 5th movement from Giacinto Scelsi, Uaxuctum. We only chose the first frame for the visual.

Visuals generated using Music2Art based on AICAN art, music by Giacinto Scelsi: Uaxuctum, 5th movement

In another example, we tried a very fast music the Sabre Dance from Khachaturian Gayaneh ballet, the results was very colorful and dazzling.

Visuals generated using Music2Art, Music: Sabre Dance from Khachaturian Gayaneh ballet

Nature-Inspired Aesthetics

For the concert with London Contemporary Orchestra, in Collaboration with the Universal Assembly Unit, we trained the system with nature-inspired visuals including videos of volcanos, thunder storms, caves, underwater, and others. The algorithm learns the aesthetics from these sources and use that in its generation. The results is very captivative chromatic visual experience.

This is a trial where we used aesthetics from a cave for the 5th movement of Giacinto Scelsi, Uaxuctum

Visuals generated using Music2Art, music by Giacinto Scelsi: Uaxuctum, 5th movement

In another trial, we used underwater aesthetics to accompany John Luther Adams Become Ocean

John Luther Adams, Become Ocean, underwater aesthetics

At the concert, several other visuals will be used. It is exciting to see what will happen at the concert. We have little control over the generation in real-time. The process were only tested in studio setting with recorded music, never in a concert hall with a live full size orchestra. We cannot predict what will be generated. Anything can go wrong. We hope everything will go right.

The Music

At the concert two great pieces of contemporary classical music will be played. The first is by Italian composer Giacinto Scelsi, titled “Uaxuctum: The Legend of the Maya City, destroyed by the Maya people themselves for religious reasons”. “it’s a work so demanding that this is the first time it has been heard in the UK, even though it was written over 50 years ago” said Huw Humphreys, the Head of Music at the Barbican Center. The second is by American composer John Luther Adams titled “Become Ocean” which musically depicts the world consumed by seas. Alex Ross, the New Yorker music critic described it as “may be the loveliest apocalypse in musical history.”

The AI generated visual element complements the music to give the audience an enchanted multi-sensory experience in a concert titled “Other Worlds.”

Project Credits:

The algorithm is developed by Artrendex Inc, a NY-based startup that builds innovative AI for the creative domain. Art direction is done by London-based Universal Assembly Unit. The development of the project has been the result of over six months of research and development aided by Devin Gharakhanian, Benjamin Heim and Simon Hendry. The concert will be performed by London Contemporary Orchestra and Choir, conducted by Robert Ames.

Credit: BecomingHuman By: Ahmed Elgammal

Previous Post

Dell releases new tool to detect BIOS attacks

Next Post

Deltec Bank Bahamas says Customer Experience in Banking in Augmented with Artificial Intelligence - Press Release

Related Posts

The Symbolic World: Raising A Turing’s Child Machine (1/2) | by Puttatida Mahapattanakul | Feb, 2021
Neural Networks

The Symbolic World: Raising A Turing’s Child Machine (1/2) | by Puttatida Mahapattanakul | Feb, 2021

March 3, 2021
The Ways in Which Big Data can Transform Talent Management and Human Resources | by Amelia Jackson | Feb, 2021
Neural Networks

The Ways in Which Big Data can Transform Talent Management and Human Resources | by Amelia Jackson | Feb, 2021

March 3, 2021
Why small businesses and startups should always use Analytics and AI | by Yogesh Chauhan | Feb, 2021
Neural Networks

Why small businesses and startups should always use Analytics and AI | by Yogesh Chauhan | Feb, 2021

March 2, 2021
Data Annotation Service: a Potential and Problematic Industry Behind AI | by ByteBridge
Neural Networks

Data Annotation Service: a Potential and Problematic Industry Behind AI | by ByteBridge

March 2, 2021
Can India beat the global AI challenge? Can we avoid huge job extinction here? | by Yogesh Chauhan | Jan, 2021
Neural Networks

Can India beat the global AI challenge? Can we avoid huge job extinction here? | by Yogesh Chauhan | Jan, 2021

March 2, 2021
Next Post
Deltec Bank Bahamas says Customer Experience in Banking in Augmented with Artificial Intelligence – Press Release

Deltec Bank Bahamas says Customer Experience in Banking in Augmented with Artificial Intelligence - Press Release

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Microsoft account hijack vulnerability earns bug bounty hunter $50,000
Internet Security

Microsoft account hijack vulnerability earns bug bounty hunter $50,000

March 3, 2021
New Chrome 0-day Bug Under Active Attacks – Update Your Browser ASAP!
Internet Privacy

New Chrome 0-day Bug Under Active Attacks – Update Your Browser ASAP!

March 3, 2021
6 Ways Machine Learning Can Improve Supply Chain’s Bottom Line
Machine Learning

6 Ways Machine Learning Can Improve Supply Chain’s Bottom Line

March 3, 2021
Malaysia Airlines suffers data security ‘incident’ spanning nine years
Internet Security

Malaysia Airlines suffers data security ‘incident’ spanning nine years

March 3, 2021
URGENT — 4 Actively Exploited 0-Day Flaws Found in Microsoft Exchange
Internet Privacy

URGENT — 4 Actively Exploited 0-Day Flaws Found in Microsoft Exchange

March 3, 2021
This Protein Therapeutics Company Integrates Wet Lab For High-Speed Characterization With Machine Learning Technologies To Guide The Search For Better Antibodies
Machine Learning

This Protein Therapeutics Company Integrates Wet Lab For High-Speed Characterization With Machine Learning Technologies To Guide The Search For Better Antibodies

March 3, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • Microsoft account hijack vulnerability earns bug bounty hunter $50,000 March 3, 2021
  • New Chrome 0-day Bug Under Active Attacks – Update Your Browser ASAP! March 3, 2021
  • 6 Ways Machine Learning Can Improve Supply Chain’s Bottom Line March 3, 2021
  • Malaysia Airlines suffers data security ‘incident’ spanning nine years March 3, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates