AI, music, and VR creation instruments collide on this lovely 360° video.

Artificial Intelligence and Virtual Reality are two of probably the most compelling and quickest rising sectors in at this time’s technological panorama. Analysts typically give attention to which of those world-changers are “profitable” or which is “higher” for buyers and most of the people to give attention to. One pair of artists, nonetheless, needs to ask “porque no los dos?”

Taryn Southern is an artist, singer, producer and common YouTuber. Danny Bittman is likely one of the most prolific creators of immersive artwork on this planet at this time. And collectively they’ve created Life Support — a bit of music composed and created solely by synthetic intelligence.

Part of YouTube’s VR Creator Lab, the video challenge could be seen in 360° on desktop, cellular gadgets, and VR headsets. We reached out to the duo to search out out the method and particulars behind this one-of-a-kind creative endeavor.

What was the method you used to create this expertise and marry it with the music?

Danny Bittman: Taryn and I first labored intently for a month to outline the story. We finally needed to show somebody who was plugged right into a fantastical movement based mostly immersion system, I.E their life assist system, however we needed to convey that some sort of stability between VR and RR is critical to be able to keep sanity.

From the beginning we needed to create as a lot of the expertise in VR as potential, so we tossed round potential plot concepts then instantly drafted them out in Tilt Brush to get a way of scale and movement. I then used a digital digital camera in Tilt to movie potential 360 digital camera paths of the environments, despatched a reduce to Taryn, and mentioned the place to go from there.

Once we locked our story, I started the method of leaping forwards and backwards between blocks and Tilt Brush by utilizing the Poly integration. The home, metropolis, and floor within the first shot, for instance, have been all made in blocks, whereas the mountains, bushes, and metropolis lights have been painted in Tilt. We additionally used an unreleased app known as AnimVR which allowed us to import our whole scenes into the app and paint animations in VR like these luminescent strings and the bursts of sunshine that the primary character generates.

But I didn’t need to change to a 2D workflow as soon as I imported into Unity, and so I used Virtual Desktop’s show overlay to work in Unity on a digital large display screen. With this method, I used to be in a position to preview my scene in VR whereas nonetheless sustaining management over my variables in Unity. I simply needed to press play, and the scene would seem round me and my digital monitor. I might then use my screens overlay to tweak lighting colours, object scale, and even modify my digital camera path animation, all whereas nonetheless in VR.

I needed to maintain the streak going although, so I used the digital overlay together with Adobe Premiere’s dwell 360 previewer. It was quiet a dream as a conventional filmmaker to make the most of my previous workflows whereas working with this new Medium. Actually, the one distinction between conventional movie modifying and this 360 workflow was that as an alternative of getting a 2D monitor taking on my UI house, my world grew to become my monitor. I had a lot extra room in Adobe for my timeline and coloration correction nodes.

I did need to do all the character and stable object animation in Maya, however ultimately I’d estimate that I accomplished about 70% of the video with a headset on.

Taryn Southern: Danny did a unbelievable job of describing the artwork creation and course and collaborative workflow. I targeted totally on the music – arranging and modifying the instrumentation stems, writing lyrics, and doing my finest to match sounds with the visuals Danny was creating.

What have been a few of the largest challenges you confronted throughout manufacturing and the way did you overcome them?

Southern: Danny might communicate higher to the challenges on the VR creation aspect, however I’d say one of many largest challenges we needed to overcome was determining learn how to please the YouTube viewers. We knew most individuals could be watching this video on their telephones, not on headsets – so we needed to be cognizant to create an awesome visible expertise for them. The draw back, in fact, is then it’s a must to make selections that may negatively impact the headset expertise. Ultimately we needed to make some compromises to accommodate these concerns – however this downside speaks to bigger points that VR creators have in distributing their work throughout a number of platforms that attain totally different audiences.

On the music aspect, my largest problem is making an attempt to successfully “construct” a music in order that it feels dynamic. I felt I might have performed a greater job of that with my first single.

Bittman: Character animation and processing energy have been the most important killers. Rendering 4K stereoscopic 360 isn’t straightforward for a pc to deal with, and so it generally took hours simply to export and take a look at a 30 second shot. I additionally needed to work whereas on different enterprise journeys, and so I did a lot of the last processing on my Razer laptop computer. It surprisingly did an exceptional job, however sooner or later I’ll be working with a number of GPUs. The Character animation was the opposite monster as a result of this was truly my first time doing character animation for an enormous challenge. I’ve in fact rigged and moved characters earlier than, however by no means on this scale. So I needed to spend a great period of time educating myself as I used to be going

Taryn and I additionally have been struggling to make the pictures look good each in 360 and on YouTube in 2D. We knew a big portion of her viewers won’t transfer the world round, and so we needed to ensure the video regarded good from begin to end from one perspective. Because of this, the expertise performs extra like a 180 expertise. However, we have been in a position to construct this limitation into the story to reward those that view it in 360, as a result of the primary character basically jumps out and in of immersive experiences, and so after we’re trying on the character linked to the machine, all the knowledge is in entrance of us, however once they launch right into a digital world like within the flying scene, issues are taking place throughout us.

Why did you resolve to create this challenge?

Southern:  The challenge initially started as an experiment in creating music with AI. I began composing music with synthetic intelligence platforms, after which writing an entire album grew to become a inventive problem. I thens submitted a pitch to YouTube, and was notified that they’d chosen my challenge as one in every of 10 creator grants. As a part of the grant program, I might make a number of immersive 360 movies for the album.

Danny was the primary individual I reached out to. I used to be dying to collaborate with him after seeing his work on-line, he’s loopy gifted. We iterated on a couple of totally different concepts earlier than finally deciding to do a music that explored the connection between people and their digital worlds. Right now, our digital lives primarily exist of e mail and social media updates — nevertheless it’s not a stretch to say that many people really feel depending on and hooked on their digital lives, even to the purpose that their actual lives undergo. That’s the place the idea for “Life Support” was born.

Bittman: I’ve been eager to work on a 360 challenge for some time since I’ve been creating VR artwork since 2016, and I used to be extraordinarily intrigued by Taryn’s AI music idea, so it was an ideal match. Plus, Taryn has been an avid VR consumer because the starting, so I knew the inventive course of could be pleasant since she intuitively understands the medium.

What do you suppose makes this challenge precious/fascinating to the world?

Bittman: At the top of the day, whereas Taryn focuses of music and I give attention to visuals, we’re each simply making an attempt to figure out learn how to accomplice with rising know-how to prototype and polish our concepts sooner. While Taryn labored with Amper’s AI to create Life Support, she nonetheless used her creativity to pick, tweak, and add to the concepts her pc generated. She might focus extra on sculpting her sounds moderately than sourcing the clay so to say. Then on my finish, I used to be in a position to reduce off weeks value of labor by working straight in VR with out having to translate issues on a 2D display screen. While each of our workflows might use extra polish, I consider it’s tasks like ours that may finally assist new creators generate AAA artwork both by themselves or with a couple of mates in a brief period of time. It’s all about diminishing the training curve to creating digital artwork.

Southern: From a technical standpoint, I actually love how this piece incorporates totally different applied sciences for a really combined media really feel.  For occasion, the 2D mind photographs within the refrain are precise fMRI movies of my mind! We additionally created movies of the AI code used to compose the music in a display screen towards the top.


From a philosophical standpoint, I care rather a lot concerning the theme of the music. Last fall, I took 2 weeks off from social media use – deleted the apps from my telephone, stopped checking e mail greater than as soon as a day – and I observed an nearly fast enchancment in my temper and cognitive skills. Ultimately, I’m a know-how fanatic – I do consider that know-how has improved our lives for the higher – however I additionally suppose there’s worth in cautious reflection. How we select to have interaction with our tech is a worthy dialog – notably throughout the VR/AR neighborhood, as what we construct now may have profound impacts on the subsequent era.

Who was all concerned within the creation of this challenge?

Southern: On the video aspect, Danny actually did all of it. He’s a superb visible artist and director. It’s been an honor working with him.
On the music aspect, Amper (the AI) was a most glorious collaborator. Their staff over there was instrumental in serving to me streamline the writing course of, and Ethan Carlson, one in every of my favourite producers, actually helped me take the music to the subsequent stage together with his manufacturing abilities.

I additionally need to shout out Jenn Duong, my producing accomplice on the VR Creator Lab movies – she’s only a kick ass accomplice and pal with nice instincts.

And, in fact, YouTube – with out the grant program and their mentorship, this video wouldn’t have been potential!!

Bittman: I’m certain Taryn has this query answered, however my mother and father helped me beta take a look at our cuts. They each have tried VR, nevertheless it’s nice to listen to their recent perspective on how they needed to go searching the scenes.

What are you engaged on subsequent?

Bittman: Professionally I’m working with a couple of studios to generate immersive worlds with VR creations software, together with serving to different musicians create VR music movies. But I’m additionally engaged on a private challenge concerning the similarities between dream logic and VR cinematics. I’m fascinated by making a VR expertise that flows and conveys symbolism like a dream, however I’m additionally extraordinarily intrigued by the opportunity of turning the challenge into an MR efficiency piece. Nothing is locked but, nevertheless it’s one thing I’m planning to finish earlier than Winter 2018. For probably the most half I’ll be pushing teasers out of my Twitter all year long, together with an extended overdue tutorial collection

Southern: The subsequent VR music video, directed by Jenn Duong and animated by Vladimir Ilic with put up supervision by VR Playhouse, can be launched in early March. Like Life Support, it additionally integrates 2D video into an animated 360 VR world.

After that, I’m engaged on a documentary targeted on the way forward for human intelligence….and I additionally get to launch this album!


Disclaimer: VRScout labored with each artists in a joint partnership with YouTube final yr. 

Image Credits: Danny Bittman 

The put up Song Made By AI Gets Stunning 360° Music Video appeared first on VRScout.

This article sources info from VRScout