Sunday, December 4, 2016

Vlog #3: Audio/Video Synchronization Issues

Episode #3 of the ITS Gaming vlog! Audio drift has improved, making audio/video synchronization easier. This means more gaming videos coming up soon!

The Don't Starve Together session had an audio/video drift of at least 8 seconds and as high as 17 seconds. In our latest test, we reduced this drift to only 1 second. The drift was being caused by straining limited resources, in particular RAM. Increasing system memory from 4GB to 8GB seems to have improved the situation. We are also recording in shorter sessions, to mitigate the effects of any drift.

But, an audio/video drift of 1 second is still noticeable. If you place a block in Minecraft, you won't hear the sound effect for an entire second. So we will still need to correct the drift.

The gameplay footage also recorded the computer's clock. Using this as a somewhat stable reference point, Mike discovered that the video is not drifting. But, the audio is shorter than the video by 1 second. This means that we will need to stretch the audio to fit the video.

Unfortunately, Blackmagic Design's DaVinci Resolve does not support stretching audio clips. So, we will need to synchronize the audio/video in REAPER. This isn't bad though: REAPER has excellent time stretching tools.

One thing REAPER is not good at is video. This isn't much of a surprise, considering REAPER is a DAW (digital audio workstation). Emphasis on the "audio" part. We can't import our gameplay or facecam footage into REAPER. To see what is happening along with the audio, we will need to transcode the video files into a format that REAPER can digest.

The video files play back sluggishly, especially when viewing all three at once. To make video editing easier, we will be reducing the quality of the videos. These low-quality video files are called "proxies". Proxy workflows for video editing aren't new, and the tools we have support it. Once editing is finished, the original high quality video files can be swapped back in. The end result has no hint that a low quality video was used in the editing process.

We are currently synchronizing and editing a test session of Minecraft. Once the editing is finished, there should be a new video published. Stay tuned!

Friday, December 2, 2016

A behind-the-scenes look at the automated dialogue replacement process that we went through for the upcoming vlog! The vlog will be posted shortly as well. Stay tuned!

Automated Dialogue Replacement (ADR) is a process to recover from bad audio recordings. You might have good footage, but the room you recorded in was too noisy and you can't make out what is being said. ADR fixes this problem by completely replacing the audio with a clean copy.

There are two types of ADR. The one you choose will depend on your workflow and what your end goal is. In the first type of ADR, the talent watches a looping section of footage where the audio needs to be replaced. Usually they are given a buffer at the beginning so that they have time to get ready to record a new take. The buffer may include a series of beeps or some other audible warning. They then recite their lines in sync with the video. During this process, they cannot hear the original audio. This is useful if you want to get a different performance. It's also useful if the mimicked nature of the second ADR method is too obvious.

In the second type of ADR, you setup a looping section like in the first type. But, this time the talent can hear the original audio. Their goal is to repeat the lines over and over again. They get to the point where they aren't consciously speaking individual words. This is usually much easier to sync with the footage, but can sound mono-tone.