Author Topic: What is the best way sync video and audio?  (Read 5570 times)

0 Members and 1 Guest are viewing this topic.

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
What is the best way sync video and audio?
« on: July 28, 2011, 04:32:34 pm »
Hi,
I am contemplating going from just doing audio recordings to adding in video. Thus my questions with some up front assumptions on my part. Feel free to correct where I am off base:

* If one records using mics other than the one on the camera and those mics are fed directly into your DAW, you now have a good audio source or at least the potential for such.

* I would capture video and a second sound source to SD card directly into the camera.

* I would use Vegas as the final video and sound merge point once the sound tracks are mastered in the DAW and exported as a .wav file.


What then would be the method to use in making sure both the audio and video are in perfect sync? What would Vegas look for in this regard to line up the sound .wav file to the .mov file coming in from the camera SD card? Is there an actual sync timing signal available from such a setup? Or are you left with just lining up the video frames as best you can in Vegas to the audio .wav files with no sync signal?


I will mention my capture camera is a simple Canon ELPH 300 HS... So as far as I know I do not have a bunch of audio inputs for external mics.


Thanks for any help you can offer,

LB
You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline Tourniquet

  • Honorable Ex-Mod
  • Stadium Superstar
  • *****
  • Posts: 4035
  • Good Vibes 72
  • Time exists in abundance until it runs out
Re: What is the best way sync video and audio?
« Reply #1 on: July 28, 2011, 05:37:10 pm »
Your assumptions are correct.
syncing can be a bit of a headache (has been for me at times)

I've used vegas, but not extensively so I can only say that it can do something, not necessarily tell you how at the moment.

The best way I found was to zoom in until you're looking at the waveform of a fraction of a second.
Your cameras audio will be synchronized with the video already so use it to align the start of your imported audio.

If you're very lucky everything will line up perfectly, more likely is that the audio will start in sync and gradually desync the longer it runs. an unfortunate side effect of using separate devices to record is that their internal clocks don't tend to agree exactly on what constitutes a second.
it's easily fixed, just go to the end of the file and stretch it until the ends of the waveform match up too (think you just drag the end of the file, but you might need to hold a particular key down)
once it's synced with the camera you can decide whether you want to keep the cameras audio or not (i've found it desireable at times) if you do, just balance the volumes to your liking. if you don't, unlink it from the video and delete it.

Last point I'll mention is that you may need to experiment with codecs to find a good balance of quality and file size (I tend to get 50-100mb for acceptable quality over 3-4 minutes). you'll also need to check the file when it's finished rendering, some codecs can desync everything for no good reason.
       Road Case        Singing Primer guide

Offline TB-AV

  • Honorable Ex-Mod
  • All Time Legend
  • *****
  • Posts: 13805
  • Good Vibes 330
Gone

Offline Tourniquet

  • Honorable Ex-Mod
  • Stadium Superstar
  • *****
  • Posts: 4035
  • Good Vibes 72
  • Time exists in abundance until it runs out
Re: What is the best way sync video and audio?
« Reply #3 on: July 28, 2011, 08:58:42 pm »
Is there anything that program won't do.... I've had no end of problems finding a video editor that would handle my camcorders output file while also having a reasonably intuitive interface and a range of rendering variables. seems that's too much to ask as no one program did the job completely.

Had to update to try the video render function but it's handled everything flawlessly. Quite ridiculous that reaper can outdo the lot when it's not even it's primary function. That's several less steps I need to take to make videos now and fewer forehead shaped dents in my walls.

Cheers TB ;)
       Road Case        Singing Primer guide

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
Re: What is the best way sync video and audio?
« Reply #4 on: July 28, 2011, 09:17:58 pm »
Well as usual TB & Trourni are on the money. I can go with this methode and will for lack of a better term call it "the hand clap sync." as it is a good manual sync. cue.

Yet does anyone know if or what is used on profesional stuff? My thinking is there is some sort of track that is devoted to sync. I do see in the DAW a box for "[]" "Externally Clocked". I think that is for syncing possibly to the ADAT Sync...? But is that somehow related to a better camera being able to be synced it to as well? Sounds like it is time to pull out the docs and hurt my brain on some of this stuff.

@ Tourni - Well I agree Reaper can do a lot but if you notice the guy in the video tutorial did little editing of the video. I mean if you want fades, transitions, titles, picture in picture and such stuff I think you need to ues a tool like Vegas...? That brings up another question... where is it best to assemble the final merge of video and audio? Better in Vegas or say in your DAW such as reaper? I can see right now if you start to get fancy it would be real easy to get out of sync in a hurry. I could see where you might want a hand clap cue sync in many spots as you make the video/audio recording any place you feel there might be a video insertion or modification. Makes me think of the old film days when they say "action" and snap the cue board to start and restart the production of the film.

Thanks for the help guys!

LB
You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline Loco

  • Stadium Superstar
  • ******
  • Posts: 1421
  • Good Vibes 21
  • Love is too plebeian.
Re: What is the best way sync video and audio?
« Reply #5 on: July 28, 2011, 09:40:23 pm »
The best way I found was to zoom in until you're looking at the waveform of a fraction of a second.

+1

and also you can pick a muted string instead of clapping hands, it will give you a pretty good reference point in waveform

i've never had any problems with internal clocks being out of sync, though
Prudence is a rich, ugly, old maid courted by incapacity.

Offline Tourniquet

  • Honorable Ex-Mod
  • Stadium Superstar
  • *****
  • Posts: 4035
  • Good Vibes 72
  • Time exists in abundance until it runs out
Re: What is the best way sync video and audio?
« Reply #6 on: July 28, 2011, 10:28:34 pm »
@ Tourni - Well I agree Reaper can do a lot but if you notice the guy in the video tutorial did little editing of the video. I mean if you want fades, transitions, titles, picture in picture and such stuff I think you need to ues a tool like Vegas...? That brings up another question... where is it best to assemble the final merge of video and audio? Better in Vegas or say in your DAW such as reaper?

Depends on your needs. I don't tend to use any video editing beyond trimming so Reaper can cater to all my needs.
If you want anything more fancy you're better served doing all video work in Vegas.

Just keep in mind that every time a video gets rendered, detail gets lost. (same goes for audio for that matter).
Think of it like photocopying a photocopy of a photocopy of a..... you get the idea. With a video it can degrade much quicker.

i've never had any problems with internal clocks being out of sync, though

Try recording with your computer, a portable audio recorder, your mobile and a camcorder. import all the audio files to a multitrack, sync the starts and then get ready for your ears to be offended. ;)
       Road Case        Singing Primer guide

Offline Loco

  • Stadium Superstar
  • ******
  • Posts: 1421
  • Good Vibes 21
  • Love is too plebeian.
Re: What is the best way sync video and audio?
« Reply #7 on: July 28, 2011, 10:39:38 pm »
it might be the fact that my ears are offense-proof when it comes to slight sound nuances... ::) ;D
Prudence is a rich, ugly, old maid courted by incapacity.

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
Re: What is the best way sync video and audio?
« Reply #8 on: July 28, 2011, 11:14:39 pm »
Loco,
My ears are shot after years of abuse... sigh, so a lot can get by mine.

@ T - Yep, makes sense on over rendering things. My thinking was you get the sound sorted out in the DAW and then Work the video over in Vegas. Once that is done you bring in the finished audio and stitch or mary the two together. If you are just doing a straight forward take like playing and singing a song at the same time. Then it may require little video post processing other than the trim, just as you say. All good stuff and if I figure out the timming signal deal I will post here what I find. I just think that broadcast wav files and video have a true sync track or signal that each are pegged off of. The pros have to have more than a visual cue to do their thing I would think.

LB
You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline TB-AV

  • Honorable Ex-Mod
  • All Time Legend
  • *****
  • Posts: 13805
  • Good Vibes 330
Re: What is the best way sync video and audio?
« Reply #9 on: July 29, 2011, 01:25:43 am »
Quote
Is there anything that program won't do.... I've had no end of problems finding a video editor that would handle my camcorders output file while also having a reasonably intuitive interface and a range of rendering variables. seems that's too much to ask as no one program did the job completely.

Vegas is pretty much format agnostic. You can trow most anything at it and it will play in the time line. It is a fantastic and powerful program.

Also the generational loss you mention. That is not necessarily the case if you take care of your initial footage. Obviously if you are compressing, something has to give but Vegas has a very good codec that is totally customizable. It's made by Main Concept.

LBro, I would render in Vegas. There are tutorials on Youtube. also you may want to check out Creative Cow Vegas forum. There are several smart people there but if Douglas Spotted Eagle is still over there, he is a Vegas master... not to mention musician and videographer, Grammy winner, and all round nice guy.

Vegas and Reaper are complimentary and have a very similar feel.

Gone

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
Re: What is the best way sync video and audio?
« Reply #10 on: July 29, 2011, 03:00:30 am »
TB,
Thanks for the great info and scoop. As I started to dig into this a couple things become clear real quick.
1. This is a deep subject once you start looking at  PTS and DTS Mpeg time stamping. From what I gather there is a some pretty nasty stuff going on and it looks like for SURE I do not have this on my gear, at least on the camera side.
2. The clap methode and or muted string or whatever cue one uses can be a sync track in of itself just as TB and others have stated. That track would be the audio recorded on the camera video itself it seems. I can see this getting a bit complex as well if one for instance builds up an audio track starting with drums, then adding rthm. guitar/bass/keys and what not. Then playing that back and laying down vocals with video of them. Later if you add in video of the guitar lead parts it starts to get to be a lot of work... Thus I tip my hat to those doing well on here. I can see a need for not only multiple mics, but also cameras to pull off a live gig. Lol, that shoots the budget all to heck.

Here is a good technical link on the issue and I did find others as well. Now my brain hurts!
http://www.nhk.or.jp/strl/publica/bt/en/le0011.pdf

LB


You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline TB-AV

  • Honorable Ex-Mod
  • All Time Legend
  • *****
  • Posts: 13805
  • Good Vibes 330
Re: What is the best way sync video and audio?
« Reply #11 on: July 29, 2011, 04:36:27 am »
Man, what in teh world are you trying to do. Just record your performance on your camera. Audio and Video.


Then lock your new audio to the camera audio. How ever many tracks it doesn't matter.

Mix down then mute camera audio track.

Render original video and mixed audio.

Post on YT

Gone

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
Re: What is the best way sync video and audio?
« Reply #12 on: July 29, 2011, 04:55:28 am »
Well TB you do have a point and I can at times be accused of trying to make things too difficult. lol, but a lot of time I assemble audio in multiple takes... so my thinking was to relate video in the same fashion. I mean what if you are working in a virtual setting with a couple others? So I just wanted to find out how best to go about not just a one off video of self but also understand what it would take to go further. The answer still is just as you say and to have a cue track like in your first post. Anything more than that is out of my reach and overkill for what I am doing or plan to do.

One area of note, I did find a spec where by most stuff would not be seen as out of sync if you can stay within about 15 ms of each other. My hunch is it could be out further than that and it just depends on how the video is taken and a few lesser factors...

LB
You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline Tourniquet

  • Honorable Ex-Mod
  • Stadium Superstar
  • *****
  • Posts: 4035
  • Good Vibes 72
  • Time exists in abundance until it runs out
Re: What is the best way sync video and audio?
« Reply #13 on: July 29, 2011, 08:59:38 am »
Vegas is pretty much format agnostic. You can trow most anything at it and it will play in the time line. It is a fantastic and powerful program.
In theory it should handle the format (VOB). in practice it throws a wobbly every time (rather amusing as it's a sony camcorder).

Also the generational loss you mention. That is not necessarily the case if you take care of your initial footage. Obviously if you are compressing, something has to give but Vegas has a very good codec that is totally customizable. It's made by Main Concept.
If you get everything just right it can minimise the loss, but it's still going to loose something in translation (nothing is truly lossless), not enough to notice on the first pass though.

2. The clap methode and or muted string or whatever cue one uses can be a sync track in of itself just as TB and others have stated. That track would be the audio recorded on the camera video itself it seems. I can see this getting a bit complex as well if one for instance builds up an audio track starting with drums, then adding rthm. guitar/bass/keys and what not. Then playing that back and laying down vocals with video of them. Later if you add in video of the guitar lead parts it starts to get to be a lot of work... Thus I tip my hat to those doing well on here. I can see a need for not only multiple mics, but also cameras to pull off a live gig. Lol, that shoots the budget all to heck.

one of reapers neat little tricks is that you can set the BPM for the grid. so long as you've played everything to a set tempo and trim each file to it's audio start everything snaps neatly into beat without fine tuning.

I've done multiple takes for solos over previous drum and rhythm tracks.

Record or create drum track.
put on a pair a headphones so you can head your drum track from reaper.
create a new track and record rhythm over drums.
Nothing to sync as you've recorded directly to the DAW at the right time.
create a new track and record Lead over drums & rhythm.
Again, nothing to sync.

If it's files from other people, If they use reaper and record as above the project file can be passed around and still no sync work to be done.
If all you get is an audio file you've gotta get up close and make minor adjustments until it sound right.
       Road Case        Singing Primer guide

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
Re: What is the best way sync video and audio?
« Reply #14 on: July 29, 2011, 09:34:37 am »
Hi Tourni,
Yes I have done just what your saying with my DAW. In that I lay down a drum track first. It will be right on the money of course as it is canned. I almost think of it as the click or audio timming track. Multi audio takes not a problem either. Even if a guitar is off beat a bit the underlying drum track is correct and so far so good.

Hoewever what happens if we have all the instruments (audio) in and done in the DAW except for lead guitar. Then we pass the files back and forth with each now adding a video file of say the lead guitar playing. Then the lead singer with video and finally the backing vocalist does that part with yet another video. Each would not only record with the camera sound, but also a DAW track. So do we simply use a timing audio track/video cue to line them all up in the 'producers' master DAW based on the audio cue? Then once the sound is mastered it all goes into Vegas and those audio/visual cues are used to keep all in sync? Sorry to be dense about this, but I have never done video this way before.

Thanks,

LB
You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline Tourniquet

  • Honorable Ex-Mod
  • Stadium Superstar
  • *****
  • Posts: 4035
  • Good Vibes 72
  • Time exists in abundance until it runs out
Re: What is the best way sync video and audio?
« Reply #15 on: July 29, 2011, 10:57:31 am »
You'll need a common Cue which means you'll have to hear part of the backing track on the video.

For example. lets say you use that clap as the cue. you need to hear that but you don't want to hear the rest of the backing track in their recording. You gotta leave enough time between the cue for them to plug in headphones, get to their video mark, gear up if needed, and get settled to start (unless they have a spare pair of hands to plug them in). It can make attempted takes quite long though.

You can sync without a cue, bit time consuming but possible. in that case you render each component of the audio seperately so that each video can sync up to its isolated track rather than the full mix. just import the full mix at the end.
       Road Case        Singing Primer guide

Offline TB-AV

  • Honorable Ex-Mod
  • All Time Legend
  • *****
  • Posts: 13805
  • Good Vibes 330
Re: What is the best way sync video and audio?
« Reply #16 on: July 29, 2011, 01:52:19 pm »
Quote
In theory it should handle the format (VOB). in practice it throws a wobbly every time (rather amusing as it's a sony camcorder).

Try renaming the file to .mpeg

VOB and even MPEG are delivery formats and were never really intended for editing.

About the generational loss.

If your video comes out of your camera as X then that is the best it will ever be. Well actually you can correct it and make it better actually in Vegas. Then you can save it as AVI. It will be a huge file. It will not only be a loss, it will be better. But either way, just save the X or corrected AVI. then do all your rendering from those files.

Now let's say you render them to mpg. If you need to make changes you don't want to go back and edit your rendered mpg, you want to go back to your project X or AVI file and edit / render those. so your next render will still be a first gen off the master. Always work off an unrendered master.

Same with audio, you loose no resolution off your initial wav files.

Obviously the render itself will be a loss and that is why you usually have to jump through a little hoop to get VOB or MPEG on the time line. They are the wrong type files to be editing but these new cameras have started using it ( and people ripping DVDs ).

About playing in sync..... If you take a file no matter what the BPM, if you play in time with it, you are in sync. I'm not sure what you mean about Reaper and setting the BPM. Do mean if someone had a file recorded at 120 BPM and your Reaper project is at 120 BPM they will sync?  That would be try of any daw provided you aligned them. I might not be following what you mean though.




Gone

Offline TB-AV

  • Honorable Ex-Mod
  • All Time Legend
  • *****
  • Posts: 13805
  • Good Vibes 330
Re: What is the best way sync video and audio?
« Reply #17 on: July 29, 2011, 02:17:32 pm »
LBro, if you are going to do cuts which would be typical for most any music video what I would do is this......

that is provided you have exact song, time, timing that you know you want to finish with. IOW, not some sort of free form jam session.

Make a video/audio of the exact song you want finished. As long as it's in time it's fine. This will be your rough.

ROUGH VIDEO / MASTER
-------- Video -- let's pretend this video is mostly extraneous shots like buildings, sunsets or whatever.
-------- Audio -- this audio is basically in time and the near exact finished song

LEAD SINGER VIDEO
-------- Video1 - from Above - they really don;t even need this
-------- Audio1 - from above
---------Video2 - close up of what this guy is going to record - singers video
---------Avdio2-  the singers performance

The lead singer now sends you Vid2/Aud2 and you put it in project like this.

ROUGH VIDEO / Master
-------- Video -- let's pretend this video is mostly extraneous shots like buildings, sunsets or whatever.
-------- Audio -- this audio is basically in time and the near exact finished song
---------Video2 - close up of what this guy is going to record - singers video
---------Avdio2-  the singers performance

repeat for each instrument.

Then edit the video.

Now your initial Rough can also be sent with SMPTE time track if you like and each contributor can simply note the time stamp or samples where their file they send you starts. It all depends on how you want to handle that.

Once you get all your tracks done and every thing is timed up to look nice you render it to  uncompressed AVI ( huge file ). that is your master and can be worked from later if need be with no loss.

So the point is each guy get's his V/A tracks in sync with each other and they in turn will sync with your master.

It would be good for your Rough to have beep spike at the very beginning but to be honest you will probably want to nudge things around by ear to get it all to sit just right. But you will always be moving say Lead Voc Vid/Aud or Bass Vid/Aud. IOW, you will be moving sync'd pairs. You most likely won't disconnect their performance ( unless they screwed up or you start doing special video effects where it doesn''t matter.






Gone

Offline Tourniquet

  • Honorable Ex-Mod
  • Stadium Superstar
  • *****
  • Posts: 4035
  • Good Vibes 72
  • Time exists in abundance until it runs out
Re: What is the best way sync video and audio?
« Reply #18 on: July 29, 2011, 03:08:49 pm »
Try renaming the file to .mpeg

VOB and even MPEG are delivery formats and were never really intended for editing.

Tried that before. helped with some programs, but not vegas. I'm good with reaper handling it now.

About the generational loss.

If your video comes out of your camera as X then that is the best it will ever be. Well actually you can correct it and make it better actually in Vegas. Then you can save it as AVI. It will be a huge file. It will not only be a loss, it will be better. But either way, just save the X or corrected AVI. then do all your rendering from those files.

That was the problem I had before, I had to shove the Vob file into one program (which i didn't like overall but could handle the file) and re-render it just to get it accepted by vegas. doesn't sit right with me processing the file like that but I had no choice at the time.

About playing in sync..... If you take a file no matter what the BPM, if you play in time with it, you are in sync. I'm not sure what you mean about Reaper and setting the BPM. Do mean if someone had a file recorded at 120 BPM and your Reaper project is at 120 BPM they will sync?  That would be try of any daw provided you aligned them. I might not be following what you mean though.

that is what I mean, onces it's lined up to the beat, it'll line up to any beat. half the battle is knowing what the software can do.
       Road Case        Singing Primer guide

Offline LBro

  • Stadium Superstar
  • ******
  • Posts: 3462
  • Good Vibes 109
  • Well I am working on it.
    • YouTube offerings
Re: What is the best way sync video and audio?
« Reply #19 on: July 30, 2011, 10:57:12 am »
TB,
Wow - great and well laid out reply. I thinks I got it now and that really helps. Yes, it makes complete sense that others video and audio SHOULD be in sync with themselves. I will have to copy off this info and put in a recording log book or something for future reff.

A Tourni - Yeah Vegas I have looked at and know little about it. Just getting my feet wet in video editing again. I used to make some family films a long time ago but the tools were down right primitive! On another note I thought Vegas could import just about anything now???

LB
You can rock a bit here:
https://www.youtube.com/channel/UCNOrtongjfX4RI33JWKwn7Q/featured

"I just experiment in guitar and music, at times something good comes of it..."
LB

Offline TB-AV

  • Honorable Ex-Mod
  • All Time Legend
  • *****
  • Posts: 13805
  • Good Vibes 330
Re: What is the best way sync video and audio?
« Reply #20 on: July 30, 2011, 04:36:40 pm »
Vegas generally locks the Vid/Aud tracks as pairs. They can be unlocked, nudged, deleted or whatever. Then re-locked if desired.

Obviously if you have a closeup of someones hands then unlocking them and moving one will not be in sync. However, for artistic purpose and used with other edits and cuts, the final video might look better and the audio tracks can be made to fit the pocket.


Rule of thumb.....

Great video with terrible audio generally sucks. ( see any Youtube video of live concert footage ( ok, the video on those suck too but you get the point ) )

Crappy video with great audio is often quite passable and accepted. ( See Youtube video  --  Beck-Loser )


Yes,, there is very little you can't get on to Vegas timeline. and mixed media as well. Some formats ( typically delivery formats ) might have you jumping through a little hoop or installing another codec but once you get it, it's a done deal.

Vegas is incredibly powerful. If you decide to get serious about it, Douglas Spotted Eagle has a book that covers all the must know basics. Actually he has several books but his Vegas Editing workshop books are very good. You could probably even get an old one an learn 90% of any thing you need.

Vegas forums.....
http://forums.creativecow.net/sonyvegas
John Rofrano and Mike Kujbida are also top notch.

http://vasst.com/home.aspx
There is a mind numbing bit of free Vegas ( and other ) stuff on this site if you set up an account.

Everything in Vegas gets stored as a simple .veg file and there are tons of them there. It's like having a template that you apply your media to.

http://forums.digitalmedianet.com/cgi-bin/displaywwugindex.fcgi?forum=sonic-foundry_vegas

If you can ask a question in those forums and not get an answer then rest assured you have a very serious or unusual problem. Normally you will get about three different methods to do whatever it is you want.

there is of course the Sony forums as well which are also very good.



VEGAS Specs
http://www.sonycreativesoftware.com/vegaspro/techspecs


« Last Edit: July 30, 2011, 05:21:57 pm by TB-AV »
Gone

 

Get The Forum As A Mobile App