Genlock
When working with video or other data feeds in a live context (ex. broadcast, virtual production), it is important to ensure that each device in the processing pipeline outputs exactly one frame for every frame of input, and that all devices process inputs or outputs with steady, predictable timing. This helps to avoid frame drops caused by devices receiving more frames than they can handle at once (buffer overflows) or devices not having any input frames to process (buffer underflows).
This is usually achieved by feeding a signal that pulses at a specified frame rate (known as the genlock signal) into all the devices in the pipeline. When the signal is pulsed, every device outputs one frame. The genlock signal is generated by one of the devices in the pipeline or an external reference clock. Devices synchronized in this manner are said to be genlocked.
The Live Capture package includes features used to synchronize the Unity Engine update loop to an external signal, such as a genlock signal, instead of executing the update loop as fast as possible or at the refresh rate of the output display.
Enabling genlock
Open the Quality Settings by selecting Edit > Project Settings, and then select Quality:
a. Set VSync Count to Don't Sync.
Open the Live Capture Project Settings by selecting Edit > Project Settings, and then select Live Capture:
a. Set Genlock Source to the desired genlock source. There may be additional options that appear based on the selected source; configure them as required.
b. It is recommended to set Frame Rate to match the frame rate of the genlock signal.
The engine should now be genlocked while in play mode.
Working with genlock
Use the Synchronization Window to monitor the genlock status.
Avoiding frame drops
If a genlock signal pulse is received while the previous frame is still being rendered, the frame corresponding to that pulse is skipped and is not rendered. This is known as "dropping a frame". Frame drops are typically caused by inconsistent frame times or high average frame times.
To avoid frame drops, you can try the following optimizations:
- Reduce the number of objects in the scene.
- Reduce the rendering quality or resolution.
- Spread computations out over many frames to average out the performance impact.
Technical Notes
There are a few scripting APIs that may interfere with genlock and should not be used while genlock is active:
- QualitySettings.vSyncCount must be left at the default value (0).
- Application.targetFrameRate must be left at the default value (-1).
- Time.captureDeltaTime should only be set by genlock sources.
In general, it is important to avoid throttling the update loop of the engine elsewhere, as the engine can only meaningfully synchronize to one signal at a time.