UGTK / Toolkit / Gameplay Systems / Dynamic Track Audio System Completed

Dynamic Track Audio System

Summary

The Dynamic Track Audio System analyses an audio track (resolved from its trackId through the Audio Manager / Audio Database) to auto-detect musical segments and syncopation peaks via an envelope/novelty analysis. It owns no AudioSource: it stays synced to Mn_AudioManager playback events and fires segment and syncopation events as the track plays.

Modules Dependencies

Audio System: playback is delegated to Mn_AudioManager.Instance, and the player subscribes to its AudioPlayed/Paused/Resumed/Stopped events (trackId uses the AudioDatabaseKey attribute).
Database System: optional SO_AudioDatabase used to resolve the AudioClip from the id when it is not currently playing.

Setup

The idea is that one piece of music is cut into segments, and the game decides which segment plays next rather than restarting the track.

Add the TrackSegment Controller prefab, which carries Mn_DynamicTrackController.
Give it the clips, one per segment, and set where each segment starts and ends.
Have Mn_DynamicTrackSegmentPlayer play them; AudioSegmentUtils does the sample arithmetic that turns your times into positions inside the clip.

Set the loop points in samples, not seconds, whenever a transition has to be inaudible. A segment boundary that lands a few milliseconds off produces a click on every repeat, and a click that only appears on the fourth loop is very hard to attribute later.

MusicSceneTest is the scene to start from; otherscene and otherotherscene exist to prove that the music keeps playing across a scene load.

How To Use

Add Mn_DynamicTrackSegmentPlayer to a GameObject and set the trackId (picked from the Audio Manager database) and, optionally, the databaseOptional reference.
Tune the analysis: autoDetectSegments, segmentCount, sensitivity, minSyncGapMs, minSegmentGapMs, and the advanced envelope/threshold fields.
Subscribe to OnSegmentStarted(index, start, end), OnSegmentEnded(index) and OnSyncopationDetected(time).
Optionally drive it with Mn_DynamicTrackController, which forwards PlayFromStart, TogglePlay, StopPlayback, Next/PrevSegment, SetTrackId and the sensitivity/threshold setters to the player.
Play the track through the Audio Manager; the player advances only while the manager reports the track is playing.

Testing Scene

Open MusicSceneTest in the module's Debug/ folder.

What to do, in order:

Press PlayFromStart() and let the first segment run to its end.
Call NextSegment() in the middle of a bar and listen to where the change lands.
Call PrevSegment() and then NextSegment() quickly.
Change the segment list and press Rebuild().

Step 2 is the whole point of a dynamic track: a segment change that happens the instant it is requested, rather than on the next musical boundary, is audible as a stumble. The other two scenes in the folder exist to test the change across a scene load.

The recording is the Dynamic Controls block on TrackSegment Controller, which is where the four commands live and where each says what it does to playback and to the analysis.

The scene needs an audio manager, and now has one. The segment player asks Mn_AudioManager for the clip behind its Track Id; without a manager in the scene there was nobody to ask, the console answered Audio database is null and nothing played. The demo now carries the module's own AudioManager prefab, which arrives with the audio database and the three pools already assigned - press Play From Start and the music pool serves a controller and the track runs.

If you build your own scene, that is the piece to remember: one Mn_AudioManager anywhere in it, with a database. The inspector warns you above Track Id when it cannot find one.

Technical Info

Mn_DynamicTrackSegmentPlayer: resolves the clip, runs envelope/novelty segmentation, tracks a playhead aligned to Audio Manager events, and raises the segment/syncopation events.
Mn_DynamicTrackController: thin façade exposing play/seek/segment-navigation and tuning methods for UI or code.
AudioSegmentUtils: audio analysis helpers (envelope, novelty, peak detection).
Mn_DynamicTrackSegmentPlayer_Editor: custom inspector.