- Audio Highlight Detection
- Tuple Matrix Formation
- Tuple Comparison for Detecting Remakes
Audio Highlight Detection
The intensity of a segment of an audio signal is summarized by the root-mean-square value. The audio track of a video is divided into windows of equal size and the rms value is computed for each audio window. From the resulting rms sequence, the rms ratio is computed for successive items in the sequence.
The rms ratio is marked as low when the value is below a user defined threshold. In our implementation, we use 5 as the threshold, and the video frames corresponding to such windows are considered as "important".
Tuple Matrix Formation
For the audio highlighted segments, wavelet transform is computed from the rms signal of the corresponding audio .
RMS waveform of audio signal detected as highlight and its corresponding wavelet transform
This wavelet transformed signal is treated as image and is splitted into windows of small interval both vertically and horizontally. For each window, dominant value is computed and stored which represents tuple image.
Segmenting wavelet signal to form tuples
From these tuples number values are extracted based on the dominant value and tuple matrix is formed.
Tuple formation
Tuple Comparison for Detecting Remakes
We compute two similarity metrics from this tuple matrix. Our first feature captures information about different types of music that occurs in the audio. The tuples in the tuple matrix along with its frequency of occurrences forms our first feature. The frequencies of the intersecting tuples is summed up to form similarity measure. This measure is normalized by dividing it by the total number of tuples in both audios.
Our second feature works with the sequence of tuple in tuple matrices. For each tuple matrix, the number of common tuples from the tuple matrixes are computed. This results in a similarity matrix of size N/c$ where each point corresponds to a tuple. The similarity measure is computed by summing up the exponential values of data from the similarity matrix. This measure is used for identifying remakes which have same music.
Tuple formation