Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection

This paper presents the algorithms that task organisers deployed for the automatic Human Behaviour Analysis (HBA) task of the MediaEval 2018. HBA task aims to investigate alternate modalities of video and body-worn acceleration for the detection of speaking status. For unimodal estimation from acceleration, a transfer learning approach, Transductive Parameter Transfer (TPT), which is shown to perform satisfactorily in a similar setting[4] is employed. For the estimation from the video modality, bags of Dense Trajectories were used in a multiple instance learning approach (MILES) [2]. Finally, late fusion is used for combining the outputs from both modalities. The multi-modal approach resulted in a mean AUC of 0.658, outperforming the performance of both single modality approaches. Copyright held by the owner/author(s).