Modeling and recognizing human trajectories with beta process hidden Markov models

Abstract Trajectory-based human activity recognition aims at understanding human behaviors in video sequences, which is important for intelligent surveillance. Some existing approaches to this problem, e.g., the hierarchical Dirichlet process hidden Markov models (HDP-HMM), have a severe limitation, namely the motions are shared among trajectories from the same activity and not shared among activities (classes). To overcome this shortcoming, we propose a new method for modeling human trajectories based on the beta process hidden Markov models (BP-HMM) where the motions are selectively shared among trajectories. All the trajectories from different activities can be jointly modeled with a BP-HMM, which allows motions being shared among activities. Using our technique, the number of available motions and the sharing patterns can be inferred automatically from training data. We develop an efficient Markov chain Monte Carlo algorithm for model training. Experiments on both synthetic and real data sets demonstrate the effectiveness of our approach.