Sequential pattern analysis with right granularity

Sequential pattern analysis targets on finding statistically relevant temporal structures where the values are delivered in a sequence. This is a fundamental problem in data mining with diversified applications in many science and business fields, such as multimedia analysis (motion gesture/video sequence recognition), marketing analytics (buying path prediction), and financial modelling (trend of stock prices). Given the overwhelming scale and the heterogeneous nature of the sequential data, new techniques for sequential pattern analysis are required to derive competitive advantages and unlock the power of the big data. In this dissertation, we develop novel approaches for sequential pattern analysis with applications in dynamic business environments, including operation and management tasks in healthcare industry as well as B2B (Business-to-Business) marketing. Our major contribution is to identify the right granularity for sequential pattern analysis, including both sequential pattern modelling and mining. Due to space limitation, this submission presents mainly the 'temporal skeletonization', our approach to identifying the meaningful granularity for sequential pattern mining. Our key idea is to summarize the temporal correlations in an undirected graph. Then, the 'skeleton' of the graph serves as a higher granularity on which hidden temporal patterns are more likely to be identified. In the meantime, the embedding topology of the graph allows us to translate the rich temporal content into a metric space. This opens up new possibilities to explore, quantify, and visualize sequential data. Our approach has shown to provide substantial improvements over the state-of-the-art methods in challenging tasks of sequential pattern mining and sequence clustering. Evaluation on a Business-to-Business (B2B) marketing application demonstrates that our approach can effectively discover critical buying paths from noisy customer event data.