An NER-based product identification and lucene-based product linking approach to CPROD1 challenge: Description of submission system to CPROD1 Challenge

This paper presents our methodology for CPROD1 Challenge, which is to identify the product mentions from text and then link the product to the entries in the catalog file. Our solution follows 2 steps. First, we use processing pipelines to extract product mentions by incorporating multiple techniques including traditional named entities recognition (NER), regular expression rules and gazetteer-based string matching. Second, we view product linking task into an information retrieval (IR) problem, where the description catalog file is populated into a database. Thus, each product mention acts as a search query and the returned results from catalog entry database serve as the links. The F1 scores of our submission on public and private test data are 24.82% and 16.04%, respectively.