MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data

Zhifeng Xie; Wenling Zhang; Huiming Ding; Lizhuang Ma

doi:10.1007/978-3-030-47426-3_12

MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data

Zhifeng Xie, Wenling Zhang, Huiming Ding, Lizhuang Ma

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Feature engineering usually needs to excavate dense-and-implicit cross features from multi-filed sparse data. Recently, many state-of-the-art models have been proposed to achieve low-order and high-order feature interactions. However, most of them ignore the importance of cross features and fail to suppress the negative impact of useless features. In this paper, a novel multi-scale feature-crossing attention network (MsFcNET) is proposed to extract dense-and-implicit cross features and learn their importance in the different scales. The model adopts the DIA-LSTM units to construct a new attention calibration architecture, which can adaptively adjust the weights of features in the process of feature interactions. On the other hand, it also integrates a multi-scale feature-crossing module to strengthen the representation ability of cross features from multi-field sparse data. The extensive experimental results on three real-world prediction datasets demonstrate that our proposed model yields superior performance compared with the other state-of-the-art models.

Original language	English
Title of host publication	Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings
Editors	Hady W. Lauw, Ee-Peng Lim, Raymond Chi-Wing Wong, Alexandros Ntoulas, See-Kiong Ng, Sinno Jialin Pan
Publisher	Springer
Pages	142-154
Number of pages	13
ISBN (Print)	9783030474256
DOIs	https://doi.org/10.1007/978-3-030-47426-3_12
State	Published - 2020
Externally published	Yes
Event	24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020 - Singapore, Singapore Duration: 11 May 2020 → 14 May 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12084 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020
Country/Territory	Singapore
City	Singapore
Period	11/05/20 → 14/05/20

Keywords

Attention network
Factorization machines
Feature engineering
Feature interactions

Access to Document

10.1007/978-3-030-47426-3_12

Cite this

Xie, Z., Zhang, W., Ding, H., & Ma, L. (2020). MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data. In H. W. Lauw, E.-P. Lim, R. C.-W. Wong, A. Ntoulas, S.-K. Ng, & S. J. Pan (Eds.), Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings (pp. 142-154). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12084 LNAI). Springer. https://doi.org/10.1007/978-3-030-47426-3_12

Xie, Zhifeng ; Zhang, Wenling ; Ding, Huiming et al. / MsFcNET : Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data. Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings. editor / Hady W. Lauw ; Ee-Peng Lim ; Raymond Chi-Wing Wong ; Alexandros Ntoulas ; See-Kiong Ng ; Sinno Jialin Pan. Springer, 2020. pp. 142-154 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d9b4ab5d5b7e4e6aa14ca518060024b9,

title = "MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data",

abstract = "Feature engineering usually needs to excavate dense-and-implicit cross features from multi-filed sparse data. Recently, many state-of-the-art models have been proposed to achieve low-order and high-order feature interactions. However, most of them ignore the importance of cross features and fail to suppress the negative impact of useless features. In this paper, a novel multi-scale feature-crossing attention network (MsFcNET) is proposed to extract dense-and-implicit cross features and learn their importance in the different scales. The model adopts the DIA-LSTM units to construct a new attention calibration architecture, which can adaptively adjust the weights of features in the process of feature interactions. On the other hand, it also integrates a multi-scale feature-crossing module to strengthen the representation ability of cross features from multi-field sparse data. The extensive experimental results on three real-world prediction datasets demonstrate that our proposed model yields superior performance compared with the other state-of-the-art models.",

keywords = "Attention network, Factorization machines, Feature engineering, Feature interactions",

author = "Zhifeng Xie and Wenling Zhang and Huiming Ding and Lizhuang Ma",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020 ; Conference date: 11-05-2020 Through 14-05-2020",

year = "2020",

doi = "10.1007/978-3-030-47426-3\_12",

language = "英语",

isbn = "9783030474256",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "142--154",

editor = "Lauw, \{Hady W.\} and Ee-Peng Lim and Wong, \{Raymond Chi-Wing\} and Alexandros Ntoulas and See-Kiong Ng and Pan, \{Sinno Jialin\}",

booktitle = "Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings",

address = "德国",

}

Xie, Z, Zhang, W, Ding, H & Ma, L 2020, MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data. in HW Lauw, E-P Lim, RC-W Wong, A Ntoulas, S-K Ng & SJ Pan (eds), Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12084 LNAI, Springer, pp. 142-154, 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020, Singapore, Singapore, 11/05/20. https://doi.org/10.1007/978-3-030-47426-3_12

MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data. / Xie, Zhifeng; Zhang, Wenling; Ding, Huiming et al.
Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings. ed. / Hady W. Lauw; Ee-Peng Lim; Raymond Chi-Wing Wong; Alexandros Ntoulas; See-Kiong Ng; Sinno Jialin Pan. Springer, 2020. p. 142-154 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12084 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - MsFcNET

T2 - 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020

AU - Xie, Zhifeng

AU - Zhang, Wenling

AU - Ding, Huiming

AU - Ma, Lizhuang

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2020

Y1 - 2020

N2 - Feature engineering usually needs to excavate dense-and-implicit cross features from multi-filed sparse data. Recently, many state-of-the-art models have been proposed to achieve low-order and high-order feature interactions. However, most of them ignore the importance of cross features and fail to suppress the negative impact of useless features. In this paper, a novel multi-scale feature-crossing attention network (MsFcNET) is proposed to extract dense-and-implicit cross features and learn their importance in the different scales. The model adopts the DIA-LSTM units to construct a new attention calibration architecture, which can adaptively adjust the weights of features in the process of feature interactions. On the other hand, it also integrates a multi-scale feature-crossing module to strengthen the representation ability of cross features from multi-field sparse data. The extensive experimental results on three real-world prediction datasets demonstrate that our proposed model yields superior performance compared with the other state-of-the-art models.

AB - Feature engineering usually needs to excavate dense-and-implicit cross features from multi-filed sparse data. Recently, many state-of-the-art models have been proposed to achieve low-order and high-order feature interactions. However, most of them ignore the importance of cross features and fail to suppress the negative impact of useless features. In this paper, a novel multi-scale feature-crossing attention network (MsFcNET) is proposed to extract dense-and-implicit cross features and learn their importance in the different scales. The model adopts the DIA-LSTM units to construct a new attention calibration architecture, which can adaptively adjust the weights of features in the process of feature interactions. On the other hand, it also integrates a multi-scale feature-crossing module to strengthen the representation ability of cross features from multi-field sparse data. The extensive experimental results on three real-world prediction datasets demonstrate that our proposed model yields superior performance compared with the other state-of-the-art models.

KW - Attention network

KW - Factorization machines

KW - Feature engineering

KW - Feature interactions

UR - https://www.scopus.com/pages/publications/85085729154

U2 - 10.1007/978-3-030-47426-3_12

DO - 10.1007/978-3-030-47426-3_12

M3 - 会议稿件

AN - SCOPUS:85085729154

SN - 9783030474256

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 142

EP - 154

BT - Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings

A2 - Lauw, Hady W.

A2 - Lim, Ee-Peng

A2 - Wong, Raymond Chi-Wing

A2 - Ntoulas, Alexandros

A2 - Ng, See-Kiong

A2 - Pan, Sinno Jialin

PB - Springer

Y2 - 11 May 2020 through 14 May 2020

ER -

Xie Z, Zhang W, Ding H, Ma L. MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data. In Lauw HW, Lim EP, Wong RCW, Ntoulas A, Ng SK, Pan SJ, editors, Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings. Springer. 2020. p. 142-154. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-47426-3_12

MsFcNET: Multi-scale Feature-Crossing Attention Network for Multi-field Sparse Data

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this