Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction

Miaomiao Yang; Tao Wu; Jiali Mao; Kaixuan Zhu; Aoying Zhou

doi:10.1007/978-981-97-2262-4_30

Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction

Miaomiao Yang
, Tao Wu
, Jiali Mao^*
, Kaixuan Zhu
, Aoying Zhou

^*Corresponding author for this work

School of Data Science and Engineering

East China Normal University
Shanghai Engineering Research Center of Big Data Management

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Transportation time prediction (TIP) of a truck is one of key tasks for supporting the services in bulk logistics like route planning. But TIP prediction is challenging as it involves travel time prediction and dwell time prediction, which are influenced by various complex factors. Besides, there exists mutually constrained effects between travel time prediction and dwell time prediction. In this paper, we propose an Attention Mechanism based Multi-Task prediction framework consisting of travel pattern learning, stay pattern learning and transportation time modeling, called AMP. In view of that low prediction performance resulted by uncertain dwell time and mutually constrained effects between travel time and dwell time, we put forward a stay pattern learning module based on transformer and multi-factor attention mechanism. Furthermore, we design a multi-task learning based prediction module embedded with a mutual cross-attention mechanism to enhance overall prediction performance. Experimental results on a large-scale logistics data set demonstrate that our proposal can reduce MAPE by an average of 9.2%, MAE by an average of 19.5%, and RMSE by an average of 23.0% as compared to the baselines.

Original language	English
Title of host publication	Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings
Editors	De-Nian Yang, Xing Xie, Vincent S. Tseng, Jian Pei, Jen-Wei Huang, Jerry Chun-Wei Lin
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	376-388
Number of pages	13
ISBN (Print)	9789819722648
DOIs	https://doi.org/10.1007/978-981-97-2262-4_30
State	Published - 2024
Event	28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024 - Taipei, Taiwan, Province of China Duration: 7 May 2024 → 10 May 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14649 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024
Country/Territory	Taiwan, Province of China
City	Taipei
Period	7/05/24 → 10/05/24

Keywords

Attention mechanism
Bulk logistics
Multi-task learning

Access to Document

10.1007/978-981-97-2262-4_30

Cite this

Yang, M., Wu, T., Mao, J., Zhu, K., & Zhou, A. (2024). Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction. In D.-N. Yang, X. Xie, V. S. Tseng, J. Pei, J.-W. Huang, & J. C.-W. Lin (Eds.), Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings (pp. 376-388). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14649 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-97-2262-4_30

Yang, Miaomiao ; Wu, Tao ; Mao, Jiali et al. / Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction. Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings. editor / De-Nian Yang ; Xing Xie ; Vincent S. Tseng ; Jian Pei ; Jen-Wei Huang ; Jerry Chun-Wei Lin. Springer Science and Business Media Deutschland GmbH, 2024. pp. 376-388 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{1268a4775d6049ada9864f8d0dfbd23e,

title = "Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction",

abstract = "Transportation time prediction (TIP) of a truck is one of key tasks for supporting the services in bulk logistics like route planning. But TIP prediction is challenging as it involves travel time prediction and dwell time prediction, which are influenced by various complex factors. Besides, there exists mutually constrained effects between travel time prediction and dwell time prediction. In this paper, we propose an Attention Mechanism based Multi-Task prediction framework consisting of travel pattern learning, stay pattern learning and transportation time modeling, called AMP. In view of that low prediction performance resulted by uncertain dwell time and mutually constrained effects between travel time and dwell time, we put forward a stay pattern learning module based on transformer and multi-factor attention mechanism. Furthermore, we design a multi-task learning based prediction module embedded with a mutual cross-attention mechanism to enhance overall prediction performance. Experimental results on a large-scale logistics data set demonstrate that our proposal can reduce MAPE by an average of 9.2\%, MAE by an average of 19.5\%, and RMSE by an average of 23.0\% as compared to the baselines.",

keywords = "Attention mechanism, Bulk logistics, Multi-task learning",

author = "Miaomiao Yang and Tao Wu and Jiali Mao and Kaixuan Zhu and Aoying Zhou",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.; 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024 ; Conference date: 07-05-2024 Through 10-05-2024",

year = "2024",

doi = "10.1007/978-981-97-2262-4\_30",

language = "英语",

isbn = "9789819722648",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "376--388",

editor = "De-Nian Yang and Xing Xie and Tseng, \{Vincent S.\} and Jian Pei and Jen-Wei Huang and Lin, \{Jerry Chun-Wei\}",

booktitle = "Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings",

address = "德国",

}

Yang, M, Wu, T, Mao, J, Zhu, K & Zhou, A 2024, Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction. in D-N Yang, X Xie, VS Tseng, J Pei, J-W Huang & JC-W Lin (eds), Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14649 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 376-388, 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Taipei, Taiwan, Province of China, 7/05/24. https://doi.org/10.1007/978-981-97-2262-4_30

Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction. / Yang, Miaomiao; Wu, Tao; Mao, Jiali et al.
Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings. ed. / De-Nian Yang; Xing Xie; Vincent S. Tseng; Jian Pei; Jen-Wei Huang; Jerry Chun-Wei Lin. Springer Science and Business Media Deutschland GmbH, 2024. p. 376-388 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14649 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction

AU - Yang, Miaomiao

AU - Wu, Tao

AU - Mao, Jiali

AU - Zhu, Kaixuan

AU - Zhou, Aoying

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

PY - 2024

Y1 - 2024

N2 - Transportation time prediction (TIP) of a truck is one of key tasks for supporting the services in bulk logistics like route planning. But TIP prediction is challenging as it involves travel time prediction and dwell time prediction, which are influenced by various complex factors. Besides, there exists mutually constrained effects between travel time prediction and dwell time prediction. In this paper, we propose an Attention Mechanism based Multi-Task prediction framework consisting of travel pattern learning, stay pattern learning and transportation time modeling, called AMP. In view of that low prediction performance resulted by uncertain dwell time and mutually constrained effects between travel time and dwell time, we put forward a stay pattern learning module based on transformer and multi-factor attention mechanism. Furthermore, we design a multi-task learning based prediction module embedded with a mutual cross-attention mechanism to enhance overall prediction performance. Experimental results on a large-scale logistics data set demonstrate that our proposal can reduce MAPE by an average of 9.2%, MAE by an average of 19.5%, and RMSE by an average of 23.0% as compared to the baselines.

AB - Transportation time prediction (TIP) of a truck is one of key tasks for supporting the services in bulk logistics like route planning. But TIP prediction is challenging as it involves travel time prediction and dwell time prediction, which are influenced by various complex factors. Besides, there exists mutually constrained effects between travel time prediction and dwell time prediction. In this paper, we propose an Attention Mechanism based Multi-Task prediction framework consisting of travel pattern learning, stay pattern learning and transportation time modeling, called AMP. In view of that low prediction performance resulted by uncertain dwell time and mutually constrained effects between travel time and dwell time, we put forward a stay pattern learning module based on transformer and multi-factor attention mechanism. Furthermore, we design a multi-task learning based prediction module embedded with a mutual cross-attention mechanism to enhance overall prediction performance. Experimental results on a large-scale logistics data set demonstrate that our proposal can reduce MAPE by an average of 9.2%, MAE by an average of 19.5%, and RMSE by an average of 23.0% as compared to the baselines.

KW - Attention mechanism

KW - Bulk logistics

KW - Multi-task learning

UR - https://www.scopus.com/pages/publications/85192823762

U2 - 10.1007/978-981-97-2262-4_30

DO - 10.1007/978-981-97-2262-4_30

M3 - 会议稿件

AN - SCOPUS:85192823762

SN - 9789819722648

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 376

EP - 388

BT - Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings

A2 - Yang, De-Nian

A2 - Xie, Xing

A2 - Tseng, Vincent S.

A2 - Pei, Jian

A2 - Huang, Jen-Wei

A2 - Lin, Jerry Chun-Wei

PB - Springer Science and Business Media Deutschland GmbH

T2 - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024

Y2 - 7 May 2024 through 10 May 2024

ER -

Yang M, Wu T, Mao J, Zhu K, Zhou A. Attention Mechanism Based Multi-task Learning Framework for Transportation Time Prediction. In Yang DN, Xie X, Tseng VS, Pei J, Huang JW, Lin JCW, editors, Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 376-388. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-97-2262-4_30