EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition

Yingnan Fu; Tingting Liu; Ming Gao; Aoying Zhou

doi:10.1007/978-3-031-41676-7_8

EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition

Yingnan Fu
, Tingting Liu
, Ming Gao^*
, Aoying Zhou

^*Corresponding author for this work

School of Data Science and Engineering

East China Normal University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Printed mathematical expression recognition (PMER) aims to transcribe a printed mathematical expression image into a structural expression. The task is useful in a wide spectrum of applications, including personalized question recommendation and automatic problem solving. In this paper, we propose a new method named EDSL, shorted for an Encoder-Decoder architecture with Symbol-Level features, to recognize printed mathematical expressions from input images. Its encoder consists of a segmentation module to identify all symbols and their spatial information from the image in an unsupervised manner, and a reconstruction module to recover symbol dependencies after symbol segmentation. Furthermore, we employ a position correction attention mechanism to capture the spatial relationship between symbols, and apply a transformer model to alleviate the negative impact from long output. We conduct extensive experiments on two real datasets to verify the effectiveness and rationality of our proposed EDSL model. The experimental results illustrated that EDSL outperformed state-of-the-art methods by an accuracy margin of 3.47 % and 4.04 % in the two datasets, respectively.

Original language	English
Title of host publication	Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings
Editors	Gernot A. Fink, Rajiv Jain, Koichi Kise, Richard Zanibbi
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	134-151
Number of pages	18
ISBN (Print)	9783031416750
DOIs	https://doi.org/10.1007/978-3-031-41676-7_8
State	Published - 2023
Event	17th International Conference on Document Analysis and Recognition, ICDAR 2023 - San José, United States Duration: 21 Aug 2023 → 26 Aug 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14187 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th International Conference on Document Analysis and Recognition, ICDAR 2023
Country/Territory	United States
City	San José
Period	21/08/23 → 26/08/23

Keywords

encoder-decoder network
position correction attention
printed mathematical expression recognition
segmentation

Access to Document

10.1007/978-3-031-41676-7_8

Cite this

Fu, Y., Liu, T., Gao, M., & Zhou, A. (2023). EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition. In G. A. Fink, R. Jain, K. Kise, & R. Zanibbi (Eds.), Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings (pp. 134-151). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14187 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-41676-7_8

Fu, Yingnan ; Liu, Tingting ; Gao, Ming et al. / EDSL : An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition. Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings. editor / Gernot A. Fink ; Rajiv Jain ; Koichi Kise ; Richard Zanibbi. Springer Science and Business Media Deutschland GmbH, 2023. pp. 134-151 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{958076134143456d9cc527ba154bf049,

title = "EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition",

abstract = "Printed mathematical expression recognition (PMER) aims to transcribe a printed mathematical expression image into a structural expression. The task is useful in a wide spectrum of applications, including personalized question recommendation and automatic problem solving. In this paper, we propose a new method named EDSL, shorted for an Encoder-Decoder architecture with Symbol-Level features, to recognize printed mathematical expressions from input images. Its encoder consists of a segmentation module to identify all symbols and their spatial information from the image in an unsupervised manner, and a reconstruction module to recover symbol dependencies after symbol segmentation. Furthermore, we employ a position correction attention mechanism to capture the spatial relationship between symbols, and apply a transformer model to alleviate the negative impact from long output. We conduct extensive experiments on two real datasets to verify the effectiveness and rationality of our proposed EDSL model. The experimental results illustrated that EDSL outperformed state-of-the-art methods by an accuracy margin of 3.47 \% and 4.04 \% in the two datasets, respectively.",

keywords = "encoder-decoder network, position correction attention, printed mathematical expression recognition, segmentation",

author = "Yingnan Fu and Tingting Liu and Ming Gao and Aoying Zhou",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 17th International Conference on Document Analysis and Recognition, ICDAR 2023 ; Conference date: 21-08-2023 Through 26-08-2023",

year = "2023",

doi = "10.1007/978-3-031-41676-7\_8",

language = "英语",

isbn = "9783031416750",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "134--151",

editor = "Fink, \{Gernot A.\} and Rajiv Jain and Koichi Kise and Richard Zanibbi",

booktitle = "Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings",

address = "德国",

}

Fu, Y, Liu, T, Gao, M & Zhou, A 2023, EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition. in GA Fink, R Jain, K Kise & R Zanibbi (eds), Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14187 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 134-151, 17th International Conference on Document Analysis and Recognition, ICDAR 2023, San José, United States, 21/08/23. https://doi.org/10.1007/978-3-031-41676-7_8

EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition. / Fu, Yingnan; Liu, Tingting; Gao, Ming et al.
Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings. ed. / Gernot A. Fink; Rajiv Jain; Koichi Kise; Richard Zanibbi. Springer Science and Business Media Deutschland GmbH, 2023. p. 134-151 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14187 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - EDSL

T2 - 17th International Conference on Document Analysis and Recognition, ICDAR 2023

AU - Fu, Yingnan

AU - Liu, Tingting

AU - Gao, Ming

AU - Zhou, Aoying

PY - 2023

Y1 - 2023

N2 - Printed mathematical expression recognition (PMER) aims to transcribe a printed mathematical expression image into a structural expression. The task is useful in a wide spectrum of applications, including personalized question recommendation and automatic problem solving. In this paper, we propose a new method named EDSL, shorted for an Encoder-Decoder architecture with Symbol-Level features, to recognize printed mathematical expressions from input images. Its encoder consists of a segmentation module to identify all symbols and their spatial information from the image in an unsupervised manner, and a reconstruction module to recover symbol dependencies after symbol segmentation. Furthermore, we employ a position correction attention mechanism to capture the spatial relationship between symbols, and apply a transformer model to alleviate the negative impact from long output. We conduct extensive experiments on two real datasets to verify the effectiveness and rationality of our proposed EDSL model. The experimental results illustrated that EDSL outperformed state-of-the-art methods by an accuracy margin of 3.47 % and 4.04 % in the two datasets, respectively.

AB - Printed mathematical expression recognition (PMER) aims to transcribe a printed mathematical expression image into a structural expression. The task is useful in a wide spectrum of applications, including personalized question recommendation and automatic problem solving. In this paper, we propose a new method named EDSL, shorted for an Encoder-Decoder architecture with Symbol-Level features, to recognize printed mathematical expressions from input images. Its encoder consists of a segmentation module to identify all symbols and their spatial information from the image in an unsupervised manner, and a reconstruction module to recover symbol dependencies after symbol segmentation. Furthermore, we employ a position correction attention mechanism to capture the spatial relationship between symbols, and apply a transformer model to alleviate the negative impact from long output. We conduct extensive experiments on two real datasets to verify the effectiveness and rationality of our proposed EDSL model. The experimental results illustrated that EDSL outperformed state-of-the-art methods by an accuracy margin of 3.47 % and 4.04 % in the two datasets, respectively.

KW - encoder-decoder network

KW - position correction attention

KW - printed mathematical expression recognition

KW - segmentation

UR - https://www.scopus.com/pages/publications/85172187779

U2 - 10.1007/978-3-031-41676-7_8

DO - 10.1007/978-3-031-41676-7_8

M3 - 会议稿件

AN - SCOPUS:85172187779

SN - 9783031416750

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 134

EP - 151

BT - Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings

A2 - Fink, Gernot A.

A2 - Jain, Rajiv

A2 - Kise, Koichi

A2 - Zanibbi, Richard

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 21 August 2023 through 26 August 2023

ER -

Fu Y, Liu T, Gao M , Zhou A. EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition. In Fink GA, Jain R, Kise K, Zanibbi R, editors, Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 134-151. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-41676-7_8