FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision

Wenbiao Tao; Hanlun Zhu; Keren Tan; Jiani Wang; Yuanyuan Liang; Huihui Jiang; Pengcheng Yuan; Yunshi Lan

doi:10.1007/978-3-031-70371-3_32

FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision

Wenbiao Tao, Hanlun Zhu, Keren Tan, Jiani Wang, Yuanyuan Liang, Huihui Jiang, Pengcheng Yuan, Yunshi Lan

School of Data Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Knowledge graph question answering (KGQA) in the finance domain aims to answer questions based on a dynamic knowledge graph (KG), which suffers from frequent updates. Moreover, the lack of high-quality annotated data renders data-driven and training-dependent approaches ineffective. To bridge the gap, we develop FinQA, which is a training-free dynamic knowledge graph question answering system in finance with large language model based (LLM-based) revision. Specifically, FinQA gives considerations to the following aspects: (1) constructing a dynamic finance knowledge graph partitioned based on data update frequencies; (2) proposing a training-free question-answering (QA) system to parse natural language to graph query language (NL2GQL) and achieving high-efficient coordination with the dynamic KG; (3) integrating the QA system with an open-source LLM to further boost the accuracy.

Original language	English
Title of host publication	Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings
Editors	Albert Bifet, Povilas Daniušis, Jesse Davis, Tomas Krilavičius, Meelis Kull, Eirini Ntoutsi, Kai Puolamäki, Indrė Žliobaitė
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	418-423
Number of pages	6
ISBN (Print)	9783031703706
DOIs	https://doi.org/10.1007/978-3-031-70371-3_32
State	Published - 2024
Event	European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024 - Vilnius, Lithuania Duration: 9 Sep 2024 → 13 Sep 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14948 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024
Country/Territory	Lithuania
City	Vilnius
Period	9/09/24 → 13/09/24

Keywords

Finance Knowledge Graph
Large Language Model Revision
Question Answering
Training-free

Access to Document

10.1007/978-3-031-70371-3_32

Cite this

Tao, W., Zhu, H., Tan, K., Wang, J., Liang, Y., Jiang, H., Yuan, P., & Lan, Y. (2024). FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision. In A. Bifet, P. Daniušis, J. Davis, T. Krilavičius, M. Kull, E. Ntoutsi, K. Puolamäki, & I. Žliobaitė (Eds.), Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings (pp. 418-423). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14948 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-70371-3_32

Tao, Wenbiao ; Zhu, Hanlun ; Tan, Keren et al. / FinQA : A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision. Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings. editor / Albert Bifet ; Povilas Daniušis ; Jesse Davis ; Tomas Krilavičius ; Meelis Kull ; Eirini Ntoutsi ; Kai Puolamäki ; Indrė Žliobaitė. Springer Science and Business Media Deutschland GmbH, 2024. pp. 418-423 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{790ec194f20a4fe9b561f4b0dbfeab4c,

title = "FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision",

abstract = "Knowledge graph question answering (KGQA) in the finance domain aims to answer questions based on a dynamic knowledge graph (KG), which suffers from frequent updates. Moreover, the lack of high-quality annotated data renders data-driven and training-dependent approaches ineffective. To bridge the gap, we develop FinQA, which is a training-free dynamic knowledge graph question answering system in finance with large language model based (LLM-based) revision. Specifically, FinQA gives considerations to the following aspects: (1) constructing a dynamic finance knowledge graph partitioned based on data update frequencies; (2) proposing a training-free question-answering (QA) system to parse natural language to graph query language (NL2GQL) and achieving high-efficient coordination with the dynamic KG; (3) integrating the QA system with an open-source LLM to further boost the accuracy.",

keywords = "Finance Knowledge Graph, Large Language Model Revision, Question Answering, Training-free",

author = "Wenbiao Tao and Hanlun Zhu and Keren Tan and Jiani Wang and Yuanyuan Liang and Huihui Jiang and Pengcheng Yuan and Yunshi Lan",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024 ; Conference date: 09-09-2024 Through 13-09-2024",

year = "2024",

doi = "10.1007/978-3-031-70371-3\_32",

language = "英语",

isbn = "9783031703706",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "418--423",

editor = "Albert Bifet and Povilas Daniu{\v s}is and Jesse Davis and Tomas Krilavi{\v c}ius and Meelis Kull and Eirini Ntoutsi and Kai Puolam{\"a}ki and Indrė {\v Z}liobaitė",

booktitle = "Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings",

address = "德国",

}

Tao, W, Zhu, H, Tan, K, Wang, J, Liang, Y, Jiang, H, Yuan, P & Lan, Y 2024, FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision. in A Bifet, P Daniušis, J Davis, T Krilavičius, M Kull, E Ntoutsi, K Puolamäki & I Žliobaitė (eds), Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14948 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 418-423, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024, Vilnius, Lithuania, 9/09/24. https://doi.org/10.1007/978-3-031-70371-3_32

FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision. / Tao, Wenbiao; Zhu, Hanlun; Tan, Keren et al.
Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings. ed. / Albert Bifet; Povilas Daniušis; Jesse Davis; Tomas Krilavičius; Meelis Kull; Eirini Ntoutsi; Kai Puolamäki; Indrė Žliobaitė. Springer Science and Business Media Deutschland GmbH, 2024. p. 418-423 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14948 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - FinQA

T2 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024

AU - Tao, Wenbiao

AU - Zhu, Hanlun

AU - Tan, Keren

AU - Wang, Jiani

AU - Liang, Yuanyuan

AU - Jiang, Huihui

AU - Yuan, Pengcheng

AU - Lan, Yunshi

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024

Y1 - 2024

N2 - Knowledge graph question answering (KGQA) in the finance domain aims to answer questions based on a dynamic knowledge graph (KG), which suffers from frequent updates. Moreover, the lack of high-quality annotated data renders data-driven and training-dependent approaches ineffective. To bridge the gap, we develop FinQA, which is a training-free dynamic knowledge graph question answering system in finance with large language model based (LLM-based) revision. Specifically, FinQA gives considerations to the following aspects: (1) constructing a dynamic finance knowledge graph partitioned based on data update frequencies; (2) proposing a training-free question-answering (QA) system to parse natural language to graph query language (NL2GQL) and achieving high-efficient coordination with the dynamic KG; (3) integrating the QA system with an open-source LLM to further boost the accuracy.

AB - Knowledge graph question answering (KGQA) in the finance domain aims to answer questions based on a dynamic knowledge graph (KG), which suffers from frequent updates. Moreover, the lack of high-quality annotated data renders data-driven and training-dependent approaches ineffective. To bridge the gap, we develop FinQA, which is a training-free dynamic knowledge graph question answering system in finance with large language model based (LLM-based) revision. Specifically, FinQA gives considerations to the following aspects: (1) constructing a dynamic finance knowledge graph partitioned based on data update frequencies; (2) proposing a training-free question-answering (QA) system to parse natural language to graph query language (NL2GQL) and achieving high-efficient coordination with the dynamic KG; (3) integrating the QA system with an open-source LLM to further boost the accuracy.

KW - Finance Knowledge Graph

KW - Large Language Model Revision

KW - Question Answering

KW - Training-free

UR - https://www.scopus.com/pages/publications/85203865203

U2 - 10.1007/978-3-031-70371-3_32

DO - 10.1007/978-3-031-70371-3_32

M3 - 会议稿件

AN - SCOPUS:85203865203

SN - 9783031703706

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 418

EP - 423

BT - Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings

A2 - Bifet, Albert

A2 - Daniušis, Povilas

A2 - Davis, Jesse

A2 - Krilavičius, Tomas

A2 - Kull, Meelis

A2 - Ntoutsi, Eirini

A2 - Puolamäki, Kai

A2 - Žliobaitė, Indrė

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 9 September 2024 through 13 September 2024

ER -

Tao W, Zhu H, Tan K, Wang J, Liang Y, Jiang H et al. FinQA: A Training-Free Dynamic Knowledge Graph Question Answering System in Finance with LLM-Based Revision. In Bifet A, Daniušis P, Davis J, Krilavičius T, Kull M, Ntoutsi E, Puolamäki K, Žliobaitė I, editors, Machine Learning and Knowledge Discovery in Databases. Research Track and Demo Track - European Conference, ECML PKDD 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 418-423. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-70371-3_32