TY - GEN
T1 - Overview of the PromptCBLUE Shared Task in CHIP2023
AU - Zhu, Wei
AU - Wang, Xiaoling
AU - Chen, Mosha
AU - Tang, Buzhou
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.
PY - 2024
Y1 - 2024
N2 - This paper presents an overview of the PromptCBLUE shared task (http://cips-chip.org.cn/2023/eval1) held in the CHIP-2023 Conference. This shared task reformulates the CBLUE benchmark, and provide a good testbed for Chinese open-domain or medical-domain large language models (LLMs) in general medical natural language processing. Two different tracks are held: (a) prompt tuning track, investigating the multitask prompt tuning of LLMs, (b) probing the in-context learning capabilities of open-sourced LLMs. Many teams from both the industry and academia participated in the shared tasks, and the top teams achieved amazing test results. This paper describes the tasks, the datasets, evaluation metrics, and the top systems for both tasks. Finally, the paper summarizes the techniques and results of the evaluation of the various approaches explored by the participating teams.
AB - This paper presents an overview of the PromptCBLUE shared task (http://cips-chip.org.cn/2023/eval1) held in the CHIP-2023 Conference. This shared task reformulates the CBLUE benchmark, and provide a good testbed for Chinese open-domain or medical-domain large language models (LLMs) in general medical natural language processing. Two different tracks are held: (a) prompt tuning track, investigating the multitask prompt tuning of LLMs, (b) probing the in-context learning capabilities of open-sourced LLMs. Many teams from both the industry and academia participated in the shared tasks, and the top teams achieved amazing test results. This paper describes the tasks, the datasets, evaluation metrics, and the top systems for both tasks. Finally, the paper summarizes the techniques and results of the evaluation of the various approaches explored by the participating teams.
KW - Large language models
KW - Medical natural language processing
KW - PromptCBLUE
KW - in-context learning
KW - parameter efficient fine-tuning
UR - https://www.scopus.com/pages/publications/85189635915
U2 - 10.1007/978-981-97-1717-0_1
DO - 10.1007/978-981-97-1717-0_1
M3 - 会议稿件
AN - SCOPUS:85189635915
SN - 9789819717163
T3 - Communications in Computer and Information Science
SP - 3
EP - 20
BT - Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings
A2 - Xu, Hua
A2 - Chen, Qingcai
A2 - Lin, Hongfei
A2 - Wu, Fei
A2 - Liu, Lei
A2 - Tang, Buzhou
A2 - Hao, Tianyong
A2 - Huang, Zhengxing
A2 - Lei, Jianbo
A2 - Li, Zuofeng
A2 - Zong, Hui
PB - Springer Science and Business Media Deutschland GmbH
T2 - Evaluation track of the 9th China Health Information Processing Conference, CHIP 2023
Y2 - 27 October 2023 through 29 October 2023
ER -