Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework

Kaustubh Beedkar; Aurélien Bertrand; Haralampos Gavriilidis; Augusto Fonseca; Zoi Kaoudi; Mingxi Liu; Volker Markl; Juri Petersen; Fabio Porto; Víctor Ribeiro; Mads Sejer Pedersen; Lucas Tavares; Michalis Vargiamis; Chen Xu

doi:10.1145/3722212.3725081

Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework

Kaustubh Beedkar, Aurélien Bertrand, Haralampos Gavriilidis, Augusto Fonseca, Zoi Kaoudi, Mingxi Liu, Volker Markl, Juri Petersen, Fabio Porto, Víctor Ribeiro, Mads Sejer Pedersen, Lucas Tavares, Michalis Vargiamis, Chen Xu

School of Data Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Apache Wayang is an open-source framework, which provides a systematic and efficient solution for unifying data analytics over disparate data sources and via integrating multiple heterogeneous data systems. It achieves that by decoupling applications from the underlying systems. In addition, it provides an optimizer so that users do not have to specify the platforms on which their pipeline should run but the optimizer can determine the best way given a cost metric. In this demonstration, we showcase how the flexible architecture of Wayang enables seamless integration with multiple heterogeneous data systems and how the query optimizer can lead to better performance.

Original language	English
Title of host publication	SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data
Editors	Amol Deshpande, Ashraf Aboulnaga, Babak Salimi, Badrish Chandramouli, Bill Howe, Boon Thau Loo, Boris Glavic, Carlo Curino, Daisy Zhe Wang, Dan Suciu, Daniel Abadi, Divesh Srivastava, Eugene Wu, Faisal Nawab, Ihab Ilyas, Jeffrey Naughton, Jennie Rogers, Jignesh Patel, Joy Arulraj, Jun Yang, Karima Echihabi, Kenneth Ross, Khuzaima Daudjee, Laks Lakshmanan, Minos Garofalakis, Mirek Riedewald, Mohamed Mokbel, Mourad Ouzzani, Oliver Kennedy, Oliver Kennedy, Paolo Papotti, Peter Alvaro, Peter Bailis, Renee Miller, Senjuti Basu Roy, Sergey Melnik, Stratos Idreos, Sudeepa Roy, Theodoros Rekatsinas, Viktor Leis, Wenchao Zhou, Wolfgang Gatterbauer, Zack Ives
Publisher	Association for Computing Machinery
Pages	35-38
Number of pages	4
ISBN (Electronic)	9798400715648
DOIs	https://doi.org/10.1145/3722212.3725081
State	Published - 22 Jun 2025
Event	2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025 - Berlin, Germany Duration: 22 Jun 2025 → 27 Jun 2025

Publication series

Name	Proceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)	0730-8078

Conference

Conference	2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025
Country/Territory	Germany
City	Berlin
Period	22/06/25 → 27/06/25

Keywords

query optimization
unified analytics

Access to Document

10.1145/3722212.3725081

Cite this

Beedkar, K., Bertrand, A., Gavriilidis, H., Fonseca, A., Kaoudi, Z., Liu, M., Markl, V., Petersen, J., Porto, F., Ribeiro, V., Sejer Pedersen, M., Tavares, L., Vargiamis, M., & Xu, C. (2025). Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework. In A. Deshpande, A. Aboulnaga, B. Salimi, B. Chandramouli, B. Howe, B. T. Loo, B. Glavic, C. Curino, D. Zhe Wang, D. Suciu, D. Abadi, D. Srivastava, E. Wu, F. Nawab, I. Ilyas, J. Naughton, J. Rogers, J. Patel, J. Arulraj, J. Yang, K. Echihabi, K. Ross, K. Daudjee, L. Lakshmanan, M. Garofalakis, M. Riedewald, M. Mokbel, M. Ouzzani, O. Kennedy, O. Kennedy, P. Papotti, P. Alvaro, P. Bailis, R. Miller, S. B. Roy, S. Melnik, S. Idreos, S. Roy, T. Rekatsinas, V. Leis, W. Zhou, W. Gatterbauer, ... Z. Ives (Eds.), SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data (pp. 35-38). (Proceedings of the ACM SIGMOD International Conference on Management of Data). Association for Computing Machinery. https://doi.org/10.1145/3722212.3725081

Beedkar, Kaustubh ; Bertrand, Aurélien ; Gavriilidis, Haralampos et al. / Apache Wayang in Action : Enabling Data Systems Integration via a Unified Data Analytics Framework. SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data. editor / Amol Deshpande ; Ashraf Aboulnaga ; Babak Salimi ; Badrish Chandramouli ; Bill Howe ; Boon Thau Loo ; Boris Glavic ; Carlo Curino ; Daisy Zhe Wang ; Dan Suciu ; Daniel Abadi ; Divesh Srivastava ; Eugene Wu ; Faisal Nawab ; Ihab Ilyas ; Jeffrey Naughton ; Jennie Rogers ; Jignesh Patel ; Joy Arulraj ; Jun Yang ; Karima Echihabi ; Kenneth Ross ; Khuzaima Daudjee ; Laks Lakshmanan ; Minos Garofalakis ; Mirek Riedewald ; Mohamed Mokbel ; Mourad Ouzzani ; Oliver Kennedy ; Oliver Kennedy ; Paolo Papotti ; Peter Alvaro ; Peter Bailis ; Renee Miller ; Senjuti Basu Roy ; Sergey Melnik ; Stratos Idreos ; Sudeepa Roy ; Theodoros Rekatsinas ; Viktor Leis ; Wenchao Zhou ; Wolfgang Gatterbauer ; Zack Ives. Association for Computing Machinery, 2025. pp. 35-38 (Proceedings of the ACM SIGMOD International Conference on Management of Data).

@inproceedings{c48e1a326a7a4ce9be5781f3e7f47348,

title = "Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework",

abstract = "Apache Wayang is an open-source framework, which provides a systematic and efficient solution for unifying data analytics over disparate data sources and via integrating multiple heterogeneous data systems. It achieves that by decoupling applications from the underlying systems. In addition, it provides an optimizer so that users do not have to specify the platforms on which their pipeline should run but the optimizer can determine the best way given a cost metric. In this demonstration, we showcase how the flexible architecture of Wayang enables seamless integration with multiple heterogeneous data systems and how the query optimizer can lead to better performance.",

keywords = "query optimization, unified analytics",

author = "Kaustubh Beedkar and Aur{\'e}lien Bertrand and Haralampos Gavriilidis and Augusto Fonseca and Zoi Kaoudi and Mingxi Liu and Volker Markl and Juri Petersen and Fabio Porto and V{\'i}ctor Ribeiro and \{Sejer Pedersen\}, Mads and Lucas Tavares and Michalis Vargiamis and Chen Xu",

note = "Publisher Copyright: {\textcopyright} 2025 ACM.; 2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025 ; Conference date: 22-06-2025 Through 27-06-2025",

year = "2025",

month = jun,

day = "22",

doi = "10.1145/3722212.3725081",

language = "英语",

series = "Proceedings of the ACM SIGMOD International Conference on Management of Data",

publisher = "Association for Computing Machinery",

pages = "35--38",

editor = "Amol Deshpande and Ashraf Aboulnaga and Babak Salimi and Badrish Chandramouli and Bill Howe and Loo, \{Boon Thau\} and Boris Glavic and Carlo Curino and \{Zhe Wang\}, Daisy and Dan Suciu and Daniel Abadi and Divesh Srivastava and Eugene Wu and Faisal Nawab and Ihab Ilyas and Jeffrey Naughton and Jennie Rogers and Jignesh Patel and Joy Arulraj and Jun Yang and Karima Echihabi and Kenneth Ross and Khuzaima Daudjee and Laks Lakshmanan and Minos Garofalakis and Mirek Riedewald and Mohamed Mokbel and Mourad Ouzzani and Oliver Kennedy and Oliver Kennedy and Paolo Papotti and Peter Alvaro and Peter Bailis and Renee Miller and Roy, \{Senjuti Basu\} and Sergey Melnik and Stratos Idreos and Sudeepa Roy and Theodoros Rekatsinas and Viktor Leis and Wenchao Zhou and Wolfgang Gatterbauer and Zack Ives",

booktitle = "SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data",

address = "美国",

}

Beedkar, K, Bertrand, A, Gavriilidis, H, Fonseca, A, Kaoudi, Z, Liu, M, Markl, V, Petersen, J, Porto, F, Ribeiro, V, Sejer Pedersen, M, Tavares, L, Vargiamis, M & Xu, C 2025, Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework. in A Deshpande, A Aboulnaga, B Salimi, B Chandramouli, B Howe, BT Loo, B Glavic, C Curino, D Zhe Wang, D Suciu, D Abadi, D Srivastava, E Wu, F Nawab, I Ilyas, J Naughton, J Rogers, J Patel, J Arulraj, J Yang, K Echihabi, K Ross, K Daudjee, L Lakshmanan, M Garofalakis, M Riedewald, M Mokbel, M Ouzzani, O Kennedy, O Kennedy, P Papotti, P Alvaro, P Bailis, R Miller, SB Roy, S Melnik, S Idreos, S Roy, T Rekatsinas, V Leis, W Zhou, W Gatterbauer & Z Ives (eds), SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data. Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, pp. 35-38, 2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025, Berlin, Germany, 22/06/25. https://doi.org/10.1145/3722212.3725081

Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework. / Beedkar, Kaustubh; Bertrand, Aurélien; Gavriilidis, Haralampos et al.
SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data. ed. / Amol Deshpande; Ashraf Aboulnaga; Babak Salimi; Badrish Chandramouli; Bill Howe; Boon Thau Loo; Boris Glavic; Carlo Curino; Daisy Zhe Wang; Dan Suciu; Daniel Abadi; Divesh Srivastava; Eugene Wu; Faisal Nawab; Ihab Ilyas; Jeffrey Naughton; Jennie Rogers; Jignesh Patel; Joy Arulraj; Jun Yang; Karima Echihabi; Kenneth Ross; Khuzaima Daudjee; Laks Lakshmanan; Minos Garofalakis; Mirek Riedewald; Mohamed Mokbel; Mourad Ouzzani; Oliver Kennedy; Oliver Kennedy; Paolo Papotti; Peter Alvaro; Peter Bailis; Renee Miller; Senjuti Basu Roy; Sergey Melnik; Stratos Idreos; Sudeepa Roy; Theodoros Rekatsinas; Viktor Leis; Wenchao Zhou; Wolfgang Gatterbauer; Zack Ives. Association for Computing Machinery, 2025. p. 35-38 (Proceedings of the ACM SIGMOD International Conference on Management of Data).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Apache Wayang in Action

T2 - 2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025

AU - Beedkar, Kaustubh

AU - Bertrand, Aurélien

AU - Gavriilidis, Haralampos

AU - Fonseca, Augusto

AU - Kaoudi, Zoi

AU - Liu, Mingxi

AU - Markl, Volker

AU - Petersen, Juri

AU - Porto, Fabio

AU - Ribeiro, Víctor

AU - Sejer Pedersen, Mads

AU - Tavares, Lucas

AU - Vargiamis, Michalis

AU - Xu, Chen

PY - 2025/6/22

Y1 - 2025/6/22

N2 - Apache Wayang is an open-source framework, which provides a systematic and efficient solution for unifying data analytics over disparate data sources and via integrating multiple heterogeneous data systems. It achieves that by decoupling applications from the underlying systems. In addition, it provides an optimizer so that users do not have to specify the platforms on which their pipeline should run but the optimizer can determine the best way given a cost metric. In this demonstration, we showcase how the flexible architecture of Wayang enables seamless integration with multiple heterogeneous data systems and how the query optimizer can lead to better performance.

AB - Apache Wayang is an open-source framework, which provides a systematic and efficient solution for unifying data analytics over disparate data sources and via integrating multiple heterogeneous data systems. It achieves that by decoupling applications from the underlying systems. In addition, it provides an optimizer so that users do not have to specify the platforms on which their pipeline should run but the optimizer can determine the best way given a cost metric. In this demonstration, we showcase how the flexible architecture of Wayang enables seamless integration with multiple heterogeneous data systems and how the query optimizer can lead to better performance.

KW - query optimization

KW - unified analytics

UR - https://www.scopus.com/pages/publications/105010210649

U2 - 10.1145/3722212.3725081

DO - 10.1145/3722212.3725081

M3 - 会议稿件

AN - SCOPUS:105010210649

T3 - Proceedings of the ACM SIGMOD International Conference on Management of Data

SP - 35

EP - 38

BT - SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data

A2 - Deshpande, Amol

A2 - Aboulnaga, Ashraf

A2 - Salimi, Babak

A2 - Chandramouli, Badrish

A2 - Howe, Bill

A2 - Loo, Boon Thau

A2 - Glavic, Boris

A2 - Curino, Carlo

A2 - Zhe Wang, Daisy

A2 - Suciu, Dan

A2 - Abadi, Daniel

A2 - Srivastava, Divesh

A2 - Wu, Eugene

A2 - Nawab, Faisal

A2 - Ilyas, Ihab

A2 - Naughton, Jeffrey

A2 - Rogers, Jennie

A2 - Patel, Jignesh

A2 - Arulraj, Joy

A2 - Yang, Jun

A2 - Echihabi, Karima

A2 - Ross, Kenneth

A2 - Daudjee, Khuzaima

A2 - Lakshmanan, Laks

A2 - Garofalakis, Minos

A2 - Riedewald, Mirek

A2 - Mokbel, Mohamed

A2 - Ouzzani, Mourad

A2 - Kennedy, Oliver

A2 - Papotti, Paolo

A2 - Alvaro, Peter

A2 - Bailis, Peter

A2 - Miller, Renee

A2 - Roy, Senjuti Basu

A2 - Melnik, Sergey

A2 - Idreos, Stratos

A2 - Roy, Sudeepa

A2 - Rekatsinas, Theodoros

A2 - Leis, Viktor

A2 - Zhou, Wenchao

A2 - Gatterbauer, Wolfgang

A2 - Ives, Zack

PB - Association for Computing Machinery

Y2 - 22 June 2025 through 27 June 2025

ER -

Beedkar K, Bertrand A, Gavriilidis H, Fonseca A, Kaoudi Z, Liu M et al. Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework. In Deshpande A, Aboulnaga A, Salimi B, Chandramouli B, Howe B, Loo BT, Glavic B, Curino C, Zhe Wang D, Suciu D, Abadi D, Srivastava D, Wu E, Nawab F, Ilyas I, Naughton J, Rogers J, Patel J, Arulraj J, Yang J, Echihabi K, Ross K, Daudjee K, Lakshmanan L, Garofalakis M, Riedewald M, Mokbel M, Ouzzani M, Kennedy O, Kennedy O, Papotti P, Alvaro P, Bailis P, Miller R, Roy SB, Melnik S, Idreos S, Roy S, Rekatsinas T, Leis V, Zhou W, Gatterbauer W, Ives Z, editors, SIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data. Association for Computing Machinery. 2025. p. 35-38. (Proceedings of the ACM SIGMOD International Conference on Management of Data). doi: 10.1145/3722212.3725081

Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this