Explaining GBDT by Probabilistic Finite-State Automata

Explainable artificial intelligence becomes vital for human users to understand and trust the decision-making process and results of machine learning methods. Unfortunately, most machine learning models are black-box and the algorithms running behind are opaque. In this work, we propose an approach to interpreting GBDT (Gradient Boosting Decision Tree Explanation) by extracting probabilistic finite-state automata from the trained model. Our method is inspired by and built upon a previous work that extracts probabilistic automata from RNN (Recurrent Neural Networks). To adapt the approach to our situation, we propose a series of techniques to ensure that the extracted probabilistic automaton approximates the GBDT model as accurately as possible. We conduct experiments on real-world datasets and our experimental results show that our method maintains a high level of fidelity of the extracted model as the size of the given GBDT model grows.