Research story · Decision trees and classification
How can a prediction rule be both reliable and easy to read?
Classifiers and decision trees learn rules from data to predict an outcome. Over the years, this line of work has asked how much those rules can be trusted, has used classifiers on real problems, from ports to health care fraud, and has developed oblique regression trees that try to be both accurate and easy to read.
At a glance
- Period
- 2013 to the present
- Threads
- Reliability of classifiers; classifiers in practice; oblique trees; multivariate regression trees
- Selected collaborators
- Claudio Conversano, Francesco Mola, Andrea Carta, Giulia Contu, Marco Ortu. All authors are listed with each publication.
- Key paper
- Computational Statistics 2025 (open access)
- Latest
- An interpretable ensemble of oblique trees (IFCS 2026)
The question
Classification and regression trees split the data with simple rules, such as “income above a threshold”, and predict with the average or the most frequent class of each group. Two questions run through this line of work. How much can we trust the rule that a classifier has learned from the data at hand? And can a tree stay easy to read while following the structure of the data?
A tree for an applied prediction problem 2013–2014
At a transhipment container terminal, planning each work shift depends on when ships actually arrive, and the arrival times notified by ship operators are often revised. After an exploratory analysis of the delays recorded at a Mediterranean terminal, the article proposed using a classification and regression tree (CART), an established method, to reduce the uncertainty about arrivals (Transport, 2014 (opens in a new tab)). A conference paper had compared machine learning methods to predict delays in container terminals (WCTR 2013).
How reliable is a classifier? 2014–2019
A classifier may perform well on the data at hand and still learn an unstable rule, especially when classes are imbalanced or observations are few. We proposed to assess the reliability of a multi-class classifier by retraining it on resampled versions of the data (ECDA 2014; Springer, 2016 (opens in a new tab)). User-defined misclassification costs, applied to the resulting confusion matrices and modeled with a Beta regression, give a cost-sensitive classification index; a second index measures how much the distribution of the predicted classes differs from the observed one. Both lie between 0 and 1, so each classifier becomes a point in a square, and its position on the original and resampled data shows how reliable it is.
Other conference contributions of those years dealt with a bivariate cost-sensitive classifier performance index (COMPSTAT 2014), the reliability of a classifier (CLADAG 2015), combined methods for multi-label classification (CLADAG 2017), the sequential automatic search of a subset of classifiers, SASSC, also applied to insurance data (IFCS 2017; MIPIS 2017), and regularization in tree-based classification models (ISBIS 2019).
Classifiers in practice 2021–2023
Health care fraud detection applies classifiers to large claims data in which fraud is rare and its distribution skewed. We compared several classification algorithms and data pre-processing methods on publicly available claims payment data of U.S. Medicare Part B, under several fraud scenarios, and discussed their accuracy and computational demand, to help practitioners choose and apply a method (Applied Stochastic Models in Business and Industry, 2021 (opens in a new tab)). The data are on GitHub.
Supervised learning is useful beyond classification. Near-infrared spectra of paper books dated between 1851 and 2000, combined with three supervised machine learning methods, predicted the year of publication: two of the methods reached an accuracy of up to two years, and the most informative wavelengths could be linked to the structure of cellulose and of proteins (Journal of the American Chemical Society, 2023 (opens in a new tab); see also TECHNART 2023). A conference contribution used machine learning models to discriminate tourist landscapes from eye-tracking data (ENBIS 2021).
Oblique trees 2025–2026
Each rule of a standard tree involves a single variable. When the response changes along a combination of variables, a tree needs a staircase of splits to follow it: it becomes deeper, less stable and harder to read. Oblique trees split on a linear combination of predictors instead: at each node, a hyperplane rather than a line parallel to one of the axes.
Splits parallel to the axes
Oblique splits
The difficulty is to find good hyperplanes quickly. TORS, which we developed, does it in three steps at each node (Computational Statistics, 2025 (opens in a new tab)):
- it selects the predictors most correlated with the response;
- it splits the continuous response into two classes;
- it uses a weighted support vector machine with a linear kernel to find the hyperplane that minimizes the deviance.
We compared TORS with four other oblique trees and a standard regression tree (CART), using five-fold cross-validation. On 3,840 simulated data sets, TORS obtained the lowest root mean squared error more often than the other five models. On 109 real benchmark data sets, it had the lowest error on 52, more than any other model, and was first or second on about three quarters; its relative performance was best on the smallest data sets. Because each split shows which variables it uses, TORS is easier to interpret than oblique trees that do not select variables. With only the two predictors most correlated with the response, the setting that gave the best predictions in the simulations, each split can be drawn in a plane.
- TORS
- ODT
- CART
- HHCART
- BUTIA
- CO2
WOLTReg, a related method that we proposed, finds the oblique splits with a weighted logistic regression instead of a support vector machine, again with a selection of predictors at each node. On benchmark data sets it worked better when only a few predictors defined each split, and in the comparison reported in the chapter it had a lower root mean squared error than other oblique trees and a standard tree (Springer, 2025 (opens in a new tab); CLADAG 2025).
Further contributions concern feature-selective oblique trees for regression, applied to the wages of STEM graduates in Italy (ICSTA 2025 (opens in a new tab)), a new approach to oblique decision trees for regression (ISI 2025), and an interpretable ensemble of oblique trees based on weighted support vector machines (IFCS 2026).
Multivariate regression trees for territorial data 2023–2026
Multivariate regression trees predict several responses at once and group similar units along the way. They were used to study Italian mortality rates (CLADAG 2023) and the determinants of tourist arrivals in the 307 municipalities of Sardinia, with arrivals by nationality as responses and geography, demographic structure and transport accessibility as predictors. The density of accommodation facilities emerged as the main determinant, together with a persistent difference between the coast and the inland (Springer, 2026 (opens in a new tab); see also IFCS 2026).
Publications in this line
2026
In Statistical Science: From Theory to Applied Research III, pp. 38–44. Springer
doi:10.1007/978-3-032-30881-8_7 (opens in a new tab)Cite
@incollection{contu2026territorial, author = {Contu, G. and Atzori, F. and Carta, A. and Ortu, M. and Frigau, L.}, title = {Territorial Determinants of Tourist Arrivals: A Multivariate Regression Tree Analysis}, booktitle = {Statistical Science: From Theory to Applied Research III}, editor = {Martella, F. and Arima, S. and Marino, M. F. and Mollica, C.}, series = {Italian Statistical Society Series on Advances in Statistics}, publisher = {Springer}, address = {Cham}, year = {2026}, pages = {38--44}, doi = {10.1007/978-3-032-30881-8_7}, isbn = {978-3-032-30880-1}, }Multivariate Regression Trees for the Analysis of Socio-Economic Phenomena: Evidence from Tourism and Health Outcomes
Book of Abstracts of IFCS 2026
Cite
@inproceedings{contu2026multivariate, author = {Contu, G. and Carta, A. and Frigau, L. and Ortu, M.}, title = {Multivariate Regression Trees for the Analysis of Socio-Economic Phenomena: Evidence from Tourism and Health Outcomes}, booktitle = {Book of Abstracts of IFCS 2026}, year = {2026}, }An Interpretable Ensemble of Weighted SVM–Based Oblique Decision Trees
Book of Abstracts of IFCS 2026
Cite
@inproceedings{carta2026interpretable, author = {Carta, A. and Frigau, L. and Contu, G. and Ortu, M.}, title = {An Interpretable Ensemble of Weighted SVM–Based Oblique Decision Trees}, booktitle = {Book of Abstracts of IFCS 2026}, year = {2026}, }A novel approach for oblique decision trees for regression
Book of Abstracts of ISI 2025, p. 28
Cite
@inproceedings{carta2026novel, author = {Carta, A. and Mola, F. and Frigau, L. and Contu, G. and Ortu, M.}, title = {A novel approach for oblique decision trees for regression}, booktitle = {Book of Abstracts of ISI 2025}, year = {2026}, pages = {28}, isbn = {978-9-073592-43-8}, }
2025
Tree Oblique for Regression with Weighted Support Vector Machine (opens in a new tab)
Computational Statistics, 40(9): 5257–5291
Regression trees whose splits are oblique hyperplanes found by a weighted support vector machine. In the experiments of the paper, TORS most often had the lowest prediction error among six tree models.
doi:10.1007/s00180-025-01647-w (opens in a new tab)Cite
@article{carta2025tree, author = {Carta, A. and Frigau, L.}, title = {Tree Oblique for Regression with Weighted Support Vector Machine}, journal = {Computational Statistics}, year = {2025}, volume = {40}, number = {9}, pages = {5257--5291}, doi = {10.1007/s00180-025-01647-w}, issn = {1613-9658}, }Weighted Logistic Oblique Tree for Regression (opens in a new tab)
In Supervised and Unsupervised Statistical Data Analysis, pp. 70–82. Springer
doi:10.1007/978-3-032-03042-9_7 (opens in a new tab)Cite
@incollection{carta2025weighted, author = {Carta, A. and Frigau, L.}, title = {Weighted Logistic Oblique Tree for Regression}, booktitle = {Supervised and Unsupervised Statistical Data Analysis}, series = {Studies in Classification, Data Analysis, and Knowledge Organization}, publisher = {Springer}, address = {Cham}, year = {2025}, pages = {70--82}, doi = {10.1007/978-3-032-03042-9_7}, isbn = {978-3-032-03042-9}, }Weighted Logistic Oblique Tree For Regression
CLADAG VOC 2025 - Book of Abstract, p. 202
Cite
@inproceedings{carta2025weightedb, author = {Carta, A. and Frigau, L.}, title = {Weighted Logistic Oblique Tree For Regression}, booktitle = {CLADAG VOC 2025 - Book of Abstract}, year = {2025}, pages = {202}, isbn = {978-8-899594-24-4}, }ICSTA 2025 - Proceedings of the 7th International Conference on Statistics: Theory and Applications. Avestia Publishing
doi:10.11159/icsta25.149 (opens in a new tab)Cite
@inproceedings{carta2025feature, author = {Carta, A. and Frigau, L.}, title = {Feature-Selective Oblique Trees for Regression: Application to STEM Graduate Wage Prediction in Italy}, booktitle = {ICSTA 2025 - Proceedings of the 7th International Conference on Statistics: Theory and Applications}, publisher = {Avestia Publishing}, year = {2025}, number = {149}, doi = {10.11159/icsta25.149}, isbn = {978-1-990800-59-7}, }
2023
Journal of the American Chemical Society, 145(22): 12305–12314
doi:10.1021/jacs.3c02835 (opens in a new tab)Cite
@article{coppola2023near, author = {Coppola, F. and Frigau, L. and Markelj, J. and Malešič, J. and Conversano, C. and Strlič, M.}, title = {Near-Infrared Spectroscopy and Machine Learning for Accurate Dating of Historical Books}, journal = {Journal of the American Chemical Society}, year = {2023}, volume = {145}, number = {22}, pages = {12305--12314}, doi = {10.1021/jacs.3c02835}, issn = {1520-5126}, }Machine learning and NIR spectroscopy for dating of books
TECHNART2023 Book of Abstracts, p. 67
Cite
@inproceedings{coppola2023machine, author = {Coppola, F. and Frigau, L. and Markelj, J. and Malešič, J. and Conversano, C. and Strlič, M.}, title = {Machine learning and NIR spectroscopy for dating of books}, booktitle = {TECHNART2023 Book of Abstracts}, year = {2023}, pages = {67}, isbn = {978-989-9164-08-6}, }Multivariate Regression Tree to Investigate the Italian Mortality Rates
CLADAG 2023 Book of Abstracts and Short Papers, pp. 394–397
Cite
@inproceedings{contu2023multivariate, author = {Contu, G. and Frigau, L. and Ortu, M. and Pau, S.}, title = {Multivariate Regression Tree to Investigate the Italian Mortality Rates}, booktitle = {CLADAG 2023 Book of Abstracts and Short Papers}, year = {2023}, pages = {394--397}, isbn = {978-88-919-3563-2}, }
2021
Health Care Fraud Classifiers in Practice (opens in a new tab)
Applied Stochastic Models in Business and Industry, 37(6): 1182–1199
doi:10.1002/asmb.2633 (opens in a new tab) Data (opens in a new tab)Cite
@article{ekin2021health, author = {Ekin, T. and Frigau, L. and Conversano, C.}, title = {Health Care Fraud Classifiers in Practice}, journal = {Applied Stochastic Models in Business and Industry}, year = {2021}, volume = {37}, number = {6}, pages = {1182--1199}, doi = {10.1002/asmb.2633}, issn = {1526-4025}, }Application of machine learning models to discriminate tourist landscapes using eye-tracking data
ENBIS 2021 Book of Abstracts, pp. 12–13
Cite
@inproceedings{zammarchi2021application, author = {Zammarchi, G. and Contu, G. and Frigau, L.}, title = {Application of machine learning models to discriminate tourist landscapes using eye-tracking data}, booktitle = {ENBIS 2021 Book of Abstracts}, year = {2021}, pages = {12--13}, isbn = {979-12-20093-98-9}, }
2019
Regularization in tree-based classification models
Book of Abstracts of ISBIS 2019
Cite
@inproceedings{frigau2019regularization, author = {Frigau, L. and Banks, D.}, title = {Regularization in tree-based classification models}, booktitle = {Book of Abstracts of ISBIS 2019}, year = {2019}, }
2017
Combined methods in multi-label classification algorithms
Proceedings of CLADAG 2017. Universitas Studiorum
Cite
@inproceedings{frigau2017combined, author = {Frigau, L. and Conversano, C. and Mola, F.}, title = {Combined methods in multi-label classification algorithms}, booktitle = {Proceedings of CLADAG 2017}, publisher = {Universitas Studiorum}, address = {Mantua}, year = {2017}, isbn = {978-88-99459-71-0}, }The SASSC Algorithm: an Application to Insurance Data
Book of Abstracts of MIPIS 2017
Cite
@inproceedings{frigau2017sassc, author = {Frigau, L.}, title = {The SASSC Algorithm: an Application to Insurance Data}, booktitle = {Book of Abstracts of MIPIS 2017}, year = {2017}, }Advances in sequential automatic search of subset of classifiers
Book of Abstracts of IFCS 2017
Cite
@inproceedings{frigau2017advances, author = {Frigau, L. and Contu, G. and Mola, F.}, title = {Advances in sequential automatic search of subset of classifiers}, booktitle = {Book of Abstracts of IFCS 2017}, year = {2017}, }
2016
Assessing the reliability of a multi-class classifier (opens in a new tab)
In Analysis of Large and Complex Data, pp. 207–217. Springer
doi:10.1007/978-3-319-25226-1_18 (opens in a new tab)Cite
@incollection{frigau2016assessing, author = {Frigau, L. and Conversano, C. and Mola, F.}, title = {Assessing the reliability of a multi-class classifier}, booktitle = {Analysis of Large and Complex Data}, series = {Studies in Classification, Data Analysis, and Knowledge Organization}, publisher = {Springer}, address = {Cham}, year = {2016}, pages = {207--217}, doi = {10.1007/978-3-319-25226-1_18}, isbn = {978-3-319-25226-1}, }
2015
A note on the reliability of a classifier
Proceedings of CLADAG 2015, pp. 366–369. CUEC
Cite
@inproceedings{frigau2015note, author = {Frigau, L.}, title = {A note on the reliability of a classifier}, booktitle = {Proceedings of CLADAG 2015}, publisher = {CUEC}, address = {Cagliari}, year = {2015}, pages = {366--369}, isbn = {978-88-8467-749-9}, }
2014
Transport, 29(2): 175–184
doi:10.3846/16484142.2014.930714 (opens in a new tab)Cite
@article{pani2014data, author = {Pani, C. and Fadda, P. and Fancello, G. and Frigau, L. and Mola, F.}, title = {A data mining approach to forecast late arrivals in a transhipment container terminal}, journal = {Transport}, year = {2014}, volume = {29}, number = {2}, pages = {175--184}, doi = {10.3846/16484142.2014.930714}, issn = {1648-4142}, }A Bivariate Cost-sensitive Classifier Performance Index
Proceedings of COMPSTAT 2014, pp. 577–584. International Statistical Institute
Cite
@inproceedings{frigau2014bivariate, author = {Frigau, L. and Conversano, C. and Mola, F.}, title = {A Bivariate Cost-sensitive Classifier Performance Index}, booktitle = {Proceedings of COMPSTAT 2014}, publisher = {International Statistical Institute}, address = {The Hague}, year = {2014}, pages = {577--584}, isbn = {978-2-8399-1347-8}, }A Bivariate Cost-sensitive Classifier Performance Index
Book of Abstracts of COMPSTAT 2014
Cite
@inproceedings{frigau2014bivariateb, author = {Frigau, L. and Conversano, C. and Mola, F.}, title = {A Bivariate Cost-sensitive Classifier Performance Index}, booktitle = {Book of Abstracts of COMPSTAT 2014}, year = {2014}, }Assessing the Reliability of a Multi-class Classifier
Book of Abstracts of ECDA 2014
Cite
@inproceedings{frigau2014assessing, author = {Frigau, L. and Conversano, C. and Mola, F.}, title = {Assessing the Reliability of a Multi-class Classifier}, booktitle = {Book of Abstracts of ECDA 2014}, year = {2014}, }
2013
Delay prediction in container terminals: A comparison of machine learning methods
Proceedings of the WCTR (World Conference on Transport Research)
Cite
@inproceedings{cannas2013delay, author = {Cannas, M. and Fadda, P. and Fancello, G. and Frigau, L. and Mola, F. and Pani, C.}, title = {Delay prediction in container terminals: A comparison of machine learning methods}, booktitle = {Proceedings of the WCTR (World Conference on Transport Research)}, year = {2013}, isbn = {978-85-285-0232-9}, }