Rendered notebook
HTMLSubmission
The full write-up as exported from Jupyter: theorems, figures, tables, and the two applications.
Sam Castillo · github.io
This is Sam Castillo’s notebook for the Society of Actuaries Predictive Analytics and Futurism Section Jupyter contest: Predicting Uncertainty: Prediction Intervals from Gradient Boosting Quantile Regression. It is student-era work in mathematics and actuarial science from the University of Massachusetts Amherst. The proofs and the fitted models stay in the notebook. This page is the front door.
A point prediction hides how much the data can support it. The notebook’s line is that fortune tellers hand you one number, and actuaries hand you an interval. Quantile regression changes the loss so a gradient boosting model can aim at a percentile. Squared error targets the mean. The pinball loss, with quantile level τ, targets any percentile. Here the band runs from the 5th percentile to the 95th.
The demonstration uses the public medical-cost file that accompanies Brett Lantz’s Machine Learning with R (also on Kaggle): 1,338 patients. Age, sex, BMI, number of children, smoker, and region predict annual charges. Those charges are right-skewed, with a mean of $13,270. One third of the rows are held out, with the random seed fixed at 42. A linear regression baseline reaches an R² of 0.75 and a mean absolute error of about $4,243. The gradient boosting model of the mean improves that to an R² of 0.809 and a mean absolute error of about $3,161.
The same quantiles do two jobs. First they put an interval on a health-insurance risk score, so two members with the same average cost can still carry very different uncertainty. Actuarial Standard of Practice No. 23 is the hook: when the data are thin, the actuary should say so, and the width of the interval is one way to say it. Second, they audit the file. Of 442 patients in the holdout set, 10 fall outside the 5th-to-95th band. Several of those rows are implausible on their face: impossible body-mass indexes, and a 12-year-old listed with three children.
Percentiles do not add the way averages do. A mean can be split by group and reassembled; a 95th percentile cannot. The notebook shows that algebra, and it keeps the two short proofs — mean squared error, then the quantile loss — next to the code that uses them.
The contest entry is the notebook. The rendered HTML and the .ipynb in this repository are unchanged.
Submission
The full write-up as exported from Jupyter: theorems, figures, tables, and the two applications.
Source
scikit-learn gradient boosting, a linear baseline, and the 5th and 95th percentile fits. Random seed 42.
Original link
The link from the original repository README, plus the short URL bit.ly/2LfHf9u.
Jump straight to a section of the rendered submission.
Also collected on SamWiki, the index of Sam Castillo’s public math, machine learning, and actuarial projects.
Each card opens a source the submission already cites, and the same place in the rendered notebook. The .ipynb and the exported HTML in this repository are unchanged.
Kaggle
1,338 patients. The notebook links this public file, the companion data for Brett Lantz’s Machine Learning with R.
ASB
Data Quality. The introduction quotes section 3.1: when the data are limited, say so.
CAS
The study note the notebook uses for Value at Risk as a quantile of the loss.
ISL
Chapter 8.2 of An Introduction to Statistical Learning is the notebook’s boosting reference. The faculty page in the file no longer resolves.
Journal
Elisabeth Waldmann, “Quantile regression: A short story on how and why,” Statistical Modelling. The notebook’s citation for work outside contest leaderboards.
Investopedia
The introduction’s comparison: a dollar forecast needs the spread around the average, not only the average.
Supervised learning and the contest comparison are in Quantile regression. The interquartile range is in the outlier audit. scikit-learn and the two essays are in the discussion.