Do Masked Autoencoders Improve Downhole Prediction? An Empirical Study on Real Well Drilling Data
MAE pretraining beats supervised GRU on 3.5M timestep drilling data...
A team from the University of Calgary (Berezowski, Hassanzadeh, Ginde) published the first empirical evaluation of masked autoencoder (MAE) pretraining for downhole drilling metric prediction. Using two publicly available Utah FORGE geothermal wells containing approximately 3.5 million timesteps of multivariate drilling telemetry, they conducted a systematic full-factorial design space search across 72 MAE configurations. The task was predicting Total Mud Volume, a critical drilling metric.
The best MAE configuration reduced test mean absolute error by 19.8% relative to a supervised GRU baseline, though it trailed a supervised LSTM baseline by 6.4%. Analysis revealed that latent space width is the dominant architectural choice (Pearson r = -0.59 with test MAE), while masking ratio had negligible effect—an unexpected finding attributed to high temporal redundancy in 1 Hz drilling data. These results establish MAE pretraining as a viable paradigm for drilling analytics and identify conditions where it's most beneficial, potentially reducing costly downhole sensor requirements.
- MAE pretraining cut prediction error by 19.8% vs supervised GRU on 3.5M timestep drilling dataset
- Latent space width was the dominant design factor (r=-0.59), masking ratio had negligible effect
- Best MAE still trailed supervised LSTM by 6.4%, suggesting hybrid approaches may be optimal
Why It Matters
Self-supervised pretraining could slash costly downhole sensor needs in drilling operations.