Machine Learning for Theory Development

When people think of machine learning, powerful predictions usually come to mind.

However, machine learning can also be useful for developing theory in the social sciences (Agrawal et al., 2020; Bleidorn and Hopwood, 2019; Brandmaier et al., 2016; Cox et al., 2020; Shmueli, 2010; Van Lissa, 2022; Hofman et al., 2021; Fudenberg et al., 2019).

(1) Firstly, large datasets can contain rich information that can be mined using machine learning methods to find complex patterns or relationships that would be otherwise difficult to hypothesize about (Breiman, 2001). Thus, machine learning can be used as a hypothesis-generating tool or as a mechanism to prompt researchers to consider extending or updating existing theories.

(2) Additionally, the mining of novel digital data can produce new measures or ways to operationalize theoretical constructs (Bleidorn and Hopwood, 2019; Stachl et al., 2020).

(3) Machine learning can be used to establish prediction benchmarks or estimate prediction ceilings given the data (e.g., Nielsen et al., 2023; Snoek et al., 2023). By understanding the upper limits of predictability for certain behaviors or psychological measures, valuable information can be gained (Taleb, 2007; Karch et al., 2020). For example, maximum prediction performance can indicate the amount of noise or unexplainable randomness in a system or behavior of interest; poor predictability can indicate that measures need to be improved (e.g., by reducing measurement error); and predictive benchmarks can be used to estimate the gap between theory and practice by comparing a theoretically constructed model to a model optimized for prediction (Ehrenberg and Bound, 1993; Fudenberg et al., 2019).

(4) Finally, with recent advances in eXplainable AI (XAI) methods used to interpret even complex deep learning models (see Box 1) (e.g., Debeer and Strobl, 2020; Lundberg and Lee, 2017; Ribeiro et al., 2016; Shrikumar et al., 2017) machine learning can also be used to better understand the (potentially causal) complex relationships, in situations where there are a large number of possible predictors, heterogeneity, and non-linearities or sub-group effects.

These are just some of the key ways social science theory could benefit from machine learning methods.

Image: OpenArt, when prompted with the title.

References

Agrawal, M., Peterson, J. C., and Griffiths, T. L. (2020). Scaling up psychology via scientific regret minimization. Proceedings of the National Academy of Sciences, 117(16):8825–8835.

Bleidorn, W. and Hopwood, C. J. (2019). Using machine learning to advance personality assessment and theory. Personality and Social Psychology Review, 23(2):190–203.

Brandmaier, A. M., Prindle, J. J., McArdle, J. J., and Lindenberger, U. (2016). Theory-guided exploration with structural equation model forests., volume 21. American Psychological Association.

Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16(3):199–231.

Cox, C. R., Moscardini, E. H., Cohen, A. S., and Tucker, R. P. (2020). Machine learning for suicidology: A practical review of exploratory and hypothesis-driven approaches. Clinical Psychology Review, 82:101940.

Debeer, D. and Strobl, C. (2020). Conditional permutation importance revisited. BMC Bioinformatics, 21(1):1–30.

Ehrenberg, A. S. and Bound, J. A. (1993). Predictability and prediction. Journal of the Royal Statistical Society Series A: Statistics in Society, 156(2):167–194.

Fudenberg, D., Kleinberg, J., Liang, A., and Mullainathan, S. (2019). Measuring the completeness of theories. arXiv preprint arXiv:1910.07022.

Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H.,

Mullainathan, S., Salganik, M. J., Vazire, S., et al. (2021). Integrating explanation and prediction

in computational social science. Nature, 595(7866):181–188.

Karch, J. D., Brandmaier, A. M., and Voelkle, M. C. (2020). Gaussian process panel mod-

eling—machine learning inspired analysis of longitudinal panel data. Frontiers in psychology,

11:506696.

Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.

Nielsen, Y. A., Pfattheicher, S., and Thielmann, I. (2023). How much can personality predict prosocial behavior?

Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.

Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3):289–310.

Shrikumar, A., Greenside, P., and Kundaje, A. (2017). Learning important features through propagating activation differences. In International conference on machine learning, pages 3145–3153.

Snoek, L., Jack, R. E., Schyns, P. G., Garrod, O. G., Mittenb¨uhler, M., Chen, C., Oosterwijk, S., and Scholte, H. S. (2023). Testing, explaining, and exploring models of facial expressions of emotions. Science Advances, 9(6):eabq8421.

Stachl, C., Au, Q., Schoedel, R., Gosling, S. D., Harari, G. M., Buschek, D., V¨olkel, S. T., Schuwerk, T., Oldemeier, M., Ullmann, T., et al. (2020). Predicting personality from patterns of behavior collected with smartphones. Proceedings of the National Academy of Sciences, 117(30):17680–17687.

Taleb, N. N. (2007). Black swans and the domains of statistics. The American Statistician, 61(3):198–200.

Van Lissa, C. J. (2022). Developmental data science: How machine learning can advance theory formation in developmental psychology. Infant and Child Development, page e2370