February 23, 2020 | Written by: Martin Fleming and Subhro Das
Share this post:
AI has started to transform occupations, with the potential to make both labor and machines more efficient and productive. To better understand this transformation, our team at the MIT-IBM Watson AI Lab presented a new study at the AAAI Conference on AI, Ethics, and Society on “Learning Occupational Task-Shares Dynamics for the Future of Work” that shows how to predict changes in the economy’s demand for different tasks. As AI continues to evolve and tasks transform, the next decade is expected to see significant changes in task requirements across occupations.
Our team is studying how task demands of different occupations have changed over the last decade by leveraging a large dataset of online job postings. To understand how the occupations are evolving, we dive deeper into how tasks within them are shifting. The job postings provide information about the prevalence of tasks within each occupation. This is used to generate time-series data of tasks counts for each occupation. This provides a measure of employer demand over time for workers who can perform specific tasks. We additionally incorporate wages and employment shares data from the Bureau of Labor Statistics (BLS), which publishes annual statistics of the average wages and number of employees for 964 occupations. We normalize the task demand time-series data by the share of workers employed in each occupation to derive the unique task-shares dynamics data for each task-occupation pair. The changes in the occupations during that period are further characterized by the evolution of the task-shares within each occupation. This allows us to document trends in occupations and tasks as well as occupational wage terciles (low, medium, high).
Figure A. Task share dynamics of different Information Technology task clusters across HML wage occupations .
Figure B. One-step ahead predictions of task-shares of selected task clusters families across HML wage occupations .
We use the temporal aspects of the task-shares data to learn the dynamics of tasks and occupations, and then quantitatively predict the task-shares for the near future with confidence bounds. We train an autoregressive integrated moving average (ARIMA) model on this data to learn the dynamics of the task-shares of different task cluster families across High-Mid-Low wage occupations over its first 72 months (2010-2015). We then use this ARIMA model to make one-month-ahead predictions of the task-shares. The paper reports less than 5% mean absolute percentage error (MAPE) of predictions. Figure B shows the task-share forecasts (black lines) with 95% confidence intervals (grey areas) to compare against the true task-shares (dotted lines) for a few selected task cluster families across High (red line), Mid (green line), and Low (blue line) wage occupations. The accuracy of the task-share predictions is a clear indicator of the benefit of developing robust and more accurate forecasting models to characterize the evolution of occupations and their tasks. Such predictive capabilities on the labor market might help the workers reskill themselves, corporations retrain their employees, or new graduates to learn the skills to be able to execute the tasks of the future.
This empirical research sheds new light on the transformation of work by characterizing occupations in terms of task-shares dynamics. There are still many open questions remaining in the study. Today, we know the changes AI and related technologies will bring to the labor market are relatively small, but real. To prepare for continued adoption and advancements in technologies, the development of accurate, comprehensive and robust predictive models are crucial to provide guidance to educational institutions, workers, employers, and new graduates on skills and tasks of the future.
 Subhro Das, Sebastian Steffen, Wyatt Clarke, Prabhat Reddy, Erik Brynjolfsson, Martin Fleming. “Learning Occupational Task-Shares Dynamics for the Future of Work”. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2020.