Can machine learning models better predict stock returns if they are tailored to specific industries, or is a one-size-fits-all (generalist) approach sufficient? This question lies at the heart of a recent research paper by Matthias Hanauer, Amar Soebhag, Marc Stam, and Tobias Hoogteijling. Their findings suggest that the optimal solution lies somewhere in between: a “Hybrid” machine learning model that is aware of industry structures but still trained on the full cross-section of stocks offers the best performance.

The authors examine three types of models: a Generalist model trained on all stocks irrespective of sector, Specialist models trained separately within each of the 12 Fama-French industries, and a Hybrid model that combines the benefits of both. The Hybrid model applies industry-level normalization to returns and features—removing sector-specific biases—but retains the full dataset during training. This design improves the signal-to-noise ratio without fragmenting the data too much, as Specialist models do. In effect, the Hybrid model builds sector-neutral forecasts that benefit from large sample sizes and sector-awareness.

To test these approaches, the researchers use a comprehensive dataset of U.S. stock returns from 1957 to 2023, enriched with 153 firm-level characteristics. They employ several machine learning algorithms—elastic nets, gradient-boosted trees, neural networks, and an ensemble learner—to evaluate out-of-sample predictive power. Their results show that while Specialist models sometimes offer unique insights, their limited data leads to lower performance and less stable predictions. Generalist models perform well overall but suffer from higher portfolio volatility and unintended sector tilts. The Hybrid models, by contrast, deliver the best balance of statistical accuracy and economic performance.

Portfolios constructed using predictions from the Hybrid model exhibit higher Sharpe ratios, lower volatility, and smaller drawdowns than those based on Generalist or Specialist models. Notably, spanning tests reveal that the Hybrid portfolio cannot be replicated by any combination of the other two, highlighting its unique value. These results hold not just in the U.S. but also across international markets, suggesting that the hybridization of global and sector-specific information is a powerful approach to return forecasting. For practitioners, the takeaway is clear: machine learning models don’t need to be sector-exclusive experts, but they absolutely benefit from respecting sector boundaries.

Authors: Hanauer, Matthias Xaver and Soebhag, Amar and Stam, Marc and Hoogteijling, Tobias,

Title: Do Machine Learning Models Need to Be Sector Experts?

Link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5224253

Abstract:

We examine heterogeneous return predictability at the industry level using machine learning models trained on a comprehensive set of firm characteristics. We compare uniform (“Generalist”) models with industry-specific (“Specialist”) models and introduce a “Hybrid” model that incorporates industry membership. The Hybrid model outperforms the Specialist model in out-of-sample performance, yielding higher Sharpe ratios and lower risk compared to both alternatives. Additional analyses using international data corroborate these findings. Our results indicate that the Hybrid model benefits from a better signal-to-noise ratio by combining industry awareness with broader sample sizes, improving both estimation precision and learning efficiency.

As always, we present several interesting figures and tables:

Notable quotations from the academic research paper:

“Although the Generalist model is the standard approach in the literature (cf., Gu, Kelly, and Xiu, 2020), it is simplistic and lacks a clear economic motivation. This one-size-fitsall approach assigns equal importance to all stocks in the training process (Howard, 2024), and implicitly assumes homogeneous predictability across firms, whereas Patton and Weller (2022) show that there is strong heterogeneity in responses to risk factors in the cross-section of U.S. stocks. Allowing for heterogeneous components in the Stochastic Discount Factor (SDF), as in our Specialist specification, helps to alleviate these problems. However, training pure sector-based models ignores common components in the SDF, which are found to be strong according to Hellum, Pedersen, and Rønn-Nielsen (2023). Additionally, partitioning the cross-section of stocks into 12 smaller groups might severely limit the learning capacity of complex machine learning models, as these are known to be “data-hungry”.

To address these concerns, we introduce a “Hybrid” specification. In this specification, we fit ML models to the full cross-section of stocks but with the underlying data processed as in the Specialist specification. The idea behind considering Specialist and Hybrid specifications is that we implicitly force ML models to embed a sector structure when constructing mappings from characteristics to stock returns. This set-up accounts for the conditional role of industries in asset pricing as described by Moskowitz and Grinblatt (1999) (in contrast to a Generalist model), but does not suffer from small data problems (in contrast to a Specialist model).”

Are you looking for more strategies to read about? Sign up for our newsletter or visit our Blog or Screener.

Do you want to learn more about Quantpedia Premium service? Check how Quantpedia works, our mission and Premium pricing offer.

Do you want to learn more about Quantpedia Pro service? Check its description, watch videos, review reporting capabilities and visit our pricing offer.

Are you looking for historical data or backtesting platforms? Check our list of Algo Trading Discounts.

Or follow us on:

Facebook Group, Facebook Page, Twitter, Linkedin, Medium or Youtube