Farmer checking corn plant

Fields of Dreams: The Growing Disparity Between Yield Forecasts and Reality

  • Persistent Yield Shortfalls: Since 2019 for corn and 2022 for soybeans, actual US yields have consistently fallen short of early USDA projections, challenging the reliability of long-term linear trend models.
  • Weather’s Growing Influence: While yield models assume steady technological progress, recent La Niña-driven droughts and broader climate variability have introduced significant volatility, distorting short-term results.
  • Modeling Debate: Some analysts advocate for a shorter lookback window (from 2013) to better reflect recent underperformance. However, this risks overfitting to anomalies and ignoring the long-term resilience of yield trends.
  • Market Implications: If the market overreacts based on recent disappointments in USDA projections, grain and oilseed futures may become overpriced—creating tactical opportunities for bearish positioning.

What happened:

After under-promising and over-delivering for much of the first half of the last decade, realized end-of-season yields have come in below early season United States Department of Agriculture (USDA) estimates consistently since 2019 for corn and 2022 for soybeans. When it comes to yield projection, past performance is assumed to inform future results. In fact, the most common approach to generating a basic estimate of US corn and soybean yields involves deriving an “unconditional” linear trend estimate from historical yield data starting in the 1980s. This long-term linear trend assumes a constant yearly step-wise increase in per acre crop yield to account for expected productivity gains. For much of the last 45 years, this has proven to be a safe assumption as the advent of new technologies and practices such as: development and use of genetically modified (GMO) seeds that are more resistant to drought and disease, utilization of more effective fertilizer application and irrigation techniques, and implementation of better soil management techniques.

However, analysts have begun to argue that the unconditional linear trend model is no longer a good estimate of future yields, as realized yield has consistently undershot the yield predicted by this model in recent periods. The argument rests on the assumption that the recent yield underperformance versus expectations demonstrates a structural change in US agriculture that indicates a slow-down in improvements in US per acre productivity; and that a linear estimator of future yields should be based on a shorter lookback window starting in 2013 rather than the 1980s. Is there a kernel of truth to the criticisms of the linear trend model, and is a replacement warranted given recent performance?

Why it happened:

As stated above, the current yield forecasting framework relies on the steady march of technological progress. However, human-driven inputs are not the only drivers of agricultural yield. Weather is also a major driver of crop outcomes, at times exerting enough influence to overshadow gains from technological progress when it comes to determining the short-term trend in yields. In contrast to relatively stable, linear assumptions about technological progress, the impact of weather varies widely across geographies and through time. This variability in weather conditions from year-to-year means that, over long enough time horizons, headwinds to yields from poor weather are offset by tailwinds from beneficial conditions. At least that’s usually what happens. During the first part of the decade, crops experienced an uncharacteristic string of poor growing conditions due to persistent La Niña conditions, which can be associated with drought conditions such as those we experienced in the US in the spring of 2022.[1] The weather anomaly led to significant negative deviations from the unconditional trend yield in the recent past. Utilizing a yield prediction model with a shorter “look-back” would overemphasize this low probability event. Further, a changing climate and the associated weather impacts in crop-growing regions are not directional. That is, the range of weather outcomes is wider, not just worse, exposing potential yields to larger deviations on both the negative and positive side.

Our view:

We are sympathetic to arguments against a simple linear trendline such as: slowing technological progress, climate change, and the practice of seeding more marginal land. However, we continue to believe “the trend is our friend,” as the approach has stood the test of time. For example, previous short-term periods of high deviation, such as the 1970s and 80s, ultimately proved transient. It is also worth noting that those were consistently positive deviations from the trend, in contrast to today; and adjustments based on only that data would have resulted in overly rosy forecasts for yields in the following periods, demonstrating the value of a longer look-back. From a commodity trading perspective, if an increasing number of market participants subscribe to the belief that the linear model is no longer applicable due to a short period of yield overestimation, the market may be consistently too conservative on early-season yield estimates, causing grain and oilseeds futures and calendar spreads to become overvalued, and possibly opening opportunities to profit from bearish positioning.