Fama-French Factor Analysis: What Your Portfolio Is Really Doing

Beyond Simple Returns

Most investors evaluate their portfolio by checking total return and perhaps comparing it to the S&P 500. If you beat the index, you feel good. If you trailed it, you wonder what went wrong. But this surface-level comparison tells you almost nothing about why your portfolio performed the way it did.

Were you rewarded for taking on more risk? Did you benefit from a tailwind that lifted certain types of stocks? Or did you simply get lucky during a favorable stretch? Without decomposing your returns into their underlying drivers, you cannot answer these questions. You are flying blind, making allocation decisions based on outcomes rather than on the systematic forces that produced them.

Factor models solve this problem. They break your portfolio's returns into measurable, well-documented sources of risk and return. Instead of asking "did I beat the market," you can ask "which risks am I being compensated for, and is that compensation worth the volatility?" For FIRE investors building portfolios meant to sustain decades of withdrawals, this level of understanding is not optional. It is essential.

What Are Risk Factors?

A risk factor is a broad, persistent source of return that investors are compensated for bearing. The core insight is straightforward: some portion of your returns comes from deliberate or accidental exposure to these systematic factors, not from individual stock-picking skill or market timing.

Think of it this way. If your portfolio is heavily tilted toward small-cap stocks, and small-caps outperform large-caps over a given period, part of your return is explained by the size factor. That return did not come from alpha. It came from a known, documented risk premium that you happened to be exposed to.

Academic research has identified several factors that have persisted across decades of data, across international markets, and across different asset classes. The most robust of these form the foundation of factor models. Understanding which factors you are exposed to, and by how much, gives you a clearer picture of your portfolio's risk profile than any single performance number ever could.

CAPM: The One-Factor Model

The Capital Asset Pricing Model, developed in the 1960s by William Sharpe and others, is the simplest factor model. It makes a bold claim: all differences in expected returns across portfolios can be explained by a single factor, which is exposure to the overall market.

This exposure is measured by beta ($\beta$). Beta quantifies how sensitive your portfolio is to movements in the broad stock market. The CAPM regression is:

• Beta = 1.0: Your portfolio moves in lockstep with the market. A 10% market gain means roughly a 10% portfolio gain.
• Beta > 1.0: Your portfolio is more volatile than the market. It amplifies both gains and losses.
• Beta < 1.0: Your portfolio is less volatile. It dampens market swings, often because it includes bonds or defensive sectors.

After accounting for market exposure, any leftover return is called alpha. Alpha is the residual: the return your portfolio generated that CAPM cannot explain through market risk alone. Positive alpha means you outperformed on a risk-adjusted basis. Negative alpha means you underperformed relative to the risk you took.

CAPM was groundbreaking, but it has a well-known limitation. Empirical research consistently shows that market beta alone does not fully explain the cross-section of stock returns. Certain categories of stocks earn returns that CAPM cannot account for. This gap motivated the development of multi-factor models.

Fama-French Three-Factor Model

In 1993, Eugene Fama and Kenneth French published research demonstrating that two additional factors, beyond market risk, do a significantly better job of explaining portfolio returns. Their three-factor model became one of the most influential frameworks in modern finance.

The three factors are:

1. Market (Mkt-RF): The excess return of the broad market over the risk-free rate. This is the same beta from CAPM.
2. SMB (Small Minus Big): The return difference between small-cap and large-cap stocks. Historically, small-cap stocks have outperformed large-caps, compensating investors for the additional risks of owning smaller, less liquid companies.
3. HML (High Minus Low): The return difference between value stocks (high book-to-market ratio) and growth stocks (low book-to-market ratio). Value stocks have historically outperformed growth stocks, though this premium has been inconsistent in recent decades.

The model is expressed as a linear regression:

A positive SMB loading (the $s$ coefficient) means your portfolio tilts toward smaller companies. A positive HML loading (the $h$ coefficient) means it tilts toward value stocks. Negative loadings indicate tilts in the opposite direction: toward large-caps or growth stocks, respectively.

The practical significance is substantial. Many investors who believe they are generating alpha are actually just harvesting the size or value premium. The three-factor model strips away these known sources of return, giving you a cleaner measure of genuine outperformance.

The Five-Factor Model and Carhart Extension

Research did not stop at three factors. Two major extensions have become standard in portfolio analysis.

Fama-French Five-Factor Model (2015)

Fama and French expanded their original model by adding two more factors grounded in corporate fundamentals:

• RMW (Robust Minus Weak): The return difference between firms with robust (high) profitability and firms with weak (low) profitability. Companies that generate strong operating profits tend to outperform those that do not. This is sometimes called the profitability or quality factor.
• CMA (Conservative Minus Aggressive): The return difference between firms that invest conservatively and firms that invest aggressively. Companies that grow their asset base slowly tend to outperform those that reinvest heavily. This challenges the intuition that aggressive growth is always rewarded by the market.

The five-factor regression adds RMW and CMA to the original three:

Carhart Four-Factor Model (1997)

Mark Carhart extended the original three-factor model by adding a momentum factor:

• MOM (Momentum): The return difference between stocks that have performed well over the past 3 to 12 months and stocks that have performed poorly. Momentum is one of the most robust anomalies in finance. Stocks that have been rising tend to continue rising in the short term, and stocks that have been falling tend to continue falling.

The Carhart model extends the original three-factor model with momentum:

All of these models work the same way mechanically. You run a regression of your portfolio's excess returns (returns above the risk-free rate) against the relevant factor returns. The regression coefficients tell you how much of your return is attributable to each factor. The intercept of the regression is your alpha.

Factor return data is published monthly by Kenneth French on his data library and is freely available. This is the same data that Summitward uses in its portfolio analysis tools to compute your factor exposures.

Reading Your Factor Exposures

Running a factor regression produces a set of numbers for each factor. Here is how to interpret the key outputs.

Factor Loadings (Coefficients)

Each factor has a loading, which is the regression coefficient. This number tells you the sensitivity of your portfolio to that factor.

For example, a loading of 0.3 on SMB means that for every 1% the small-cap premium moves, your portfolio moves 0.3% in the same direction, all else being equal. A loading of -0.2 on HML means your portfolio has a growth tilt: when value outperforms growth by 1%, your portfolio gives up 0.2%.

T-Statistics

The t-statistic measures whether a factor loading is statistically significant or just noise. As a general rule of thumb, a t-statistic with an absolute value above 2.0 indicates significance at approximately the 95% confidence level. If your SMB loading is 0.15 but the t-statistic is only 0.8, you should not read too much into that number. The data does not support a meaningful small-cap tilt.

Alpha ($\alpha$)

Alpha is the return your portfolio generated above and beyond what the factor exposures explain. It represents genuine outperformance (or underperformance) that cannot be attributed to known risk premiums.

Positive, statistically significant alpha is the holy grail of active management. It means you added value beyond what could be achieved by simply tilting toward known factors. However, most diversified index portfolios will show near-zero alpha, and that is perfectly fine. It means you are capturing factor premiums efficiently without paying excess fees or taking uncompensated risks.

R-Squared ($R^2$)

$R^2$ tells you what percentage of your portfolio's return variation is explained by the factor model. A three-factor $R^2$ of 0.95 means that 95% of your portfolio's return movements can be attributed to market, size, and value exposures. The remaining 5% is idiosyncratic (stock-specific or unexplained). Higher R-squared values are typical of diversified portfolios. Lower values suggest concentrated positions or exposure to factors not captured by the model.

Practical Implications for FIRE Investors

Factor analysis is not just an academic exercise. It has concrete implications for how you build and maintain a portfolio designed to fund decades of financial independence.

1. Total Market Index Funds Have Neutral Factor Exposures

If you hold a total US stock market index fund, your factor exposures will be close to beta = 1.0, SMB near 0, and HML near 0. This is the expected baseline. You are capturing the equity risk premium without meaningful tilts toward any other factor. For most FIRE investors, this is a perfectly sound approach. You are not leaving money on the table; you are capturing the largest and most reliable risk premium (market risk) at the lowest possible cost.

2. Small-Cap Value Tilts Carry Real Trade-Offs

Many FIRE investors deliberately tilt toward small-cap value, motivated by the historical evidence that SMB and HML premiums have boosted long-term returns. If you hold funds like VBR (Vanguard Small-Cap Value), you should see positive SMB and HML loadings in your factor analysis.

However, this tilt comes with trade-offs you must accept. Higher expected returns come with higher volatility, larger drawdowns, and significant tracking error relative to the broad market. There will be multi-year periods where your small-cap value tilt underperforms a simple total market fund. During those periods, the temptation to abandon the strategy is strong. Factor analysis helps you understand that the underperformance is not a flaw in your approach. It is the cost of harvesting the premium.

3. Factor Analysis Reveals Unintended Bets

Your "diversified" portfolio might contain hidden factor tilts you never intended. For instance, if you overweight technology stocks through sector-specific funds or individual holdings, you may have an unintentional growth bias (negative HML loading) and possibly a momentum tilt. Factor analysis makes these implicit bets explicit so you can decide whether you are comfortable with them.

This is especially relevant for FIRE investors who hold employer stock, concentrated positions from exercised options, or thematic ETFs. Running a factor regression on your actual portfolio, rather than your target allocation, can reveal surprising exposures.

4. Factor Premiums Are Not Guaranteed in Any Given Period

The value premium (HML) was negative for much of the 2010s. Small-cap stocks underperformed large-caps for extended stretches. Momentum experienced sharp reversals in 2009 and 2020. These are not reasons to dismiss factor investing. They are reasons to set realistic expectations.

The long-term evidence across global markets and across decades of data supports the existence of these premiums. But "long-term" means 20 years or more, not 3 to 5. If your FIRE timeline depends on factor premiums materializing over a short horizon, you are taking on more risk than you may realize. The discipline to hold through dry spells is the price of admission for factor-based strategies.

A Worked Example

Consider a classic three-fund portfolio: 60% US total stock market, 30% international developed markets, and 10% US aggregate bonds. This is a common allocation among FIRE investors in the accumulation phase.

Running a Fama-French three-factor regression on this portfolio's monthly returns over a 10-year period might produce results like the following:

• Beta (Market): 0.62. The portfolio has moderate market exposure, well below 1.0. The 10% bond allocation and the international diversification (which does not perfectly correlate with the US market factor) reduce the effective beta.
• SMB: -0.05 (t-stat: -0.7). A tiny negative loading on the size factor, but statistically insignificant. The portfolio has no meaningful small-cap tilt. This is expected, since total market index funds are market-cap weighted and dominated by large-cap stocks.
• HML: 0.02 (t-stat: 0.3). Essentially zero value/growth tilt. Again, exactly what you would expect from a broad index approach.
• Alpha: 0.01% monthly (t-stat: 0.2). Not statistically significant. The portfolio is not generating excess returns beyond its factor exposures, nor is it losing value to fees or poor timing.
• R-squared: 0.94. The three factors explain 94% of the portfolio's return variation.

The interpretation is clean. This portfolio delivers exactly what it promises: efficient, low-cost exposure to the market risk premium with no meaningful factor tilts and no alpha in either direction. For a passive investor, this is a good result. It confirms that you are getting what you paid for and nothing you did not intend.

Now contrast this with a portfolio that holds 40% US small-cap value, 40% US large-cap blend, and 20% bonds. You would expect to see a higher SMB loading (perhaps 0.25 to 0.35), a positive HML loading (perhaps 0.15 to 0.25), and a slightly lower beta (due to bonds). If the factor analysis shows these loadings, your portfolio is doing what you designed it to do. If the loadings are different from what you expected, it is time to investigate why.

Related Guides

Factor analysis is one piece of the portfolio management puzzle. To go deeper on related topics:

• RMW Explained: The Profitability Factor is the deep dive on RMW specifically: Novy-Marx’s intuition, out-of-sample evidence, the international Japan failure, and the Avantis-vs-Dimensional cash-vs-operating profitability disagreement, with a tilt + tracking-error calculator.
• Tax-Aware Long-Short: Real Tax Alpha or Marketing? is the tax-aware implementation of long-short factor investing for HNW taxable investors, with the AQR research on loss capacity and the "would you own this pre-tax?" screen.
• Lifecycle Asset Allocation explains why your age and human capital should drive your stock/bond split, the decision that determines which factor exposures you carry.
• Concentration Risk covers when a single stock dominates your portfolio and how to diversify, a situation where factor analysis reveals hidden bets.
• RSU Tax Strategy addresses the tax side of concentrated positions from employer stock, which often drive unintended factor tilts.
• Tax-Loss Harvesting shows how to capture tax benefits when rebalancing factor exposures without losing your market position.
• Monte Carlo Simulation stress-tests your portfolio across thousands of scenarios, complementing factor analysis with forward-looking risk assessment.

Key Takeaways

• Factor models decompose returns into systematic sources of risk. Instead of asking whether you beat the market, ask which risk premiums drove your returns. This gives you actionable insight into your portfolio's behavior.
• Alpha is what remains after factor exposures are accounted for. Most index-based portfolios will show near-zero alpha, and that is the expected, correct outcome. Do not chase alpha at the expense of higher fees and unnecessary complexity.
• Factor tilts increase expected returns but also increase volatility and tracking error. If you deliberately tilt toward small-cap value or other factor premiums, understand the full cost in terms of drawdowns and multi-year underperformance relative to the broad market.
• Factor analysis reveals hidden bets in your portfolio. Run a regression on your actual holdings periodically. You may discover unintended exposures from concentrated positions, sector overweights, or thematic funds that shift your risk profile away from your target.
• Factor premiums require long holding periods to materialize. The evidence supports their existence over decades, not quarters. For FIRE investors with 30 to 50 year horizons, this patience is a structural advantage, but only if you have the discipline to stay the course when the premiums are not showing up.