Whenever we build a statistical model, we face a trade-off between flexibility and interpretability. We can do a simple linear model, easy to understand, interpret and use for inference. However, linear relationships are not enough for modeling complex relations. On the other hand, we can go for a deep learning model with multiple layers of neurons. These models are good at modeling complex relationships, but they need lots of data and are hard to interpret.

GAMs are right in the middle! They can be fit to complex non-linear relationships. At the same time, we are able to do inferential statistics and understand the underlying “why” in the model.

In this article, I take a first step to describe GAMs. Let’s say we have multiple predictors \(X_{1}, X_{2}, ... , X_{P}\) . Our goal is to estimate a general regression model:

\[Y=f(X_{1},X_{2},...,X_{P}) + \epsilon\]

Estimating this model can be demanding. There are many interaction terms that we need to think about.

In GAMs, we model this relationship as follows:

\[E(Y|X_{1},X_{2},...,X_{P})=\alpha + f_{1}(X_{1}) + ,f_{2}(X_{2})+...+f_{P}(X_{P})\]

Here, the \(f_{j}\) is a nonparametric function and relates the predictor variables to the outcome, non-linearly.

In a general format, we can relate the conditional mean of \(Y\) with a function \(g\) as follows:

\[g(E(Y))=\alpha + f_{1}(X_{1}) + ,f_{2}(X_{2})+...+f_{P}(X_{P})\]

Here, \(g\) can be an identity link, a logit function (for two-class classification), a probit function, or a log function.

With GAMs, we have both the flexibility in modeling non-linear relationships between the dependent and independent variables, and also we are able to interpret the relations. Note that in our model setup, we can also assume linear \(f_{j}\) functions. Or include interactive terms. For example, in predicting excess returns using stock characteristics, besides the characteristics individually, we can bring in the interaction of macroeconomic variables with the stock characteristics.

Next, I will discuss the GAMs estimation procedure.