Generalized Additive Models
The engine for Non-Linear Discovery. GAMs audit complex, wiggly relationships using smoothing splines, allowing the data to dictate its own 'shape' rather than forcing a straight-line narrative.
What is it?
Generalized Additive Models (GAM) extend linear regression by allowing non-linear relationship fits using smooth basis function splines, rather than forcing straight-line models.
When to use it
- Non-Linear Curvature: Predictor relations curve, wave, or bend non-linearly.
- Flexible Smoothness: Balance linear straightness vs. wiggly overfitting.
- Interpretability: Fit flexible curls without complex high-degree polynomials.
Spline vs. OLS Line
Compare a standard OLS straight line fit (poor representation of curved data) against a flexible GAM spline curve fit:
GAM Spline Fitting Laboratory
Adjust spline wiggliness (basis functions df) to see the spline curve adapt to non-linear noise.
Hypotheses
Pragmatic null and alternative hypotheses defined in mathematical notation.
H₀: f(X) = β₀ (smooth function has no effect, reduces to intercept-only)
Hₐ: f(X) ≠ β₀ (smooth function has non-zero effect)
Tests are performed for each smooth term. Can test linear vs non-linear using approximate F-tests or AIC/BIC comparisons.
Assumptions
The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.
Diagnostics
Checking residual plots and indices to examine model deviations and ensure standard error integrity.
- gam.check() output: basis dimension adequacy (k-index), residual plots, QQ-plots
- Partial effect plots: visualize each smooth function with confidence bands
- Concurvity check: concurvity(model) to detect collinearity among smooths
- Compare model to linear version using AIC/BIC (anova(linear_model, gam_model))
- Cross-validation to assess predictive performance and overfitting
- Check effective degrees of freedom (edf) for each smooth; edf ≈ 1 suggests linear
- Residual autocorrelation plots (ACF) if time series or spatial data
- Influence diagnostics to detect outliers affecting smooth estimation
Applied Minds
Review concrete study examples, data layout guidelines, and copy executable syntax scripts.
Non-linear Relationship Between Age and Cognitive Function
Research question: How does cognitive function change with age, accounting for education and gender? Design: Cross-sectional (n=400). Outcome: Cognitive test score (continuous, 0-100). Predictors: Age (non-linear relationship expected), education, gender. GAM captures U-shaped or inverted-U patterns that linear regression misses.
# GAM: Non-linear age effects on cognitive function
library(mgcv) # For GAM
library(ggplot2)
library(gratia) # For GAM visualization
set.seed(2025)
n <- 400
data <- data.frame(
age = runif(n, 20, 85),
education = rnorm(n, 14, 3),
gender = sample(c("Male", "Female"), n, replace=TRUE)
)
data$education <- pmax(8, pmin(22, data$education))
# Cognitive score: inverted-U with age, steeper decline after 65
age_effect <- 85 - 0.3*(data$age-45)^2/10 -
ifelse(data$age > 65, 2*(data$age-65), 0)
data$cognitive <- age_effect +
1.5*data$education +
ifelse(data$gender=="Female", 3, 0) +
rnorm(n, 0, 8)
data$cognitive <- pmax(0, pmin(100, data$cognitive))
# === STEP 1: Compare Linear vs GAM ===
# Linear model (misspecified)
lm_model <- lm(cognitive ~ age + education + gender, data=data)
summary(lm_model)
# GAM with smooth for age
gam_model <- gam(cognitive ~ s(age, k=10) + education + gender,
data=data, method="REML")
summary(gam_model)
# Compare models
AIC(lm_model, gam_model)
anova(lm_model, gam_model, test="F")
# GAM should have significantly better fit
# === STEP 2: Check GAM Diagnostics ===
gam.check(gam_model)
# Check: k-index >1 (basis adequate)
# Check: residual plots show no pattern
# Check: QQ-plot shows normality
# Concurvity (collinearity for smooths)
concurvity(gam_model, full=TRUE)
# Values <0.8 indicate no problematic concurvity
# === STEP 3: Visualize Smooth Function ===
# Partial effect plot for age
plot(gam_model, select=1, shade=TRUE, shade.col="lightblue",
main="Non-linear Effect of Age on Cognitive Function",
xlab="Age(years)", ylab="s(Age)", rug=TRUE)
# Using gratia for nicer plots
draw(gam_model, residuals=TRUE)
# Manual prediction plot
age_seq <- seq(20, 85, length=100)
pred_data <- data.frame(
age = age_seq,
education = mean(data$education),
gender = "Male"
)
pred <- predict(gam_model, newdata=pred_data, se.fit=TRUE)
pred_data$fit <- pred$fit
pred_data$lower <- pred$fit - 1.96*pred$se.fit
pred_data$upper <- pred$fit + 1.96*pred$se.fit
ggplot(pred_data, aes(x=age, y=fit)) +
geom_line(color="blue", size=1.2) +
geom_ribbon(aes(ymin=lower, ymax=upper), alpha=0.3, fill="blue") +
geom_point(data=data, aes(x=age, y=cognitive), alpha=0.3) +
labs(title="Predicted Cognitive Function by Age(GAM)",
subtitle="Male with average education",
x="Age(years)", y="Cognitive Score(0-100)") +
theme_classic()
# === STEP 4: Test Smooth Significance ===
# Approximate F-test for smooth term
summary(gam_model)$s.table
# edf: effective degrees of freedom (1 = linear, >1 = non-linear)
# p-value: test if smooth differs from zero
# === STEP 5: Compare to Polynomial ===
poly_model <- lm(cognitive ~ poly(age, 3) + education + gender, data=data)
AIC(lm_model, poly_model, gam_model)
# GAM typically has lowest AIC
cat("\n=== Interpretation ===")
cat("\nGAM revealed significant non-linear age effect(edf=",
round(summary(gam_model)$s.table[1,"edf"], 2), ", p<.001).")
cat("\nCognitive function peaks around age 45-50, then declines,")
cat("\nwith accelerated decline after 65. Linear model(AIC=",
round(AIC(lm_model)), ") misses this pattern compared to GAM(AIC=",
round(AIC(gam_model)), ").")GAM revealed significant non-linear age effect (edf=5.8, p<.001), capturing inverted-U pattern. Cognitive scores peak around age 45-50 (predicted score ~78), declining to ~55 by age 85. Linear model severely misspecified (AIC=2845 vs GAM AIC=2720, Δ=125). Education showed linear positive effect (+1.5 points per year, p<.001). Findings consistent with cognitive aging literature showing accelerated decline in late life (Salthouse, 2009).
Alternatives
Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.
- OLS Regression — Simplify the model if the 'Effective Degrees of Freedom' (edf) is near 1.0.
- Linear Multiple Regression — Return to the most efficient parsimonious path.
- Basis Dimension Audit — Increase 'k' to allow the model more flexibility to capture the signal.
- REML Selection — Optimize the smoothing parameter to balance fit against parsimony.
- Multiple Imputation GAM — Mathematically reconstruct missing scores before fitting the non-linear curve.
Post-hoc
Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.
- Compare different smoothing parameter selection methods (GCV, REML, ML)
- Vary basis dimension (k) and check stability of smooth terms
- Compare with parametric alternatives (polynomial regression)
- Examine concurvity (GAM equivalent of multicollinearity)
- Use gam.check() diagnostics for residual patterns
GAMs model non-linear relationships. Traditional post-hoc tests are not directly applicable.
Effect Size
Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.
Effective degrees of freedom quantify 'wiggliness' of smooth: edf ≈ 1 indicates linear relationship; edf > 1 indicates non-linearity. edf = 5 means smooth uses ~5 parameters. Maximum edf = k-1 where k is basis dimension
Analogous to R² but for GLMs. Proportion of deviance explained by model. Values typically lower than OLS R²
Visualize contribution of each smooth term holding others constant. Y-axis shows change in outcome (or log-odds, log-rate) per unit change in X
0.2
0.5
0.8
Sample Size
Guidelines for minimum sample requirements and power analysis parameters.
The 'Wiggliness Buffer': A minimum of 25 participants per smoothing spline term is required. Flexible curves collapse into mathematical phantoms if the temporal or score depth is too shallow to allow the 'Bends' to emerge.
| Effect Size | Parameters | Required n |
|---|---|---|
| Small Effect | Low Curvature (edf=1.5) | n ≈ 450 total |
| Medium Effect | Moderate Curvature (edf=3.0) | n ≈ 120 total |
| Large Effect | High Curvature (edf=5.0) | n ≈ 60 total |
The 'Over-fitting Penalty': Every level of spline complexity (k) 'Consumes' power. If your N is small, use a low 'k' (e.g., k=3) to protect your discovery from measuring random sampling ripples as 'Real' curves.
Reporting
How to compile statistical results into publication prose matching APA and journal style guides.
A generalized additive model (GAM) was fit using family and link function, e.g., 'Gaussian identity link' or 'Poisson log link'. Describe smooth terms: 'Smooth terms were included for X1 and X2 using cubic regression splines (k=10 basis functions).' Describe parametric terms: 'Gender was included as a categorical factor.' Model selection used REML for smoothing parameter estimation. Diagnostics confirmed adequate basis dimensions (k-index >1 for all smooths) and appropriate residual distribution. For each smooth: The effect of X1 was significantly non-linear (edf = X.X, p < .001), showing describe pattern: U-shaped, inverted-U, monotonic, etc.. The GAM explained X% of deviance and had better/similar fit compared to a linear model (ΔAIC = X.X, p < .001).
- Effective degrees of freedom (edf) for each smooth with p-values
- Deviance explained (or adjusted R² for Gaussian)
- AIC/BIC comparison to linear model
- Basis dimension adequacy (k-index from gam.check)
- Sample size and family/link
- Description of smooth patterns (with plots)
Manuscript Lab
Copy standard summary tables and forensic reporting grids to outline analysis details.
| Parametric Term | Estimate | SE | t | p |
|---|---|---|---|---|
| (Intercept) | 25.4 | 0.85 | 29.88 | < .001 |
| Gender (Male) | 1.12 | 0.45 | 2.48 | .014 |
The 'Wiggle' Meter. edf = 1 is a straight line. edf > 1 indicates a non-linear, flexible curve. Higher = more complex relationship.
The Smoother. A mathematical function that allows the relationship between X and Y to change shape automatically based on data.
Command Center
Syntax libraries and function parameters for executing calculations in stats packages.
# 1. Fit GAM with Smoothers
model <- mgcv::gam(score ~ gender + s(age) + s(sleep), data = df)
summary(model)
# 2. Visualize Non-linear Splines
gratia::draw(model)GAMs are the ultimate diagnostic tool. If you suspect your linear model is missing a curve, use a GAM to 'find' the shape of the relationship.
# Execute Basis Dimension Audit (Are the curves too wiggly?)
mgcv::gam.check(model)
# Compare GAM vs Linear OLS
performance::compare_performance(ols_mod, gam_mod)Common Mistakes
Analytical caveats and corrections to maintain modeling integrity.
References
Scholarly lineage and citation keys grounding the statistical framework.