statminds

GLM (Two-Group Within-Subjects Model)Parametric Reference12-stage workflow

Paired T-Test

The engine for Internal Discovery. This model audits the change within the same participants (e.g., Pre-test vs. Post-test), using each individual as their own baseline to reveal pure recovery signals.

Model familyGLM (Two-Group Within-Subjects Model)

Hypothesistwo-tailed

AliasesDependent Samples T-Test · Matched Pairs T-Test · Repeated Measures T-Test (2 Levels)

Recovery Trajectory Audit

Determine if an intervention causes a significant mean shift within the same group over time.

Within-Person Precision

Neutralize the noise of individual differences by calculating the 'Difference Score' for every participant.

Pre-Post Signal Discovery

Quantify the magnitude of clinical improvement from a fixed starting point.

What is it?

Paired Samples T-Test compares the means of two dependent, linked measurements taken from the identical subjects (e.g. Pre-Test vs. Post-Test scores).

When to use it

Before & After: Track performance shift within individuals over time.
Matched Pairs: Subjects are matched (e.g., twins, sibling pairs).

Core Idea

Instead of comparing two broad group spreads, we isolate and track the individual changes (slopes) directly within each subject:

By focusing exclusively on the within-subject differences, we strip away individual baseline differences, vastly increasing statistical power.

Hypotheses

H0: Mean Difference = 0 (No systematic change after intervention)

Ha: Mean Difference != 0 (A significant average change exists)

How it works

Compute difference (d = Post - Pre) for each individual.
Find average difference (d-bar) and standard deviation of differences (s_d).
Calculate Standard Error: SE = s_d / sqrt(N).
Compute t = d-bar / SE. Test against df = N - 1.

Assumptions

📊 Normality: The pair differences (d) are normally distributed.

👤 Paired Observations: Each subject has exactly two linked measurements.

Effect Size

Standardized shift is measured with Cohen's d for paired samples: d = d-bar / s_d. Benchmarks: 0.2 (small), 0.5 (medium), and 0.8 (large).

Quick Example

Subject	Pre-Test Score	Post-Test Score
S1	72.0	78.0 (+6.0)
S2	68.0	74.0 (+6.0)
Mean Shift	+6.0 (p = 0.008)

Interactive Sandbox

Paired Slope Line Live Laboratory

Change the average pre-to-post shift and difference SD to watch parallel slopes split or cross.

Presets

Mean Shift Difference4.0

Difference SD3.0

Sample Size (N)12

Individual Pre-to-Post Slope lines (Pre: left axis; Post: right axis; score Y range: 20..70)Green slopes indicate increase; Amber slopes indicate decrease

Calculations Output

Metric	Value
Mean Difference (D-bar)	3.5764
Difference SD (s_D)	3.1366
t-statistic	3.9497
p-value	0.0001

Statistical Verdict

✅ Significant Paired Shift

Mean difference shifts significantly from zero (d_bar = 3.58, t = 3.950, p = < 0.001). Reject H0.

The 12-Stage Precision Workflow

01Difference Parity

Hypotheses

We test the null of zero shift (μ_diff = 0) against the discovery of a non-zero trajectory of change.

02Difference Normality

Assumptions

The 'Hidden Mandate': It is the *differences* between pairs that must follow the bell curve, not the raw scores themselves.

03Trajectory Forensics

Diagnostics

Visualizing 'Before & After' spaghetti plots to ensure the mean shift isn't being driven by a tiny subset of participants.

04focus

Auditing FlowMotion participants' Pain Levels at Baseline vs. 12-week follow-up to measure pure internal recovery.

05Signed-Rank Pivot

Alternatives

Knowing when to switch to Wilcoxon Signed-Rank if the difference scores are severely skewed or contain extreme outliers.

06The Shift Strike

Significance

Executing the T-test on the 'Delta' scores—a precision strike on the internal movement of the data.

07Paired Cohen’s d

Effect Size

Interpreting the magnitude of change: quantifying how many standard deviation units the average participant moved during the study.

08The Power Advantage

Sample Size

Exploiting the 'Within-Subject Multiplier'—higher correlation between pairs allows for significantly more power with fewer participants.

09The Trajectory Statement

Reporting

Reporting the mean difference and SD: 'Participants showed a significant decrease in symptoms (M_diff = X.X, t(df) = X.XX, p = .XXX).'

10Paired = TRUE

Software

Executing 't.test(paired = TRUE)'—the command that tells the algorithm to respect the dependency between observations.

11focus

The fatal error of treating paired data as independent, which ignores the ICC and dangerously deflates your statistical power.

12focus

Tracing the model back to the foundational principles of repeated measures and the formalization of matched-pair forensics.

01Hypothesis test logic

Hypotheses

Pragmatic null and alternative hypotheses defined in mathematical notation.

We are not testing two means. We are testing one mean: the mean of the differences. Is the shift real, or is it zero?

The Delta Logic

Null · H₀

H₀: μD = 0 (mean difference between paired observations is zero)

Alternative · Hₐ

Hₐ: μD ≠ 0 (mean difference is not zero)

Why it matters two-tailed

Tests mean of difference scores. Can be one-tailed if directional change predicted.

02Model diagnostics

Assumptions

The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.

Paired tests rely on the correlation between timepoints. If the pairs aren't correlated, you have wasted power. The difference scores must be normal, not the raw data.

Correlation is Key

Assumptions

Critical / High Severity

How to check

Quick

Inspect variable type; create histograms & summary statistics to verify scale is not discrete/categorical; check that both measurements use same scale

Rigorous

Formal measurement scale validation; verify measurement units are truly interval/ratio (equal distances between values); confirm measurement precision is adequate

If violated

If ordinal with limited categories (e.g., Likert 1-5) → use Wilcoxon signed-rank test (nonparametric paired test). If many ordinal levels (7+) and approximately interval-like → may proceed cautiously with paired t-test. If binary outcome → use McNemar's test for paired categorical data. If count data → consider Poisson or negative binomial models

wilcoxon signed rankmcnemarsign test

How to check

Quick

Design review; verify each observation has corresponding pair (same subject at Time 1 and Time 2, or matched pair members); check that n is same for both measurements

Rigorous

Verify subject IDs match across timepoints; check that pairing order is preserved; ensure no mixing of independent and paired observations; verify matching criteria if using matched pairs design

If violated

If independent groups (different subjects) → use independent t-test or Welch's t-test. If >2 timepoints/conditions on same subjects → use one-way repeated measures ANOVA or LMM. If mixture of paired and unpaired → analyze separately or use LMM that can handle unbalanced data

independent t testone way repeated measures anovalinear mixed model

How to check

Quick

Data audit; verify subject IDs match between timepoints; check for missing pairs; look for duplicated IDs; verify time order (pre before post)

Rigorous

Cross-check against source data; verify randomization logs if applicable; check for outlying difference scores that might indicate pairing errors; ensure temporal ordering is correct

If violated

Fix pairing errors in data before analysis. If many unpaired observations → exclude unpaired cases (but report how many excluded) or use independent t-test (loses statistical power substantially). If pairing uncertain → treat as independent groups (conservative approach). Never guess at pairings

independent t test

How to check

Quick

Compute difference scores (Time2 - Time1 or Post - Pre); create boxplot of differences; identify values >1.5 IQR beyond quartiles; check for extreme z-scores (|z| > 3) in differences

Rigorous

Standardized/studentized residuals of differences (|r| > 3 suspect); check if outliers are due to data entry errors or represent true extreme responders

If violated

First: verify data entry errors and correct if found (e.g., decimal point errors). If legitimate outliers: (1) Report results with and without outliers to assess sensitivity; (2) Use Wilcoxon signed-rank test (rank-based, resistant to outliers); (3) Use trimmed mean paired test (10-20% trimming); (4) Winsorize extreme difference scores to 5th/95th percentiles (pre-specify threshold); (5) Bootstrap confidence intervals. Never remove outliers without justification and transparency

wilcoxon signed ranksign test

How to check

Quick

Compute differences (D = Time2 - Time1); Q-Q plot of differences (points should fall on diagonal); histogram of differences; visual inspection for severe skewness or heavy tails

Rigorous

Shapiro-Wilk test on differences (n < 50; p > .05 indicates normality); Anderson-Darling test; check skewness (<|2|) and kurtosis (<|7|) of differences

If violated

If mild violation with moderate sample size (n ≥ 30 pairs): paired t-test is robust via Central Limit Theorem, proceed. If notable violation: (1) Transform differences if they have consistent skew pattern (log for right-skewed, sqrt for count-like); (2) Use Wilcoxon signed-rank test (distribution-free, tests median difference); (3) Use permutation paired test (resampling-based); (4) Bootstrap confidence intervals for mean difference. NOTE: It's the DIFFERENCES that must be normal, not the original Time1/Time2 scores

wilcoxon signed ranksign test

How to check

Quick

Design review; check for clustering (e.g., pairs nested in schools, clinics, families); look for contagion effects; verify no contamination between subjects

Rigorous

Check intraclass correlation (ICC) if clustering suspected; verify randomization was independent across pairs; test for autocorrelation if temporal ordering matters

If violated

If pairs are clustered (e.g., students nested in classrooms, patients nested in clinics) → use LMM with random intercepts for clusters, treating timepoint as within-subject factor and cluster as random effect. If order/carryover effects within crossover design → use mixed model with period and sequence terms. If contamination/contagion between subjects → redesign study or use network models

linear mixed modelgee

03Residual Forensics

Diagnostics

Checking residual plots and indices to examine model deviations and ensure standard error integrity.

Don't check normality on Group A and Group B. Check normality on (B - A). That is the only distribution that matters.

Difference Forensics

Essential checks

Q-Q plot of difference scores to assess normality
Boxplot of differences to identify outliers
Shapiro-Wilk test on differences (if n < 50)

Recommended checks

Histogram of difference scores
Scatter plot of Time1 vs Time2 (with identity line)
Descriptive statistics of differences (M, SD, n)
Check skewness and kurtosis of differences
Profile plot showing individual trajectories

04Live Instances

Applied Minds

Review concrete study examples, data layout guidelines, and copy executable syntax scripts.

From 'Before & After' to 'Left & Right', the paired design is the scalpel of experimental isolation.

Longitudinal Wisdom

Example 01

Pre-Post Cognitive Behavioral Therapy for Depression (Classic RCT)

Research question: Does 12-week CBT reduce depression symptoms? Design: Pre-post RCT with n=35 participants. Outcome: Beck Depression Inventory-II (BDI-II) score at baseline and post-treatment (continuous, 0-63, higher = more depressed). Classic paired design testing within-subject change.

DesignPre-post within-subjects

Outcome ScaleBDI-II depression score (continuous, 0-63)

# Paired-samples t-test: Pre-Post CBT for Depression
# Based on realistic effect sizes from CBT meta-analyses

# Install/load packages
library(ggplot2)
library(effsize)
library(tidyr)

# Simulate realistic data (or load: data <- read.csv("cbt_depression.csv"))
set.seed(2025)
n <- 35
data <- data.frame(
  subject_id = 1:n,
  pre_BDI = rnorm(n, mean=28.5, sd=8.2),   # Baseline: M=28.5 (moderate-severe depression)
  post_BDI = rnorm(n, mean=16.3, sd=9.1)   # Post-CBT: M=16.3 (mild depression)
)

# IMPORTANT: Add within-subject correlation (r=.65) to make data realistic
corr_matrix <- matrix(c(1, 0.65, 0.65, 1), nrow=2)
library(MASS)
scores <- mvrnorm(n, mu=c(28.5, 16.3), Sigma=cov2cor(corr_matrix) * c(8.2, 9.1) %o% c(8.2, 9.1))
data$pre_BDI <- scores[,1]
data$post_BDI <- scores[,2]

# Compute difference scores (Post - Pre; negative = improvement)
data$difference <- data$post_BDI - data$pre_BDI

# === STEP 1: Check Assumptions ===

# 1. Normality of DIFFERENCES (not original scores)
shapiro.test(data$difference)
# Result: p > .05, normality of differences OK

# Q-Q plot of differences
qqnorm(data$difference, main="Q-Q Plot of Difference Scores(Post - Pre)")
qqline(data$difference)

# Histogram of differences
hist(data$difference, breaks=10, col="steelblue", 
     main="Distribution of Change Scores(Post - Pre BDI-II)",
     xlab="Change in BDI-II(negative = improvement)")
abline(v=0, col="red", lwd=2, lty=2)  # Line at no change

# 2. Outliers in differences
boxplot(data$difference, horizontal=TRUE, 
        main="Boxplot of Difference Scores",
        xlab="Change in BDI-II(Post - Pre)")
abline(v=0, col="red", lwd=2, lty=2)
# Result: No extreme outliers

# 3. Check pairing is correct
head(data)  # Verify each subject has both pre and post
sum(is.na(data$pre_BDI) | is.na(data$post_BDI))  # Should be 0

# === STEP 2: Descriptive Statistics ===
cat("=== Descriptive Statistics ===\n")
cat("Pre-treatment: M =", round(mean(data$pre_BDI), 2), 
    ", SD =", round(sd(data$pre_BDI), 2), "\n")
cat("Post-treatment: M =", round(mean(data$post_BDI), 2), 
    ", SD =", round(sd(data$post_BDI), 2), "\n")
cat("Mean change: M =", round(mean(data$difference), 2), 
    ", SD =", round(sd(data$difference), 2), "\n")
cat("Correlation(pre-post): r =", round(cor(data$pre_BDI, data$post_BDI), 2), "\n")

# === STEP 3: Run Paired-samples t-test ===
t_result <- t.test(data$post_BDI, data$pre_BDI, paired=TRUE)
print(t_result)

# Alternative syntax (equivalent):
# t.test(data$difference, mu=0)

# Output:
# Paired t-test
# t = -7.83, df = 34, p-value < .001
# 95% CI: [-15.3, -9.1]
# Mean difference: -12.2 points

# === STEP 4: Effect Size ===
# Cohen's dz for paired data (uses SD of differences)
cohen_dz <- mean(data$difference) / sd(data$difference)
cat("\nCohen's dz:", round(cohen_dz, 2), "\n")
# dz = -1.32 (very large effect)

# Alternative: Cohen's d using correlation adjustment
library(effsize)
cohen.d(data$post_BDI, data$pre_BDI, paired=TRUE)

# === STEP 5: Visualize Results ===

# Profile plot (spaghetti plot)
data_long <- pivot_longer(data, cols=c(pre_BDI, post_BDI), 
                          names_to="timepoint", values_to="BDI_score")
data_long$timepoint <- factor(data_long$timepoint, 
                              levels=c("pre_BDI", "post_BDI"),
                              labels=c("Baseline", "Post-CBT"))

ggplot(data_long, aes(x=timepoint, y=BDI_score, group=subject_id)) +
  geom_line(alpha=0.3, color="gray50") +
  geom_point(alpha=0.3, color="gray50") +
  stat_summary(aes(group=1), fun=mean, geom="line", 
               color="red", size=1.5) +
  stat_summary(fun=mean, geom="point", 
               color="red", size=4, shape=18) +
  labs(title="Individual Trajectories: Pre-Post CBT for Depression",
       subtitle="Red line = group mean; gray lines = individuals",
       x="Timepoint", y="BDI-II Depression Score(0-63)") +
  theme_classic()

# Paired boxplot
ggplot(data_long, aes(x=timepoint, y=BDI_score, fill=timepoint)) +
  geom_boxplot(alpha=0.6) +
  geom_line(aes(group=subject_id), alpha=0.2) +
  labs(title="Pre-Post Depression Scores(Paired)",
       x="Timepoint", y="BDI-II Depression Score") +
  scale_fill_brewer(palette="Set2") +
  theme_classic() +
  theme(legend.position="none")

# === APA-Style Reporting ===
cat("\n=== APA-Style Report ===\n")
cat("A paired-samples t-test was conducted to evaluate the effectiveness of\n")
cat("12-week CBT for reducing depression symptoms. The difference scores were\n")
cat("approximately normally distributed(Shapiro-Wilk p > .05). There was a\n")
cat("significant reduction in BDI-II scores from baseline(M = 28.5, SD = 8.2)\n")
cat("to post-treatment(M = 16.3, SD = 9.1), t(34) = -7.83, p < .001,\n")
cat("dz = -1.32 (95% CI [-15.3, -9.1]). Participants showed an average\n")
cat("improvement of 12.2 points on the BDI-II, representing a very large effect\n")
cat("and clinically significant reduction in depression symptoms.\n")

Interpretation Blueprint

t(34) = -7.83, p < .001, dz = -1.32 (very large effect), 95% CI [-15.3, -9.1]. Participants showed significant improvement in depression (mean reduction = 12.2 points on BDI-II). The within-subject correlation (r=.65) demonstrates strong pairing, justifying paired t-test over independent t-test. Effect size consistent with Hofmann et al. (2012) meta-analysis showing CBT produces large effects for depression.

05Tactical Pivots

Alternatives

Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.

If the differences skew wildly, abandon the mean. The Wilcoxon Signed-Rank test awaits.

Rank-Based Pivot

Measurement Precision Ladder Ideal · Ratio / Interval Delta

Ratio

Maintain Paired T-Test. Captures the 'Delta' of recovery with maximum within-subject efficiency.

Peak Signal

Interval

Ideal for Pre-Post Scales. Ensure the 'Internal Standard' remains constant for each participant across time.

Standard Precision

Ordinal

Pivot to Wilcoxon Signed-Rank Test to audit the median rank-shift while neutralizing non-normal deltas.

Threshold Bias

Nominal

Abandon T-Test. Use McNemar's Test to model binary 'Flipping' across temporal thresholds.

Identity Only

Temporal Trajectory Audit Longitudinal Paired Snapshot

Paired

Exactly 2 stages.

Stay with Paired T-Test. The most parsimonious model for internal discovery.

Multi-Temporal

3+ timepoints.

Pivot to One-Way Repeated Measures ANOVA to audit the full trajectory of recovery.

Unbalanced

Missing follow-ups.

Pivot to Linear Mixed Models (LMM) to preserve participants with incomplete paired data.

Adaptive Technical Safeguards · adaptive safeguards

difference normality violated

Wilcoxon Signed-Rank Test — The robust rank-based equivalent for non-normal paired distributions.
Bootstrap Delta Strike — Generate significance using resampled difference scores.

extreme delta outliers

Trimmed Paired T-Test — Remove the top and bottom 5% of participants with 'Impossible' recovery scores.
Sign Test — Use purely directional math (Success/Failure) if magnitude is contaminated.

no internal correlation

Independent T-Test — If pre and post are unrelated, the paired advantage is lost—pivot to group comparison.

06Adjusted Comparisons

Post-hoc

Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.

There is no post-hoc here. There are only two moments. The comparison is absolute.

N/A Context

Adjusted Comparisons

Compare with Wilcoxon signed-rank test (nonparametric alternative)
Bootstrap confidence intervals for mean difference
Examine normality of difference scores
Check for carryover effects if applicable
Calculate Cohen's dz effect size: dz = mean_diff / SD_diff

Interpretation Guidelines

Paired t-test compares 2 related conditions. Post-hoc tests are not applicable (use repeated measures ANOVA + post-hoc for 3+ conditions).

07Standardized scale impact

Effect Size

Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.

Standardize by the standard deviation of the difference, not the pooled groups. The effect is within the change, not between the crowds.

Cohen's dz

Standardized mean difference for paired data, using SD of difference scores. Small: 0.2, Medium: 0.5, Large: 0.8. Formula: dz = MD / SDD where MD = mean of differences, SDD = SD of differences. Most appropriate for paired designs

Repeated measures Cohen's d, accounting for correlation between measurements. drm = MD / √(SD₁² + SD₂² - 2r×SD₁×SD₂) where r = correlation. More comparable to independent samples d

Unstandardized mean change (M_post - M_pre) with 95% CI. Easiest to interpret in original units (e.g., '12.2 point reduction on BDI-II')

Recommended Metric: cohens_dz for within-subject effect (most common for paired t-test); mean_difference for interpretability

Small
0.2

Medium
0.5

Large
0.8

0.50

cohens_dz for within-subject effect (most common for paired t-test); mean_difference for interpretability

Recommended Measure

Available Metrics

ReportUse cohens_dz for within-subject effect (most common for paired t-test); mean_difference for interpretability to represent clinical impact magnitude.

08Statistical Power

Sample Size

Guidelines for minimum sample requirements and power analysis parameters.

Paired designs are power-efficient. You need fewer subjects to find the same truth because you have removed the noise of individual differences.

Efficiency Protocol

Floor Requirements

The 'Difference Stability' Minimum: A minimum of 15 pairs is required to ensure the distribution of 'Change Scores' reaches statistical authority.

Effect Size	Parameters	Required n
Small Effect	d=0.20 (Small)	n ≈ 199 pairs
Medium Effect	d=0.50 (Medium)	n ≈ 34 pairs
Large Effect	d=0.80 (Large)	n ≈ 15 pairs

Key considerations

The 'Delta' Audit: Power is driven by the consistency of the change. If some participants recover while others decline, the variance of the differences (SD_diff) will explode, requiring a much larger N.

G*Power StrategyBenchmark: T-tests → Means: Difference between two dependent means (matched pairs). Parameters: Cohen's d_z, α = .05, Power = .80. The 'Paired Advantage': Using each participant as their own control typically requires 50% less N than independent designs.

09APA narrative blueprint

Reporting

How to compile statistical results into publication prose matching APA and journal style guides.

Report the Mean Difference and its Confidence Interval. That range tells the true story of the intervention's reliability.

The Change Narrative

Worked APA paragraph example

A paired-samples t-test was conducted to evaluate the effectiveness of 12-week CBT for reducing depression symptoms. The difference scores were approximately normally distributed (Shapiro-Wilk p = .18). There was a significant reduction in BDI-II scores from baseline (M = 28.5, SD = 8.2) to post-treatment (M = 16.3, SD = 9.1), t(34) = -7.83, p < .001, dz = -1.32 (95% CI [-15.3, -9.1]). Participants showed an average improvement of 12.2 points on the BDI-II, representing a very large effect and clinically significant reduction in depression symptoms.

Reusable template

A paired-samples t-test was conducted to brief description of purpose, e.g., 'evaluate the effectiveness of 12-week CBT for reducing depression'. If assumptions checked, state: 'The difference scores were approximately normally distributed (Shapiro-Wilk p > .05).' If violated: 'Due to non-normal differences (Shapiro-Wilk p < .05), Wilcoxon signed-rank test was used as a sensitivity analysis.' There was a significant/non-significant increase/reduction/change in DV from Time1/Condition1 (M = XX.X, SD = X.X) to Time2/Condition2 (M = XX.X, SD = X.X), t(df) = X.XX, p = .XXX, dz = X.XX (95% CI X.X, X.X). Interpret effect size and practical significance in context.

Essential statistics to report

t-statistic
degrees of freedom (n - 1)
p-value (exact if p > .001, otherwise p < .001)
effect size (Cohen's dz)
95% confidence interval for mean difference
descriptive statistics at both timepoints (M, SD, n)
mean and SD of difference scores
statement about normality of differences

10Exhibit Builder

Manuscript Lab

Copy standard summary tables and forensic reporting grids to outline analysis details.

Table 1: Paired Samples t-test for Cognitive Gains (Pre vs Post)

Comparison	M_diff	SD_diff	t	df	p	dz (Effect Size)
Post - Pre	12.4	4.2	8.54	59	< .001	1.10

Note. N = 60. Results based on difference scores. Correlation between trials r = .65.

dz = 1.10Superior Recovery Consistency. Nearly every participant showed an improvement, resulting in an effect size that dwarfs typical between-group studies.

Correlation (r=.65)Internal Stability. High pre-post correlation proves the measurement tool is stable and that the change is systematic, not random noise.

Header glossary

The Average Gain. The mean increase (or decrease) observed within the subjects.

Cohen's d for Paired Samples. Standardizes the gain relative to the variability of the change itself. dz > 1.0 indicates a massive, consistent shift.

11Algorithmic Logic

Command Center

Syntax libraries and function parameters for executing calculations in stats packages.

Ensure your data is structured wide (two columns) or long (one column with ID). The machine must know which values belong to whom.

Execution Engine

# 1. Execute Paired T-Test
t.test(df$post, df$pre, paired = TRUE)

# 2. Extract Paired Effect Size (Cohen's dz)
lsr::cohensD(df$post, df$pre, method = 'paired')

Library stack

statslsr

Python

pingouin

Elite Forensic Strike

Paired T-tests are actually OLS regressions on the 'Difference Scores'. If the differences aren't normally distributed, your p-value is a lie.

# Execute Normality Audit on Difference Scores
diff <- df$post - df$pre
shapiro.test(diff)

# Visual Check (QQ Plot)
performance::check_normality(lm(diff ~ 1))

12The Over-adjustment Trap

Common Mistakes

Analytical caveats and corrections to maintain modeling integrity.

The biggest sin is treating paired data as independent. That is p-hacking by incompetence. Respect the link.

Independence Failure

Why it's wrong

Independent t-test ignores the within-subject correlation and treats observations as independent. This loses substantial statistical power (paired designs can be 2-3x more powerful when r > 0.5) and fails to control for individual differences. The independent t-test is testing the wrong hypothesis (μ₁ - μ₂ = 0 instead of μD = 0).

The correction

Always review study design before choosing test. If same subjects measured twice (pre-post, crossover), or subjects matched in pairs → use paired t-test. Check if subject IDs repeat across conditions. Use independent t-test only for truly separate, unrelated groups.

Why it's wrong

Paired t-test assumes normality of DIFFERENCE scores (D = Time2 - Time1), not normality of the original Time1 or Time2 scores. The original scores can be non-normal as long as their differences are approximately normal. Testing wrong distribution leads to incorrect assumption conclusions.

The correction

ALWAYS compute difference scores first (D = Time2 - Time1 or Post - Pre). Then test normality of D using Shapiro-Wilk, Q-Q plots, and histograms of the differences. Ignore normality of original scores.

Why it's wrong

Analyzing paired data as independent inflates Type II error (reduces power) because it ignores within-subject correlation. You lose the primary advantage of repeated measures designs: controlling for individual differences. This is especially problematic when correlation is high (r > 0.5).

The correction

Verify pairing structure before analysis. If subjects measured at multiple timepoints, or if matched pairs exist → use paired t-test. Report within-subject correlation to justify paired approach. Never analyze paired data as independent.

Why it's wrong

Standard Cohen's d for independent samples is not appropriate for paired data. Paired designs require Cohen's dz (based on SD of differences) or Cohen's drm (accounting for correlation). Using wrong effect size formula produces misleading values that cannot be compared to meta-analytic benchmarks.

The correction

For paired t-test, report Cohen's dz = MD / SDD (mean difference / SD of differences). This is the standard for within-subject designs. Also report mean difference in original units for interpretability. If comparing to independent samples studies, convert dz to drm.

Why it's wrong

Paired designs are only more powerful when within-subject correlation is positive (r > 0). If r ≈ 0 (no correlation between measurements), paired and independent tests have similar power. If r < 0 (negative correlation, rare), paired design actually loses power compared to independent design.

The correction

Report within-subject correlation when using paired t-test. If r < 0.3, consider whether pairing is beneficial. High correlation (r > 0.5) strongly justifies paired design. Always check correlation between Time1 and Time2.

Why it's wrong

Paired t-test requires complete pairs. If some subjects are missing Time1 or Time2, listwise deletion removes entire pairs, reducing power. Simply analyzing available data as independent violates the pairing assumption.

The correction

If missing data: (1) Use listwise deletion and report how many pairs excluded (reduces power but valid); (2) Use linear mixed model (LMM) which can handle unbalanced data and missing observations; (3) Consider multiple imputation if missingness is MAR. Report missing data handling transparently.

Why it's wrong

In crossover trials (e.g., drug vs placebo), order of conditions can create carryover or learning effects. Simple paired t-test assumes order doesn't matter. If half receive Drug→Placebo and half Placebo→Drug, but there's carryover, paired t-test confounds treatment effect with order effect.

The correction

In crossover designs: (1) Counterbalance order and test for order×treatment interaction using mixed ANOVA or LMM; (2) Include adequate washout period; (3) Test for period and sequence effects; (4) If order effects detected, use analysis that accounts for them (e.g., mixed model with period term).

13Academic Lineage

References

Scholarly lineage and citation keys grounding the statistical framework.

Gosset built this for small samples. It remains the gold standard for efficient comparison.

Student's Legacy

[1]

Hofmann, S. G., Asnaani, A., Vonk, I. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy and Research, 36(5), 427-440.

Meta-analysis showing CBT produces large effects for depression (d ≈ 0.75). Basis for Example 1.

doi: 10.1007/s10608-012-9476-1

[2]

Lieberman, H. R., Tharion, W. J., Shukitt-Hale, B., Speckman, K. L., & Tulley, R. (2002). Effects of caffeine, sleep loss, and stress on cognitive performance and mood during U.S. Navy SEAL training. Psychopharmacology, 164(3), 250-261.

Caffeine improves reaction time with medium-to-large effects. Basis for Example 2.

doi: 10.1007/s00213-002-1217-9

[3]

Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological Science, 29(8), 1358-1369.

Meta-analysis of quasi-experimental studies showing education causally increases IQ (1-5 points per year). Basis for Example 3.

doi: 10.1177/0956797618774253

[4]

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.

Tutorial on effect sizes for paired designs. Explains Cohen's dz vs drm and when to use each.

doi: 10.3389/fpsyg.2013.00863

[5]

Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105-125.

How to convert between dz (paired) and d (independent) effect sizes for meta-analysis.

doi: 10.1037/1082-989X.7.1.105

The best control for a participant is their own past. Use the Paired test to reveal the evolution of the individual through the lens of recovery.

The Interpretive Rigor Directive

statminds · PairedMind reference · v2.2 · updated 2026-01-1715 of 15 sections

Paired T-Test

What is it?

When to use it

Core Idea

Hypotheses

How it works

Assumptions

Effect Size

Quick Example

Paired Slope Line Live Laboratory

Hypotheses

Assumptions

Diagnostics

Applied Minds

Pre-Post Cognitive Behavioral Therapy for Depression (Classic RCT)

Alternatives

Post-hoc

Effect Size

Sample Size

Reporting

Manuscript Lab

Command Center

Common Mistakes

References

Related Tests

more complex

nonparametric equivalent

independent version

one sample version

multiple timepoints