Stats

Ordinal Mixed Effect Model with 3 factors and Interactions - Beginner Help

Step-by-step statistics solution: Ordinal Mixed Effect Model with 3 factors and Interactions - Beginner Help

As an Amazon Associate, I earn from qualifying purchases. For more practice problems like this, see Schaum’s Outline of Statistics, 6th Edition.

1. What the student is asking (in plain language)

The student has fitted an ordinal mixed‑effects (cumulative link) model with three categorical predictors (WordCat, Group, Session) and all their two‑ and three‑way interactions. Random intercepts are included for participants and for the stimulus words.

The questions are:

  1. How to report the fixed‑effect results (coefficients, standard errors, significance) in a way that a reader can understand what the model is telling us about the three‑way interaction.
  2. How to visualise the interaction (e.g., predicted probabilities or predicted ordinal scores) in a simple, interpretable plot.

The answer below walks through every step needed to turn the raw summary() output into a clear statistical report and a reproducible plot.

2. Step‑by‑step solution

2.1 Understand the model that was fitted

modelOrdinalWordCat <- clmm(
    Rating_Resp ~ WordCat * Group * Session +
                   (1 | participant) + (1 | Word),
    data = completed,
    link = "logit"
)
Component Meaning
Response Rating_Resp Ordered factor with 5 levels (1 = lowest, 5 = highest).
Fixed effects Intercept + main effects of WordCat, Group, Session + all 2‑way and the 3‑way interaction.
Random intercepts Separate intercepts for each participant and each stimulus word (accounts for repeated measures).
Link Logit link → the model is a proportional‑odds cumulative logit model.
Reference levels (R’s default) WordCat = RealWord, Group = L1 Exposure, Session = Session1. All coefficients are expressed relative to these reference categories.

2.2 Extract the fixed‑effect table

summary(modelOrdinalWordCat)$coefficients
Term Estimate (β) SE z‑value p‑value
WordCat[S.NonWord] -0.3801 0.0414 -9.19 < 0.001
GroupL2 Exposure -0.1740 0.0815 -2.13 0.033
Session2 -0.0537 0.0363 -1.48 0.139
WordCat × Group 0.0478 0.0341 1.40 0.161
WordCat × Session 0.0540 0.0362 1.49 0.136
Group × Session 0.1331 0.0482 2.76 0.006
WordCat × Group × Session -0.1039 0.0482 -2.16 0.031

Note – The model also estimates threshold (cut‑point) parameters for the five ordered categories; they are not shown here because they are not of primary interest for the interaction.

2.3 Translate log‑odds (β) into odds ratios (OR)

For easier interpretation we exponentiate the coefficients:

exp_coef <- exp(coef(modelOrdinalWordCat))
exp_coef
Term Odds Ratio (OR) 95 % CI (approx)
WordCat[S.NonWord] 0.68 (0.64 , 0.73)
GroupL2 Exposure 0.84 (0.72 , 0.98)
Session2 0.95 (0.87 , 1.04)
WordCat × Group 1.05 (0.99 , 1.12)
WordCat × Session 1.06 (0.99 , 1.13)
Group × Session 1.14 (1.04 , 1.25)
WordCat × Group × Session 0.90 (0.82 , 0.99)

How the CI was obtained: confint(modelOrdinalWordCat, method = "Wald") → exponentiate the lower/upper limits.

2.4 Interpret the three‑way interaction

The three‑way interaction term (WordCat:S.NonWord:GroupL2 Exposure:Session2) has:

  • β = –0.1039 (p = 0.031) → statistically significant at α = 0.05.
  • OR = 0.90 (95 % CI 0.82–0.99).

Interpretation (in words):

For a non‑word stimulus presented in Session 2, the effect of being in the L2‑exposure group (vs. L1 exposure) is 10 % smaller (odds multiplied by 0.90) than it would be for a real word or for Session 1. In other words, the advantage (or disadvantage) of L2 exposure that is observed for real words in Session 1 disappears—or even reverses—for non‑words in the second session.

A helpful way to phrase it for a journal article:

“A significant three‑way interaction between Word Category, Group, and Session (β = ‑0.104, SE = 0.048, p = 0.031) indicates that the L2‑exposure advantage observed for real words in the first session is attenuated for non‑words in the second session (OR = 0.90, 95 % CI = 0.82–0.99).”

2.5 Report the full set of fixed effects

A compact table (often placed in the manuscript or supplementary material) can be produced with broom::tidy() or sjPlot::tab_model():

library(sjPlot)
tab_model(
    modelOrdinalWordCat,
    show.ci = TRUE,
    transform = "exp",               # show odds ratios
    show.p = TRUE,
    title = "Ordinal mixed‑effects model (cumulative logit)"
)

The table will contain:

Predictor OR 2.5 % CI 97.5 % CI p
(Intercept) – – – –
WordCat = NonWord 0.68 0.64 0.73 < 0.001
Group = L2 Exposure 0.84 0.72 0.98 0.033
Session = 2 0.95 0.87 1.04 0.139
WordCat × Group 1.05 0.99 1.12 0.161
WordCat × Session 1.06 0.99 1.13 0.136
Group × Session 1.14 1.04 1.25 0.006
WordCat × Group × Session 0.90 0.82 0.99 0.031

If the journal prefers β‑coefficients, report both β and OR side‑by‑side.

2.6 Visualising the interaction

Because the outcome is ordinal, the most interpretable plot shows predicted probabilities for each rating level as a function of the three factors.

2.6.1 Create a new data frame of all combinations

newdat <- expand.grid(
    WordCat = c("RealWord", "NonWord"),
    Group   = c("L1 Exposure", "L2 Exposure"),
    Session = c("Session1", "Session2"),
    participant = NA,   # random effects set to zero (population‑level)
    Word = NA
)

2.6.2 Obtain predicted probabilities

library(ordinal)   # clmm lives here
pred <- predict(modelOrdinalWordCat,
                newdata = newdat,
                type = "prob")   # returns a matrix: rows = combos, cols = rating levels
newdat <- cbind(newdat, pred)

2.6.3 Plot (stacked bar or line plot)

A stacked bar plot is common for ordinal outcomes:

library(ggplot2)
newdat_long <- tidyr::pivot_longer(
    newdat,
    cols = starts_with("1"),   # columns 1–5 are the rating probabilities
    names_to = "Rating",
    values_to = "Prob",
    names_prefix = "Rating"
)

ggplot(newdat_long,
       aes(x = interaction(Group, Session),
           y = Prob,
           fill = Rating)) +
  geom_bar(stat = "identity", position = "stack") +
  facet_wrap(~ WordCat) +
  labs(
    x = "Group × Session",
    y = "Predicted probability",
    fill = "Rating (1‑5)",
    title = "Predicted rating probabilities by Word Category, Group, and Session"
  ) +
  theme_minimal()

What the plot shows

  • Two panels: RealWord vs. NonWord.
  • Within each panel, four bars (L1‑Session1, L1‑Session2, L2‑Session1, L2‑Session2).
  • The colour stack indicates the probability of each rating level.

If the three‑way interaction is present, the shape of the bar for L2‑Session2 will differ between RealWord and NonWord—exactly what the coefficient tells us.

2.6.4 Alternative: line plot of cumulative probabilities

Sometimes it is clearer to plot the cumulative probability of scoring ≥ k (k = 2,…,5). Example for rating ≥ 3:

newdat$cum_ge3 <- rowSums(newdat[, paste0("3"), paste0("4"), paste0("5")])
ggplot(newdat,
       aes(x = interaction(Group, Session),
           y = cum_ge3,
           colour = WordCat,
           group = WordCat)) +
  geom_line(size = 1.2) +
  geom_point(size = 2) +
  labs(
    x = "Group × Session",
    y = "P(Rating ≥ 3)",
    colour = "Word Category",
    title = "Cumulative probability of a rating of 3 or higher"
  ) +
  theme_minimal()

Both visualisations make the interaction intuitive: the gap between L1 and L2 changes across sessions and between word categories.

2.7 Putting the report together

A concise write‑up for a journal could look like:

Model. We fitted a cumulative‑logit mixed‑effects model (clmm) with random intercepts for participants and items. Fixed effects included the three main factors (Word Category, Group, Session) and all two‑ and three‑way interactions.

Results. The three‑way interaction was significant (β = ‑0.104, SE = 0.048, p = 0.031; OR = 0.90, 95 % CI = 0.82–0.99). As illustrated in Figure 1, the L2‑exposure advantage observed for real words in Session 1 (OR ≈ 0.84) diminishes for non‑words in Session 2 (OR ≈ 0.90). All lower‑order terms are reported in Table 1.

Interpretation. For participants in the L2‑exposure group, the odds of giving a higher rating to a non‑word in the second session are about 10 % lower than the odds for a real word in the same session, indicating that exposure effects are sensitive to both stimulus type and testing session.

Figure 1. Predicted rating probabilities for each combination of Word Category, Group, and Session (population‑level predictions, random effects set to zero).

(Insert the stacked‑bar plot or cumulative‑probability line plot as Figure 1.)

3. Final answer (what to submit)

  1. Restated problem – The student needs a clear way to report and visualise the fixed‑effect estimates (especially the three‑way interaction) from a cumulative‑logit mixed model with three categorical predictors.
  2. Solution –
    • Explain the model structure and reference levels.
    • Convert coefficients to odds ratios and give 95 % confidence intervals.
    • Provide a verbatim interpretation of each term, focusing on the three‑way interaction.
    • Show how to produce a publication‑ready table (e.g., with sjPlot::tab_model).
    • Demonstrate how to compute population‑level predicted probabilities and plot them (stacked bar or cumulative

Original question: Ordinal Mixed Effect Model with 3 factors and Interactions - Beginner Help on Cross Validated (Stats Stack Exchange), licensed CC BY-SA.

More Stats Problems