Words can't bring me down

But they can bring down some, or least, so they say

At least, that’s a philosophy of life that some people subscribe to. Others think that words are sort of magic and can hurt you. Even if they are general statements uttered by strangers, not in specifically directed at you. So Sam Pratt and colleagues set out to make a measurement scale for this idea and they came up with these questions:

They then dutifully collected some survey data to see how this scale correlated with various other aspects of humanity. Their results are reported in a number of Pearson correlation matrices, but this doesn’t properly showcase their results. So I downloaded their data and had a closer look. First, the sex difference:

A non-trivial sex difference emerged, as you might expect. This is better than the paper’s report of a Pearson correlation of 0.19. The mental trick is to double the correlated to get the Cohen’s d. This works when 1) the correlation is relatively small, and 2) the groups are balanced. Both apply here so the quick and dirty method works, but it doesn’t work as well for race:

The Black-White difference is 0.56 d, whereas the paper reported a correlation between non-White of 0.10. The issue is that the sample is unbalanced, so the Pearson correlation is forced downwards and ended up hiding a quite substantial age. For age:

There is a weak negative correlation with age (-0.1), and women are higher at every age. The slope appears to be more negative for women, but it’s not beyond chance levels.

The study had a bunch of self-report mental health scales covering the usual stuff, anxiety, depression, emotional regulation, PTSD-like. I collapsed these to a single general psychopathology factor score (P). I included the neuroticism part of the Big Five/OCEAN for this. General factor from bifactor model was strong, omega_h = 0.86 fra CFA, and shows this correlation:

The correlation isn’t massive, r = 0.19, but it isn’t trivial either. Finally, politics:

Or by party choice:

Which replicated the stereotype about the insensitive libertarian, who is about 1 SD lower on this scale than the Democrat voters.

Some of the above may not be that independent, say, age could just be a proxy for leftism or non-White, since younger people are more leftist and less White. Some regressions to clarify:

The age effect was already marginal to begin with, and it disappears entirely when leftism is included in model 3. The mental health variables are approx. orthogonal due to the bifactor model, so the effect of anxiety is particularly strong since there is both an effect of high general mental pathology and an effect of anxiety separately. We can compare the effects more cleanly in another way by looking at the partial epsilon values:

These are akin to parital R2’s giving us independent variances explained by different variables. These are different than the standardized betas because standardized betas can be large and yet explain little variance if they are highly redundant with other variables. I have also taken the square root so they are on a correlation-like scale instead of the nonlinear variance scale. What it shows is that there are 5 variables of approximately equal importance in explaining the variation in belief in magic words in this dataset. These values depend on sample composition, so if the sample had fewer Blacks, the effect size would decline (you can’t explain a large proportion of variance in an outcome with a predictor that has little variance). In line with our work with the Finnish data, anxiety independent of general psychopathology seems to be particularly important for explaining some other variables. Still, the overall model only mustered to explain 20% of variance, so I don’t know what explains the remaining 80%. The scale seemed pretty reliable, so not too much of this can be ascribed to unreliability of the outcome variable.

That’s all I could think of. Thanks to Sam Pratt for making their data public. I hope this post can help their study get more attention. R notebook is here.