1

The Expression Regulation Scale (ERS): Validation of three emotion dimensions for

expressive norms with close and distant others in private and public contexts

Preprint posted on PsyArxiv on 17 November 2023.

Conal Monaghan1, Yiyun Shou1,2,3, Paige Mewton1, Anika Quayle1, Amy Dawel*1

1
School of Medicine and Psychology, The Australian National University.

2
Lloyd’s Register Foundation Institute for the Public Understanding of Risk, National

University of Singapore, Singapore.

3
Saw Swee Hock School of Public Health, National University of Singapore and

National University Health System

Author Note

This research was funded by an ANU College of Health and Medicine TRANSFORM

Career Development Fellowship to AD. We have no conflicts of interest to disclose. The

dataset is available at <OSF link when published>. *Correspondence concerning this article

should be addressed to Amy Dawel, School of Medicine and Psychology, The Australian

National University, Canberra, ACT 2600, Australia. Email: amy.dawel@anu.edu.au

 2

Abstract

The social norms for expression that guide emotional communication are critical for

successful interpersonal interaction. However, the intricate emotional architecture

underpinning these norms has remained largely unexplored. Our study is the first in-depth

investigation of expressive norms across a wide spectrum of 64 theory-informed emotions.

We measured expressive norms ranging from suppression to over-expression within two

different contexts (public vs. private) and interactant types (close relations vs. distant others).

Using a theory-building subsample (n = 507), we employed ant colony optimization (AOC) and

a suite of factor analytical techniques to distill the emotions into three dimensions:

harmonious, vulnerable, and disharmonious. Subsequent validation in a separate

confirmatory subsample (n = 506) supported this structure in all situations (conditions),

providing evidence these dimensions are robust. Notably, this new Expression Regulation

Scale (ERS) demonstrated scalar invariance across all situations using repeated measures

Confirmatory Factor Analysis. We introduce scoring metrics and norms to aid researchers and

practitioners in their analytical endeavors and highlight potential avenues for future research

aimed at enriching our understanding of expression regulation.

Keywords: Emotion, Display Rules, Ant colony optimization, Cross-context, Factor

Analysis

Public significance statement: A vital ingredient for successful social interaction is

understanding and following the learned “display rules” that govern emotional expression.

This study establishes a new scale for measuring these rules—the Expressive Regulation Scale

(ERS)—which can be used to understand their role in social relationships and wellbeing in

different settings, including at work, home, and school, and in multicultural interactions.
 3

The Expression Regulation Scale (ERS): Validation of three emotion dimensions for

expressive norms with close and distant others in private and public contexts

Successful social interaction requires that people modify or regulate their behavior,

including any outward expressions of emotion, to fit the normative expectations of the

situation at hand. For example, it may be socially inappropriate to express feelings of sadness

by crying in a professional context and doing so may have negative consequences, such as

limiting career advancement. The critical importance of these social norms for expression,

also known as display rules, is highlighted by their prominence across psychological

(Manokara et al., 2023; Matsumoto et al., 2008), anthropological (Lutz & White, 1986), and

economics (Wang et al., 2011) research. Display rules also influence potential for

psychological burnout and other wellbeing outcomes (Gross & John, 2003; Wang et al., 2011).

However, each of these fields has adopted idiosyncratic methods, which makes it difficult to

synthesize the evidence base into a coherent narrative.

One reason for this divergence is that, until recently, the emotional structure of

expressive norms was untested. In Dawel et al. (2023), we investigated a set of 24

theoretically-grounded emotions through factor analysis. This analysis suggested that

expressive norms are organized into three distinct emotional domains, each reflecting the

social purposes these emotions serve: (1) fostering social harmony and affiliation (e.g.,

sympathy, joy); (2) signaling vulnerability and requesting support (e.g., sadness, fear); and (3)

managing social power dynamics, with potential for causing social disharmony (e.g., anger,

contempt). The aim of the present study was to provide a more extensive and rigorous test

of this structure by comprehensively sampling emotion terms, evaluating the equivalence and

stability of expressive norms across four key contexts. We also aimed to produce a robust
 4

scale that can be used to quantitatively measure expressive regulation across different

settings and generate the first set of population norms, in a UK sample.

Comprehensively sampling emotion terms was critical for ensuring that our scale

development captured the full range of emotional domains. This approach also ensured

strong coverage of the full breadth of emotions within each dimension, with an adequate

number of items for each (see Sutherland et al., 2013, for importance of construct coverage).

Our comprehensive approach advances understanding beyond existing scales for measuring

expressive norms. For example, the Display Rules Assessment Inventory (DRAI; Matsumoto

et al., 2005) is constrained to Ekman’s seven basic emotions (Ekman & Cordaro, 2011),

whereas our method allows for the exploration of a broader emotional spectrum. This

enables us to uncover the underlying factor structure of these emotions, a capability not

afforded by the DRAI.

Although Dawel et al. (2023), went some way to addressing this limitation, five or

fewer emotions loaded on the vulnerable and disharmonious dimensions, compared to 11 for

the harmonious dimension. As such, the number of items limited content coverage. To

address this limitation, we sourced the literature for influential lists of emotions from

different literatures, which we then compiled and refined to obtain broad coverage of the

most common emotion terms (Supplementary Material A), ultimately testing expressive

norms for 64 different emotions. Critically, the terms included a range of social (as well as

basic) emotions, such as admiration, embarrassment, guilt, and jealousy.

The potential value of our scale also relies on its ability to provide comparative data

across different social scenarios, further extending the seminal work by Dawel et al. (2023).

In that study, we only measured one context (public) and did not specify the type of

interactant. Prior studies highlight that two of the main factors by which expressive norms
 5

vary are whether the context is public or private and whether the interactant relationship is

close or distant (Manokara et al., 2023; Matsumoto et al., 2008). Specifically, expressive

norms tend to encourage greater control or suppression in public than in private, and with

distant than close others. One possible interpretation of these effects is that expressive norms

are perceived to be stricter in these contexts. The present study addressed this limitation by

testing expressive norms for four scenarios: in private with close others, in private with more

distant others, in public with close others, and in public with more distant others.

Establishing measurement invariance in psychological confirmatory factor analysis

(CFA) is of paramount importance in ensuring the validity and reliability of research findings

(Meuleman et al., 2023; Putnick & Bornstein, 2016; Steenkamp & Baumgartner, 1998).

Measurement invariance refers to the extent to which the measurement properties of a

psychological instrument or scale remain consistent across different groups or time points. In

other words, whether emotional expressive norms are conceptually seen as the same across

each display context. This determination is not trivial, as constructs can have different

meanings across different contexts despite having the same underlying structure (Bornstein,

1995). For example, the conceptualization of harmonious emotions could vary significantly

between different contexts or cultures, due to the subjective nature of what is considered

harmonious.

Building upon the work of Dawel et al. (2023), it is important to consider that the

dimensional structure, and consequently the significance of these dimensions, may be

confined primarily to expressive norms in public settings. This highlights the pivotal role of

measurement invariance in cross-cultural and cross-group research. Without establishing

measurement invariance, there is a heightened risk of drawing inaccurate conclusions

regarding any between-group comparisons, or overgeneralizing findings where they do not

 6

apply. Therefore, ensuring measurement invariance forms a cornerstone of our research. This

step is crucial to guarantee that the final model of expression regulation is invariant, enabling

meaningful comparisons across different contexts.

The present study also addressed a third important limitation of previous research;

that researchers have focused only on control or suppression of expressions (e.g., Gross &

John, 2003), missing the potential for over-expressing or amplifying expressions. For example,

a display rule might require one to amplify a smile to politely greet someone they feel neutral

about seeing. The DRAI addresses this limitation by providing six nominal response options:

“show more than you feel it”; “express it as you feel it” “show less than you feel it”; “show it

but with another expression”; “hide your feelings by showing nothing”; “hide your feelings by

showing something else”. However, only the first of these responses concerns over-

expression. Consequently, the DRAI falls short in measuring the degree of over-expression,

thereby restricting the scope for comparative analysis across diverse emotions or settings.

Often rating scales provide scores that are not directly interpretable in relation to

what they are measuring. For example, the DRAI’s six ordinal response options often load on

a single dimension and are converted to a single set of scaled scores for analysis. The problem

with this approach is that the scalar scores are not intuitively interpretable (e.g., amplify =

1.10, express as felt = 0.94). The present study aimed to address these issues by engineering

a rating scale that produces scores that measure the degree of over-expression, as well as

suppression, and that can be understood intuitively. Specifically, participants rate how much

they think they should express each emotion from Express no emotion/hide my emotion

completely (-100) through expressing it as felt (0) to Express much more than I feel (100).

Negative scores indicate the degree of suppression, positive scores the degree of over-

expression, and scores around 0 indicate little to no expression modification or regulation.

 7

The Current Study

The primary purpose of this study was to identify a model of expressive norms

(“display rules”) and an associated measurement scale that could capture emotions

comprehensively yet parsimoniously and provide meaningful information across contexts.

We also aimed to establish the first set of normative data, for UK residents, to guide future

researchers. The current study expanded previous research in three ways. First, by using a

larger comprehensive range of emotions, we were able to ensure our prior three-dimension

model (Dawel, Ashhurst, et al., 2023) had not missed additional dimensions because of under

sampling emotion space. Our comprehensive approach also allowed us to identify additional

emotions for the vulnerable and disharmonious dimensions, which was captured by five or

fewer emotions in Dawel et al. (2023). Second, we included emotional over-expression versus

suppression, which enabled us to capture the bi-directional nature of expression modification

and produce scores that can be intuitively understood (positive scores = over-expression;

negative scores = suppression). Third, we used contemporary psychometric approaches to

identify a robust factor architecture that could generate meaningful cross-situational

information through a CFA invariance framework (Steenkamp & Baumgartner, 1998). To

achieve this, we collected data from a large sample of UK residents (N = 1,013). Initially, we

randomly split the sample in half to perform exploratory and confirmatory analyses. Once the

model was confirmed to be equivalent in both samples, we rejoined them to produce the

normative data. We also asked participants to complete an additional measure, the Emotion

Regulation Questionnaire (ERQ; (Gross & John, 2003), which asks people to self-report how

much they habitually use expressive suppression. Prior work has found small to moderate

correlations between the ERQ-suppression scale and expressive norms (e.g., for the DRAI;

(Matsumoto et al., 2005), and thus validate our new scale by testing for similar associations.
 8

Method

Transparency and Ethics

We report how we determined our sample size, and data exclusions, manipulations,

and measures in the study. We follow Journal Article Reporting Standards (Kazak, 2018). Data

were analyzed using JASP (JASP Team, 2023) and R (R Core Team, 2021).

Our sample size was determined a priori based on the requirements of factor analysis.

For Exploratory Factor Analysis (EFA), recommended sample sizes vary widely, but Monte

Carlo simulations suggest that stable parameter estimates for most underlying structures can

be estimated with 400 to 500 participants (Kyriazos, 2018). To ensure that any identified

structures were not measurement artifacts, we randomly divided the data into theory

building (exploratory) and theory (confirmatory) subsamples. We therefore aimed to collect

data from 1100 participants to obtain a minimum of 500 per subsample, having estimated

that approximately 10 percent of participants’ data would be excluded (for failing attention

check or >10% missing data on at least one of the expressive norms tasks).

For CFA, we identified the sample size to discriminate between RMSEA estimates

(effect size) of .08 (H1) and .05 (H0), with an alpha = .05 and a power of approximately 1. Our

sample size estimates of 500 met this requirement for both a final model of 10 items across

each of the final three factors (df = 402) and a reduced design of 8 emotions per dimension

(df = 249; Jak et al., 2021; MacCallum et al., 1996).

Participants

The sample was recruited to be representative of UK residents based on Census data

for age, sex, and ethnicity, using the Prolific crowdsourcing platform

(https://www.prolific.co/). Inclusion criteria also required participants to be born and raised

in the UK and speak English as their first language or fluently. Data from an additional 91
 9

participants were excluded because they failed more than one of the six attention checks (83

participants) or had >10% missing data on at least one of the display rule situations 7

participants) or both (1 participant). Participants were compensated £4.00 for the ~30-minute

survey. The study was approved by The Australian National University Human Research Ethics

Committee under protocol 2019/970.

Table 1 describes the total analyzed sample and the Exploratory and Confirmatory

subsamples, selected by random assignment. Table 1 also shows the subsamples were well-

matched on key demographic variables, with no significant differences.

Table 1

Sample Demographics

Total sample Exploratory Confirmatory t or χ2 p

N 1013 507 506
Men N (%) a 497 (49.1%) 247 (48.7%) 250 (49.4%) .018 .893
Women N (%) 510 (50.3%) 256 (50.5%) 254 (50.2%) .007 .924
Age M (SD) 45.7 (15.8) 45.6 (15.9) 45.8 (15.7) .277 .782
Age range 18-82 18-81 18-82
White N (%) 896 (88.5%) 441 (87.0%) 455 (89.9%) .219 .640
English as first language N (%) 930 (91.8%) 459 (90.5%) 471 (93.1%) .155 .694
a
Note. The response options for gender were: male, female, prefer a different term, or prefer
not to say. Men indicates participants who identified as male. Women indicates participants
who identified as female.

Study Design

Participants completed the following measures in order: expressive norms tasks (four

situations [conditions], completed in counterbalanced order: in private with people close to

you; in private with people not so close to you; in public with people close to you; in public

with people not so close to you); demographics (age, gender, ethnicity, country of residence,

number of years lived outside the UK before 18 years of age, English language fluency,
 10

ethnicity, household income); Emotional Regulation Questionnaire (ERQ; (Gross & John,

2003).

Expressive Norm Rating Task

Participants rated how they thought they should express 64 emotions in four

scenarios: in private with people very close to them; in private with people not so close to

them; in public with people very close to them; and in public with people not so close to them

(2 contexts x 2 interactants = 4 situations). Participants were instructed: “The next few tasks

ask how you think you should express emotions with people who are very close to you (for

example, family and close friends) and people not so close to you (for example, colleagues

and acquaintances) in private (for example, in either your own home or someone else's) and

in public (for example, in a restaurant or park).” At the start of each situation, participants

were given additional instructions specific to that situation. For example, for the private with

people very close to them situation participants were instructed: “For this task, we would like

to know how you think you should express emotions in private (for example, in either your

own home or someone else's) with people very close to you (for example, family and close

friends). Please take a moment, while the below clock counts down, to imagine interacting in

private with people very close to you.” These instructions remained onscreen for a minimum

of 15 seconds to encourage participants to imagine the context and interactant. Participants

then rated “How should you express each of the following emotions in <private/public> with

<people who are very close to you/people not so close to you>?” for a list of 64 emotions

using a visual analogue scale from -100 to 100, anchored by five qualitative labels: Express no

emotion/hide my emotion completely (-100); Express less than I feel (-50); Express it as I feel

it (0); Express more than I feel (50); and Express much more than I feel (100). The four
 11

situations were presented in counterbalanced order across participants and the order of the

64 emotions was randomized within each situation for each participant.

Selection of Emotions. We aimed to investigate expressive norms for a

comprehensive range of emotion terms to extend on our previous work using 24 emotions

from Keltner and colleagues (Keltner et al., 2019; see Dawel et al., 2022). To select the

emotions, we first compiled 22 influential emotion lists from the psychology and linguistics

literature, including lists from cross-cultural research (Bänziger et al., 2012; Barrett et al.,

2007; Bednarek, 2008; Church et al., 1998; Cowen & Keltner, 2020; Ekman & Cordaro, 2011;

Fontaine et al., 2007; Fredrickson, 2013; Izard, 1977; Jackson et al., 2019; Keltner et al., 2019;

Lee, 2010; Masuda et al., 2012; McNair et al., 1971; Russell, 1983; Scherer, 2005; Shaver et

al., 1987; Shiota et al., 2014; Van Goozen & Frijda, 1993; Wierzbicka, 1992). We refined the

compiled list by converting all terms to the tense required for our expressive norms tasks

(e.g., converting “angry” or “angered” to “anger”), retaining only those terms that appeared

in at least two original lists; and removing broad terms (e.g., “good” or “bad”) and terms

relating to physical states (e.g., “hungry”). This process produced a master list of 174

emotions, which we further refined to 64 by retaining only those terms that appeared in at

least four of the original lists and opting for the more common of any synonyms (e.g.,

preferring “fear” over “fright”). Supplementary Material A reports the full compilation of lists

and final 64 emotions, with frequency (Norvig, 2009; Ogren & Sandhofer, 2021; Shaver et al.,

1987; Van Goozen & Frijda, 1993), age of acquisition (Baron-Cohen et al., 2010), cross-cultural

use (Dewaele & Pavlenko, 2002; Hupka et al., 1999; Kaneko et al., 2006), and valence and

arousal rating data (Grühn, 2016). Critically, the final list of 64 emotions includes the full

range of valence and arousal covered by the compiled list of 174 emotions, indicating that

coverage was comprehensive.

 12

External Validity Measure

Emotion Regulation Questionnaire (ERQ; (Gross & John, 2003). We used the four

items from the ERQ that measure habitual use of expressive suppression: “I keep my emotions

to myself”, “When I am feeling positive emotions, I am careful not to express them”, “I control

my emotions by not expressing them”, and “When I am feeling negative emotions, I make sure

not to express them”. Note that one item refers specifically to positive emotions and one to

negative emotions. Participants responded using a 7-point Likert scale from Strongly Disagree

(1) to Strongly Agree (7). Item scores were summed to produce a total score for expressive

suppression, with higher scores indicating greater use of expressive suppression. Estimates of

internal consistency were strong and equivalent in both subsamples (Cronbach’s a = .80).

Analytic Strategy

Factor identification. All model identification and modification occurred solely in the

theory generation (exploratory) subsample. We aimed to identify a strong factor structure

that was stable and invariant across contexts, facilitating meaningful inferences to be drawn

about their comparison. First, EFA was used to identify substantive factor structures in the

data. We used a combination of the Elbow Test (Cattell, 1966) and Parallel Analysis (2000

replications, quantile = .05; (Horn, 1965) to determine viable factor structures, given they

appear relatively robust to over-dimensionalization (van der Eijk & Rose, 2015) and can be

supplemented with non-graphical interpretations such as the acceleration factor (the second

derivative of the curve) and optimal coordinates (comparison of sample and factor analysis

eigenvalues) (Raîche et al., 2013). Factor structures were estimated using the psych (Revelle,

2023) and nFactors (Raiche & Magis, 2022) packages for R.

We then extracted possible factor structures and evaluated their quality using

patterns of factor loadings and communalities, corrected item-total statistics, and inter-factor
 13

correlations. Given that expressive norms were expected to covary, we utilized oblique

rotation methods and determined final factors by integrating viable factor solutions with

theoretical models (e.g., Dawel, Ashhurst, et al., 2023).

Emotion Reduction. To increase the parsimony of the final scale, we reduced the 64

emotions to the strongest archetypes of each factor, selecting 6-10 items based on their

performance, with a strong emphasis on maintaining the diversity of emotions within each

dimension. Given the large number of possible permutations, ranging from 5.32e13 to 2.98e15,

manually testing factor models was infeasible as it would even overwhelm computerized

brute force methodology. Therefore, we employed ant colony optimization algorithms (ACO;

Deneubourg et al., 1983; Dorigo et al., 1991; Dorigo & Stützle, 2010) to efficiently resolve

complex parameter spaces and identify psychometrically robust item subsets (e.g., Kerber et

al., 2022; Olaru & Danner, 2021; Schultze & Eid, 2018). ACO provides a stochastic protocol

that imitates the pheromone trail laying behavior of particular ant species. To identify the

optimal display rule measurement models, computerized ants start by searching random

paths for food (item sets). Routes that identify more attractive food sources (better model fit

estimates) lay pheromones that increase the likelihood that future ants will follow the same

path (select the items in that set). To reduce the probability of converging on poorer solutions,

pheromones evaporate over time, allowing better-performing items to be selected with

increasing likelihood until an optimum solution is produced.

Using ACO to identify a measurement model that was invariant across all four

contexts had the additional benefit of not being biased by a particular context. That is,

manually findings solutions would generally involve finding strong solutions in a particular

context, then seeing if this solution was invariant across the other contexts. In this way, ACO

can outperform alternative item reduction techniques (Olaru et al., 2015). To ensure content
 14

breadth (Dawel, Ashhurst, et al., 2023; Keltner et al., 2019) and avoid over-fitting in the final

solution, we used a theory-driven heuristic to guide the ant colony optimization algorithm for

selecting the best confirmatory factor analysis model based on 17 emotions. The heuristic

was weak, meaning that it had a low influence on the ants’ decisions and allowed more

exploration of the search space while being directed towards theory-driven emotions. (See

Supplementary Material A for more details.)

As we placed a strong emphasis measurement invariance, ACO was specified to

optimize solutions that were scalar invariant across all four contexts within the repeated

measures confirmatory factor analysis framework. That is, identifying items that were scalar

invariant on their respective factors. Establishing measurement invariance in CFA involves a

sequence of steps, each introducing more stringent constraints on the structure across

different contexts. It's imperative to establish each step fully before advancing to the next (an

exception here is partial invariance which was not a focus of this study). First, configural

invariance demands the same pattern of loadings, the significant and non-significant loadings

of the items on their allocated factor, across all contexts. Next, metric invariance constrains

the strength of item loadings of those factors between contexts. If loadings are equivalent, it

suggests that the emotions represent their allocated latent factors similarly between each

context. However, systematic biases in emotion endorsement between contexts can still skew

mean comparisons at the metric invariance level. To account for this issue, the scalar

invariance test constrains intercepts be equal across situations, thus identifying whether

systematic biases might obscure true group differences. For our study, achieving scalar

invariance was paramount as it allows for meaningful cross-context comparisons and

legitimizes the comparison of latent means. If any of the target parameters are equivalent
 15

between contexts, then constraining them to equivalence should not worsen model fit

substantially.

We used best practices to guide measurement invariance analyses (e.g., Putnick &

Bornstein, 2016), and model fit indices for all CFA models were: Comparative Fit Index (CFI)

and Non-Normed Fit Index (NNFI) > .950, Root Mean Square Error of Approximation (RMSEA)

and Standardized Root Mean Square Residual (SRMR) < .080 and < .060 respectively. Despite

inconsistencies in recommendations for how small changes in fit estimates need to be for

practical invariance between steps, we primarily focused on the most broadly used threshold

of ΔCFI < .01 (Chen, 2007; Cheung & Rensvold, 2002; Kim et al., 2017), while also interpreting

all tests in respect to more conservative recommendations, ΔCFI < .002 (Meade et al., 2008).

All ant colony algorithms were run using the Stuart Algorithmic Rummaging Techniques

Package (Schultze, 2023) in the R Statistical Software (R Core Team, 2023), implemented using

RStudio/Posit (Posit team, 2023).

Model Refinement and Replication: We then evaluated the final models for sources

of model strain and misfit (Brown, 2015; Sellbom & Tellegen, 2019; Waldman et al., 2023).

This involved the magnitude consistency factor loadings, modification indicates,

differentiation between factors, and associations between factors and external relevant

variables. We then estimated model fit and the repeated measures CFA in the confirmatory

dataset without modification using lavaan (Rosseel, 2012) and semTools packages (Jorgensen

et al., 2008). Fit estimates were used as a strict test of our model’s robustness.

Results

Model Development

Exploratory Factor Analysis. Data exceeded guidelines for EFA appropriateness, at

7.91(cases):1(item), Kaiser-Meyer-Olkin estimates ranging from .96 to .97, and all Bartlett’s
 16

c2 values significant, p < .001. Factor identification based on Scree and Parallel analysis

(Supplementary Material B) suggested two strong factors explaining approximately 50-60%

of the variance in the data depending on context. Scree test also suggested the possibility of

a third weaker factor explaining an additional 5% variance depending on context. Parallel

analysis indicated the possibility of up to four factors. Supporting these results, the

acceleration factor (the second derivative of the curve) suggested two factors, whereas the

optimal coordinates (comparison of sample and factor analysis eigenvalues) suggested up to

four. We extracted two to four factor solutions for further evaluation.

We rejected the four-factor solution as spurious because the fourth factor comprised

only two to four emotions (loading < .32), depending on the context, and contributed only an

additional 2% of variance to each factor. Given the limited number of emotions in this factor,

it was deemed an over-depersonalization due to the number of initial emotions (van der Eijk

& Rose, 2015). The two- and three-factor solutions had strong divisions of item loadings

across all candidate factors in all contexts, with a large proportion of these loadings exceeding

.50. Pattern of factor loadings were relatively similar across all contexts, indicating the

likelihood of an underlying set of invariant emotions. Inter-factor correlations were

consistently higher in the Private-close situation (see Supplementary Material C for EFA

results). The factor correlations in the 2-factor solution ranged from .23 to -.07. In the three-

factor solution, the correlations between Factor 3 and the other two factors remained

insignificant to weak (-.08 to .23), the correlations between Factors 1 and 2 were very strong.

(.68 to .73).

Item Reduction

We reduced the structure using ACO for the three-factor solution, using theory driven

heuristics to guide the start of the model estimation. We ran one three-factor ACO, specifying
 17

each emotion from the EFA into their constituent factor, and specified the algorithm to

identify 30 emotions (10 per subscale) that minimized RMSEA, CFI, and ΔCFI (Supplementary

Material A). Additional sources of model strain were then evaluated to reduce the likelihood

of overfitting or poor performing items (Brown, 2015; Sellbom & Tellegen, 2019). All emotions

correlated strongly and consistently with their respective factor (> .70), however,

modification indices indicated strongly correlated error covariances between several pairs of

items. The content of these emotion pairs were variations in intensity of the same underlying

emotions, for example, stress and distress. Given the focus of identifying a parsimonious

model of expressive norms, we removed a further two items per dimension because they had

correlated residuals with stronger items.

Final Emotion Domain Models

We then investigated the final models comprised of three sets of eight emotions in

both the exploratory and confirmatory subsamples. Estimates of internal consistency were

strong and consistent across the samples, ranging from .91 to .95 across contexts (see

Supplementary Material D). Confirmatory Factor Analyses via Robust Maximum Likelihood

estimator were then estimated in each context separately. First, the three-factor model had

acceptable estimates of model fit across all contexts (Table 2), these were replicated in the

confirmatory sample suggesting the model is robust and replicable. Model fit estimates for

the two-factor model were consistently lower than the three-factor model, in the acceptable

(approximately .90) yet not strong (approximately .95) range. This is not surprising given the

ACO and was focused on maximizing fit for the three-factor not two-factor model. However,

these results still suggest that the vulnerable and disharmonious factors can be collapsed into

a single factor. The final factors closely resembled those identified by Dawel et al. (2023) –
 18

harmonious, disharmonious, and vulnerable – and were given these same labels. For the two-

factor model, the combined disharmonious and vulnerable factor was named “Negative”.

To ensure a comprehensive analysis, we also tested a higher-order model in which

vulnerable and disharmonious emotions were hypothesized to load onto a higher-order

factor. However, this model encountered several Heywood cases, indicated by negative latent

variable variance estimates across various variables. Due to these estimation issues (resulting

in impossible values), the higher-order model was deemed to be poorly specified.

Consequently, we decided to retain the single-level three-factor model as the more robust

and viable solution.

Table 2

Fit Estimates from Model Fit from Confirmatory Factor Analytic Models

Model Three-factor Two-factor

Context c2 df p RMSEA SRMR CFI NNFI c2 c2 p RMSEA SRMR CFI NNFI
Exploratory
Private-close 866.71 249 <.001 .058 .053 .956 .952 1246.73 251 <.001 .079 .063 .917 .909
Private-distant 590.15 249 <.001 .040 .058 .975 .972 921.71 251 <.001 .064 .063 .933 .927
Public-close 752.01 249 <.001 .054 .046 .959 .954 1066.24 251 <.001 .073 .052 .924 .916
Public-distant 732.63 249 <.001 .054 .057 .956 .951 1124.5 251 <.001 .077 .064 .909 .900
Confirmatory
Private-close 841.77 249 <.001 .055 .056 .960 .955 1218.04 251 <.001 .077 .071 .920 .912
Private-distant 692.66 249 <.001 .049 .054 .960 .956 949.23 251 <.001 .066 .059 .926 .919
Public-close 732.21 249 <.001 .053 .047 .956 .951 1046.49 251 <.001 .073 .057 .918 .909
Public-distant 720.50 249 <.001 .052 .047 .955 .951 955.74 251 <.001 .067 .054 .925 .917
Note. Estimator = MLR.

The results from the repeated measures CFA for the proposed three-factor structure

are presented in Table 3. In both the exploratory and confirmatory subsamples, the fit

estimates for configural, metric, and scalar models were within acceptable ranges. Although

the ΔCFI exceeded the more stringent criteria (ΔCFI < .002), it complied with the commonly

accepted threshold (ΔCFI < .01). This provides strong evidence that the mean comparisons

across various contexts were unbiased, both in terms of the interpretation of each dimension

(metric invariance) and their thresholds (scalar invariance).

 19

Table 3

Fit Estimates from Repeated Measures Invariance Modelling for Two and Three Factor

Emotion Domain Models

Dataset Three-factor Two-factor

Configural c2 df RMSEA CFI ΔCFI c2 df RMSEA CFI ΔCFI
Exploratory
Configural 6544.54 4254 .037 .928 7514.005 4292 .044 .899
Metric 6627.36 4317 .037 .928 .000 7594.806 4358 .043 .898 .000
Scalar 6830.51 4380 .038 .938 .004 7822.460 4424 .044 .884 .003
Confirmatory
Configural 6704.74 4254 .038 .919 7520.418 4292 .043 .893
Metric 6798.34 4317 .038 .918 .001 7624.013 4358 .043 .892 .001
Scalar 7005.87 4380 .039 .913 .005 7879.323 4424 .044 .886 .006
Note. Estimator = MLR.

For the two-factor structure, the invariance levels mirrored those of the three-factor

model. However, its fit estimates were marginally weaker, ranging between .88 and .90,

which was just below our .90 threshold. This outcome was anticipated, given that the ACO

had tailored the solution for a three-factor model. The ΔCFI for the three-factor model

structure was comparable to that of the two-factor model, with the scalar invariance

estimates falling between the more conservative and more widely utilized criteria.

Expressive Norms Across Contexts

To provide a more complete picture of descriptive statistics and validity estimates, we

used the combined dataset (exploratory and confirmatory). One-way repeated measures

ANOVAs suggested that, within each context, there were significant differences in expressive

norms between each emotion display rule dimension (see Table 4; Figure 1). The effect size

was moderate and the Bayes Factor for the same analysis suggested that the data were

substantially more likely (> 600 times) under the alternative hypothesis as compared to the
 20

null hypothesis, thus providing strong support for an effect. Post-hoc t-tests also suggested

that all pairs of display rule dimensions differed significantly.

Figure 1

Distribution (Violin) plots and Repeated Measures ANOVA of Emotion Display Across

Dimensions, Grouped by Context

Repeated Measures ANOVA of Emotion Display across Dimensions, Grouped by Context
i ii
Close private Close public
F Fisher(1.42, 1436.18) = 1018.41, p = 4.91e-219, w2p = 0.29, CI95% [0.21, 1.00], n pairs = 1,011 F Fisher(1.35, 1367.75) = 1900.37, p = 4.92e-316, w2p = 0.47, CI95% [0.45, 1.00], n pairs = 1,011

150 p Holm-adj. = 1.81e-44 150 p Holm-adj. = 5.21e-48

p Holm-adj. = 3.80e-150 p Holm-adj. = 1.29e-230
p Holm-adj. = 8.01e-180 p Holm-adj. = 8.03e-263
Suppression (-100) to Amplification (+100)

Suppression (-100) to Amplification (+100)

100 100

Pairwise test: Student's t, Bars shown: significant

Pairwise test: Student's t, Bars shown: significant

50 50

mmean = 24.42
mmean = 19.63

0 0
mmean = -10.38
mmean = -19.78
mmean = -32.36
mmean = -41.59
-50 -50

-100 -100

Harmonious Vulnerable Disharmonious Harmonious Vulnerable Disharmonious

(n = 1,011) (n = 1,011) (n = 1,011) (n = 1,011) (n = 1,011) (n = 1,011)
Dimension Dimension

loge(BF01) = -697.17, R 2 Bayesian = 0.57, CI95%

posterior HDI JZS
[0.55, 0.59], r Cauchy = 0.71

iii iv
Distant private Distant public
F Fisher(1.41, 1424.77) = 2449.05, p = 0.00, w2p = 0.51, CI95% [0.49, 1.00], n pairs = 1,011 F Fisher(1.39, 1402.51) = 2557.60, p = 0.00, w2p = 0.52, CI95% [0.50, 1.00], n pairs = 1,010

p Holm-adj. = 2.84e-35 p Holm-adj. = 1.47e-23

p Holm-adj. = 1.51e-275 p Holm-adj. = 1.02e-290
p Holm-adj. = 1.49e-303 p Holm-adj. = 8.55e-303
100
Suppression (-100) to Amplification (+100)

Suppression (-100) to Amplification (+100)

100
Pairwise test: Student's t, Bars shown: significant

Pairwise test: Student's t, Bars shown: significant

50 50

mmean = 13.20 mmean = 9.59

0 0

mmean = -43.50
-50 mmean = -51.24 -50 mmean = -49.60
mmean = -55.67

-100 -100

Harmonious Vulnerable Disharmonious Harmonious Vulnerable Disharmonious

(n = 1,011) (n = 1,011) (n = 1,011) (n = 1,010) (n = 1,010) (n = 1,010)
Dimension Dimension
 21

Table 4

Mean Differences in Expressive Norms between Contexts for the Combined Dataset

Dimension Harmonious Vulnerable Disharmonious Negative

Context Median (IQR) M (SD) Median (IQR) M (SD) Median (IQR) M (SD) Median (IQR) M (SD)
Total 10 (0, 31) 17 (27) -32 (-57, -10) -34 (33) -42 (-68, -18) -42 (33) -37 (-61, -15) -38 (32)
Private-close 14 (0, 39) 24 (29) -8 (-24, 0) -10 (30) -16 (-38, -1) -20 (32) -12 (-30, 0) -15 (29)
Private-distant 11 (0, 36) 20 (26) -30 (-52, -12) -32 (29) -41 (-65, -20) -42 (30) -36 (-56, -17) -37 (28)
Public-close 9 (0, 27) 13 (24) -43 (-66, -24) -43 (29) -53 (-75, -30) -51 (30) -48 (-69, -27) -47 (28)
Public-distant 7 (-2, 23) 10 (26) -49 (-72, -29) -50 (28) -58 (-81, -34) -56 (30) -53 (-75, -32) -53 (28)
p-value <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001
Note. IQR = inter-quartile Range. Median differences were estimated using Kruskal-Wallis
rank sum tests; mean differences were estimated using repeated-measures ANOVA. See
Supplementary Material E for breakdown of data into exploratory and confirmatory
subsamples.

Across all contexts, harmonious emotions were consistently endorsed as being over-

expressed, with a higher central mass (greater kurtosis) than the other dimensions, and a

positive skew towards over-expression. Interestingly, the level of expression for harmonious

emotions was consistent, only ranging from 24.44 (close private) to 9.56 (distant public), and

few participants endorsed any level of suppression.

In contrast, both vulnerable and disharmonious emotions were more dispersed (less

kurtotic) and suppression of them was generally endorsed across all contexts. Their

distribution was opposite to that of harmonious emotions: there was more responses in the

suppression (negative) direction of the mean, and a steep decrease in endorsement of over-

expression above 0. This trend was pronounced in all contexts except for close-private, which

appeared more symmetric. The endorsement of suppression tended to be greater for

disharmonious than vulnerable emotions across all scenarios, though this difference was

relatively minor (approximately 10 units in all scenarios). The most distinct contrast between

vulnerable and disharmonious emotions emerged in the distant-public scenario. Here,

disharmonious emotions were positively skewed, with a mode nearing -100, suggesting a
 22

strong preference for suppression. Conversely, vulnerable emotions displayed a mode

between -30 and -40, with a flatter distribution below 0, implying more varied beliefs about

how these emotions should be expressed in distant-public scenarios.

External Associations

We then evaluated the relationship between the ERS dimensions and ERQ-

suppression (Table 5). Correlations with harmonious emotions were negative but weak across

all scenarios. These associations were stronger for vulnerable and disharmonious emotions,

and stronger for private than public interactants. There was a weak negative correlation

(Spearman) between expression regulation norms and age, which was largely restricted to

the private-close scenario, such that older people reported emotions should be suppressed

more.

Table 5

Associations Between ERS, ERQ Suppression Subscale, and Age

Context Harmonious Vulnerable Disharmonious Negative

ERQ-suppression
Private-close -.06 -.20*** -.16*** -.19***
Private-distant -.08* -.19*** -.13*** -.17***
Public-close -.09** -.12*** -.05 -.09**
Public-distant -.08** -.14*** -.07* -.11***
Age
Private-close -.08* -.10** -.14*** -.12***
Private-distant -.03 -.02 -.05 -.04
Public-close -.08* .02 -.02 .00
Public-distant -.07* .03 .02 .02
Note. * p < .05. ** p < .01. *** p < .001. Negative refers to the combined Vulnerable and
Disharmonious ERS dimensions. See Supplementary Material E for breakdown of data into
exploratory and confirmatory subsamples.
 23

Discussion

The current study investigated the underlying emotional structure of the norms that

guide expression regulation—otherwise known as display rules—in a large sample of the UK

general public, matched to census data for age, sex, and ethnicity. Given the inherent

interpersonal and contextual nature of expressive norms, we measured beliefs about how

much an emotion should be expressed from suppression to over-expression across four

scenarios combining privacy and interactant closeness. We found that, across all contexts,

expressive norms can be conceptualized as organizing into three underlying emotion

domains: harmonious, vulnerable, and disharmonious. This finding was used to establish the

24-item Expression Regulation Scale (ERS; Supplementary Material G), with eight items

capturing each of the three dimensions.

We followed contemporary approaches to robust psychometrics. This included

randomly dividing the sample into theory building (exploratory) and theory testing

(confirmatory) halves, using ACO to reduce a large number of emotion candidates into

dimensions while maintaining content breadth and item diversity, and establishing scalar

invariance across scenarios in both subsamples. We demonstrated the strength of the ACO in

identifying a stable factor structure across scenarios —a feat that would be very difficult to

achieve with more traditional approaches, such as systematically removing constraints based

on a partial-invariance framework. The ACO approach has been shown to achieve better

results than manual approaches to item reduction, given these rely on research “degrees-of-

freedom” and are unlikely to determine the optimal solution (Olaru et al., 2015). As a whole,

we have identified a robust architecture and measurement instrument for understanding

expressive norms.
 24

Although we endorse a three-factor structure, a two-factor solution was also viable.

This reflected the observed similarity in expressive norms between disharmonious and

vulnerable emotions, and the relatively strong correlation between these dimensions.

Although disharmonious and vulnerable expressive norms encapsulate unique forms of

emotional communication with separate societal implications, they both can be categorized

as negative valanced emotions. Despite the commonality of these dimensions, conflating

them would obscure their key differences which are clearest in public contexts or with distant

interactants. Our findings indicated stronger endorsement of suppression for disharmonious

compared to vulnerable emotions.

These observations align with prior research that has underscored strong parallels

between these two dimensions, yet also highlighted their divergence in specific situations.

For instance, Dawel et al., (2023) discovered that although expressive norms for these two

dimensions were similar in both online and face-to-face interactions with friends and co-

workers, notable distinctions emerged when communicating with a supervisor, doctor, or

psychologist, with this differentiation being more pronounced among British participants

than Australians. There are also strong theoretical reasons to distinguish between vulnerable

and disharmonious emotions. For example, an influential hypothesis concerning gender

differences in emotion argues that norms allow for women to express vulnerability more than

men, but for men to express dominant or disharmonious emotions more. A clear benefit of

the three-factor structure is that it enables such hypotheses to be tested. Overall, we

recommend that the scale be analyzed at the three-factor level, although researchers may

opt for the two-factor structure where there are either strong theoretical reasons to or no

empirical evidence of vulnerable-disharmonious differences in their dataset.

 25

Both the three- and two-factor solutions were scalar-level (and subsequent levels)

invariant across each of the four situations. This robust invariance across scenarios suggests

that the underlying factor structure of the emotional dimensions remains consistent

irrespective of the varying relational (interactant) and social setting (context). There are

several important implications of these findings. First, they provide a solid foundation for

making meaningful comparisons of latent means across these contexts. Previous work that

compared interactant and contexts (e.g., Manokara et al., 2023; Matsumoto et al., 2008) did

not establish invariance, and therefore, it was unclear whether differences across scenarios

were due to measurement bias or substantive effects. Second, the invariance implies that the

cognitive appraisal or internal representations of these emotional dimensions might be

universally consistent, regardless of external situational demands. Although individual

emotional displays may manifest at different intensities based on specific contexts and the

depth of interpersonal trust and familiarity, a baseline pattern of expressive norms was

evident for three foundational emotion dimensions.

Generally speaking, expression norms for harmonious emotions ranged from express-

as-is to over-expression, whereas norms for vulnerable and disharmonious emotions tended

towards suppression across all contexts, with very few participants believing these emotions

should be over-expressed. The most notable difference between vulnerable disharmonious

emotions occurred in public with distant interactants, where the tendency for suppression

was notably stronger for disharmonious than vulnerable emotions.

The patterns across contexts were similarly striking. As previous research has found

(Manokara et al., 2023; Matsumoto et al., 2008), there was a tendency towards greater

expression in private than in public and with close than distant others. However, it is not the

case that people express all emotions most authentically in private settings with people close
 26

to them, as these settings produce greater levels of over-expressing harmonious emotions.

This interaction highlights the importance of our research identifying different emotion

dimensions for expressive norms as investigating concept as a unitary construct would miss

this nuance.

The internal consistency of each dimension across all scenarios and both samples

exceeded .90. This suggests that the scale can provide precise estimates of expressive norms.

It is notable is that this consistency was achieved even though we prioritized breadth over

internal consistency when selecting emotion items, ensuring the selected emotions

represented the full spectrum of emotions. Also, given each subscale was only eight items

long, this level of consistency is promising as Cronbach’s alpha tends to increase with scale

length. This tendency arises because, even if newly added items are poor quality, they can

still increase the overall reliability coefficient simply by virtue of increasing parameter

stability. It might be tempting for researchers to reduce the scale given the pressure for

shorter scales in psychological research (Rammstedt & Beierlein, 2014). However, our final

choice of an eight-item-per-subscale approach offers comprehensive coverage and stable

results that are harder to achieve with shorter scales more influenced by individual responses

to specific emotions.

Interestingly, there were only weak associations between age and emotion display

suppression primarily in close/private contexts. We found that older participants were more

likely to suppress their emotions in private settings with close interactants. This suggests that

general etiquette regarding emotional expression in public spaces or more distant

relationships remains relatively stable across age groups. Contrastingly, there might be weak

generational shifts with how emotion is expressed in private with people close to us as

younger participants endorsed more authentic emotional displays.

 27

A notable outcome of this study was the identification of a weak to moderate

correlation between ERS scores and ERQ-suppression, which captures the habitual hiding of

emotions. These results suggest a discrepancy between our perceived norms for emotion

expression and the actual daily management of these emotions. Multiple factors might

account for this discrepancy. One possibility is the divergence between how individuals

handle their own emotions (as exemplified by the ERQ item: “I keep my emotions to myself”)

versus an idealized perspective of personal or social behavioral expectations. Consistent with

this notion, prior studies have suggested that individuals maintain distinct expressive norms

for themselves in comparison to others when it comes to gender identity (Dawel, Ashhurst,

et al., 2023). The infrequent experience of specific emotions might also play a role. Even if an

individual seldom encounters a particular emotion, they might possess robust beliefs about

its appropriate expression. In contrast, the strength of the emotion likely dictates one's

capacity to purposefully choose how it is displayed, as people may have higher motivation to

regulate the expression of more intense emotions, particularly negative ones.

Limitations and Future Directions

The present study validated the ERS to measure expressive norms (“display rules”) by

asking participants how they thought they should express emotions in different scenarios.

However, this wording could be adapted to target expressive behavior. For example, by asking

participants how do they express the emotions. The wording could also be adapted to capture

habitual expression regulation, akin to that captured by the (ERQ; Gross & John, 2003); for

example, by asking “When I feel the following emotions, I…”. However, the factor structure

of the ERS is yet to be tested for these alternative wordings and thus there will be a need to

reassess the validity of adapted versions of the scales.

 28

Although our findings are limited to four broad social scenarios, they suggest a

potential universality of expressive norms that could be used to guide future research into

the nuances of how norms vary across expressors and alternative scenarios. For example, our

scenarios did not vary the relative status of the interactants, a factor that might incur greater

differentiation between vulnerable and disharmonious emotions.

Although it is clear that relationship and context factors plays an important role, our

findings did not measure and, therefore cannot speak to, the intention or motivation behind

expressive norms. That is, what are we trying to communicate and achieve in each context

which can impact on the perceived appropriateness of expressed emotion. Several

participants expressed desire for more contextual information to help guide their decisions.

For example, whether they felt the given emotion towards the interactant or an alternative

source. Future research can investigate how expressive norms vary as a result of the purpose

of communication.

The modelled the effect of situational demands by asking participants to imagine how

they would display emotions in each of the four scenarios. However, there are well-known

limitations of this approach, including self-presentation (such as social desirability),

acquiescence and motivation, the capacity for individuals to predict their own behavior, and

participant motivation. Therefore, multiple sources of data such as informant in addition to

real-world observation, or even extensions using multi-trait multi-method and contemporary

Structural Equation Modelling variations should be considered, in addition to a larger array of

convergent validity measure.

A strength of this study was a representative UK population, going beyond the

convenience sample of undergraduates that typifies psychological research. The online

collection panels have been supported as a strong method of collecting representative data
 29

(Douglas et al., 2023; Litman et al., 2021). However, data is still limited to one country group.

This was by design, to remove extraneous variables when establishing the model. However,

now that we have a viable model, we can move forward to cross-cultural testing given

differences in socio-cultural expressive norms. For example, comparing cultures that

traditionally differ in overt emotional displays. Until this is established, generalization beyond

the UK context should be considered carefully.

In addition to the factors we have discussed, it is important to consider the

motivational aspect of expressive norms. Expressive norms are often driven by various goals,

such as maintaining one's appearance, nurturing relationships, or safeguarding personal

agendas. In our current study, we did not explicitly account for these motivational factors,

and it is likely that individuals' motivations within different contexts vary considerably. For

instance, in close-private relationships, individuals may exhibit more relaxed and unfiltered

expressions, while also making efforts to maintain positive social impressions to nurture these

relationships. This intriguing possibility highlights the need to explore the role of individual

differences in our research.

Conclusions

The present study robustly established the three-dimensional structure of expressive

norms initially found in Dawel et al. (2023) and generated a new scale, the ERS, to measure

these dimensions. This structure highlights the importance of considering expressive norms

as multidimensional. A key advantage of the ERS rating method is that it measures both

suppression and over-expression, and provides a mid-point that allows raters to indicate

when they would not modify an expression in either direction. These scores are easy to

interpret: Positive scores indicate the degree of over-expression and negative scores indicate

the degree of suppression. The ERS, which has so far been used to measure expressive norms,
 30

also affords excellent potential for adaptation to self-reported expressive behavior, as well as

different interactants and contexts. Overall, the work presented herein provides a strong

foundation for unifying understanding of expression regulation, with capacity to bridge

current gaps in understanding between the various fields invested in this topic.
 31

References

Bänziger, T., Mortillaro, M., & Scherer, K. R. (2012). Introducing the Geneva

Multimodal Expression Corpus for experimental research on emotion perception. Emotion,

12(15), 1161–1179. https://doi.org/10.1037/a0025827

Baron-Cohen, S., Golan, O., Wheelwright, S., Granader, Y., & Hill, J. (2010). Emotion

word comprehension from 4 to 16 years old: A developmental survey. Frontiers in

Evolutionary Neuroscience, 2(November), 1–8. https://doi.org/10.3389/fnevo.2010.00109

Barrett, L. F., Lindquist, K. A., & Gendron, M. (2007). Language as context for the

perception of emotion. Trends in Cognitive Sciences, 11(8), 327–332.

https://doi.org/10.1016/j.tics.2007.06.003

Bednarek, M. (2008). Emotion talk across corpora. Palgrave MacMillan.

https://doi.org/10.1057/9780230285712

Bornstein, M. H. (1995). Form and Function: Implications for Studies of Culture and

Human Development. Culture & Psychology, 1(1), 123–137.

https://doi.org/10.1177/1354067X9511009

Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford

Publications.

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral

Research, 1(2), Article 2. https://doi.org/10.1207/s15327906mbr0102_10

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement

invariance. Structural Equation Modeling, 14, 464–504.

https://doi.org/10.1080/10705510701301834
 32

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing

measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233–

255. https://doi.org/10.1207/S15328007SEM0902_5

Church, A. T., Katigbak, M. S., Reyes, J. A. S., & Jensen, S. M. (1998). Language and

organisation of Filipino emotion concepts: Comparing emotion concepts and dimensions

across cultures. In Cognition and Emotion (Vol. 12, Issue 1).

https://doi.org/10.1080/026999398379781

Cowen, A. S., & Keltner, D. (2020). What the face displays: Mapping 28 emotions

conveyed by naturalistic expression. American Psychologist, 75(3), 349–364.

https://doi.org/10.1037/amp0000488

Dawel, A., Ashhurst, C., & Monaghan, C. (2022). A 3-dimensional model of emotional

display rules: Model invariance, external validity, and gender differences. Emotion.

https://doi.org/10.1037/emo0001176

Dawel, A., Ashhurst, C., & Monaghan, C. (2023). A three-dimensional model of

emotional display rules: Model invariance, external validity, and gender differences. Emotion

(Washington, D.C.), 23(5). https://doi.org/10.1037/EMO0001176

Dawel, A., Monaghan, C., Shou, Y., Mewton, P., Farrer, L. M., Gulliver, A., Newman, E.,

McCallum, S. M., & Cherbuin, Nicolas, N. (2023). Emotional display rules are the same for

online audio-visual and face-to-face scenarios across five interactant types.

https://doi.org/10.31234/osf.io/v7kgh

Deneubourg, J. L., Pasteels, J. M., & Verhaeghe, J. C. (1983). Probabilistic behaviour in

ants: A strategy of errors? Journal of Theoretical Biology, 105(2), 259–271.

https://doi.org/10.1016/S0022-5193(83)80007-1
 33

Dewaele, J. M., & Pavlenko, A. (2002). Emotion vocabulary in interlanguage. Language

Learning, 52(2), 263–322. https://doi.org/10.1111/0023-8333.00185

Dorigo, M., Maniezzo, V., & Colorni, A. (1991). Ant System: An Autocatalytic

Optimizing Process (Technical 91-016 Revised). Dipartimento di Elettronica, Politecnico di

Milano.

Dorigo, M., & Stützle, T. (2010). Ant Colony Optimization: Overview and Recent

Advances. In M. Gendreau & J.-Y. Potvin (Eds.), Handbook of Metaheuristics (pp. 227–263).

Springer US. https://doi.org/10.1007/978-1-4419-1665-5_8

Douglas, B. D., Ewell, P. J., & Brauer, M. (2023). Data quality in online human-subjects

research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. Plos

One, 18(3), e0279720. https://doi.org/10.1371/journal.pone.0279720

Ekman, P., & Cordaro, D. (2011). What is meant by calling emotions basic. Emotion

Review, 3(4), 364–370. https://doi.org/10.1177/1754073911410740

Fontaine, J. R. J., Scherer, K. R., Roesch, E. B., & Ellsworth, P. C. (2007). The world of

emotions is not two-dimensional. Psychological Science, 18(12), 1050–1057.

https://doi.org/10.1111/j.1467-9280.2007.02024.x

Fredrickson, B. L. (2013). Positive emotions broaden and build. In P. Devine & P. Ashby

(Eds.), Advances on Experimental Social Psychology (Vol. 47, pp. 1–53). Academic Press.

https://doi.org/10.1016/B978-0-12-407236-7.00001-2

Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation

processes: Implications for affect, relationships, and well-being. Journal of Personality and

Social Psychology, 85(2), 348–362. https://doi.org/10.1037/0022-3514.85.2.348

Grühn, D. (2016). An English word database of EMOtional TErms (EMOTE).

Psychological Reports, 119(1), 290–308. https://doi.org/10.1177/0033294116658474

 34

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.

Psychometrika, 30(2), Article 2. https://doi.org/10.1007/bf02289447

Hupka, R. B., Lenton, A. P., & Hutchinson, K. A. (1999). Universal development of

emotion categories in natural language. Journal of Personality and Social Psychology, 77(2),

247–278. https://doi.org/10.1037/0022-3514.77.2.247

Izard, C. E. (1977). Human emotions. Plenum Press.

Jackson, J. C., Watts, J., Henry, T. R., List, J.-M., Forkel, R., Mucha, P. J., Greenhill, S. J.,

Gray, R. D., & Lindquist, K. A. (2019). Emotion semantics show both cultural variation and

universal structure. Science, 1522, 1517–1522.

Jak, S., Jorgensen, T. D., Verdam, M. G. E., Oort, F. J., & Elffers, L. (2021). Analytical

power calculations for structural equation modeling: A tutorial and Shiny app. Behavior

Research Methods, 53(4), 1385–1406. https://doi.org/10.3758/s13428-020-01479-0

JASP Team. (2023). JASP (0.17) [Computer software]. https://jasp-stats.org/

Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2008).

semTools: Useful tools for structural equation modeling (0.5-1) [R Package]. https://CRAN.R-

project.org/package=semTools

Kaneko, T., Kobayashi, T., & Takami, M. (2006). The use of emotional expressions in

English by non-native speakers: A corpus-based comparative study. Gakuen, 785, 45–55.

Kazak, A. E. (2018). Editorial: Journal article reporting standards. American

Psychologist, 73(1), 1–2. https://doi.org/10.1037/amp0000263

Keltner, D., Sauter, D., Tracy, J., & Cowen, A. (2019). Emotional expression: Advances

in basic emotion theory. Journal of Nonverbal Behavior, 43(2), 133–160.

https://doi.org/10.1007/s10919-019-00293-3
 35

Kerber, A., Schultze, M., Müller, S., Rühling, R. M., Wright, A. G. C., Spitzer, C., Krueger,

R. F., Knaevelsrud, C., & Zimmermann, J. (2022). Development of a Short and ICD-11

Compatible Measure for DSM-5 Maladaptive Personality Traits Using Ant Colony Optimization

Algorithms. Assessment, 29(3), 467–487. https://doi.org/10.1177/1073191120971848

Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. T. (2017). Measurement Invariance Testing

with Many Groups: A Comparison of Five Approaches. Structural Equation Modeling: A

Multidisciplinary Journal, 24(4), 524–544. https://doi.org/10.1080/10705511.2017.1304822

Kyriazos, T. A. (2018). Applied psychometrics: Sample zize and sample power

considerations in factor analysis (EFA, CFA) and SEM in general. Psychology, 9(8), 2207–2230.

https://doi.org/10.4236/psych.2018.98126

Lee, S. Y. M. (2010). A linguistic approach to emotion detection and classification

[Doctorial dissertation, Hong Kong Polytechnic University].

https://theses.lib.polyu.edu.hk/handle/200/7201

Litman, L., Moss, A., Rosenzweig, C., & Robinson, J. (2021). Reply to MTurk, Prolific or

panels? Choosing the right audience for online research. Choosing the Right Audience for

Online Research (January 28, 2021).

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3775075

Lutz, C., & White, G. M. (1986). The anthropology of emotions. Annual Review of

Anthropology, 15(1), 405–436. https://doi.org/10.1146/annurev.an.15.100186.002201

MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and

determination of sample size for covariance structure modeling. Psychological Methods, 1,

130–149. https://doi.org/10.1037/1082-989x.1.2.130
 36

Manokara, K., Fischer, A., & Sauter, D. (2023). Display rules differ between positive

emotions: Not all that feels good looks good. Emotion, 23(1), 243–260.

https://doi.org/10.1037/emo0001078

Masuda, T., Wang, H., Ishii, K., & Ito, K. (2012). Do surrounding figures’ emotions affect

judgment of the target figure’s emotion? Comparing the eye-movemenl patterns of European

Canadians, Asian Canadians, Asian international students, and Japanese. Frontiers in

Integrative Neuroscience, 6(72), 1–9. https://doi.org/10.3389/fnint.2012.00072

Matsumoto, D., Yoo, S. H., Fontaine, J., Anguas-Wong, A. M., Arriola, M., Ataca, B.,

Bond, M. H., Boratav, H. B., Breugelmans, S. M., Cabecinhas, R., Chae, J., Chin, W. H.,

Comunian, A. L., Degere, D. N., Djunaidi, A., Fok, H. K., Friedlmeier, W., Ghosh, A., Glamcevski,

M., … Grossi, E. (2008). Mapping expressive differences around the world: The relationship

between emotional display rules and individualism versus collectivism. Journal of Cross-

Cultural Psychology, 39(1), 55–74. https://doi.org/10.1177/0022022107311854

Matsumoto, D., Yoo, S. H., Hirayama, S., & Petrova, G. (2005). Development and

validation of a measure of display rule knowledge: The display rule assessment inventory.

Emotion (Washington, D.C.), 5(1), 23–40. https://doi.org/10.1037/1528-3542.5.1.23

McNair, D. M., Lorr, M., & Droppleman, L. F. (1971). Manual for the profile of mood

states. Educational and Industrial Testing Services.

Meuleman, B., Żółtak, T., Pokropek, A., Davidov, E., Muthén, B., Oberski, D. L., Billiet,

J., & Schmidt, P. (2023). Why Measurement Invariance is Important in Comparative Research.

A Response to Welzel et al. (2021). Sociological Methods & Research, 52(3), 1401–1419.

https://doi.org/10.1177/00491241221091755
 37

Norvig, P. (2009). Natural language corpus data. In T. Segaran & J. Hammerbacher

(Eds.), Beautiful Data: The Stories Behind Elegant Data Solutions (pp. 219–242). O’Reilly

Media, Inc.

Ogren, M., & Sandhofer, C. M. (2021). Emotion words in early childhood: A language

transcript analysis. Cognitive Development, 60, 101122–101122.

https://doi.org/10.1016/j.cogdev.2021.101122

Olaru, G., & Danner, D. (2021). Developing Cross-Cultural Short Scales Using Ant

Colony Optimization. Assessment, 28(1), 199–210.

https://doi.org/10.1177/1073191120918026

Olaru, G., Witthöft, M., & Wilhelm, O. (2015). Methods matter: Testing competing

models for designing short-scale Big-Five assessments. Journal of Research in Personality, 59,

56–68. https://doi.org/10.1016/j.jrp.2015.09.001

Posit team. (2023). RStudio: Integrated Development Environment for R (2023.6.1.524

(Mountain Hydrangea)) [Computer software]. Posit Software, PBC. http://www.posit.co/

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and

reporting: The state of the art and future directions for psychological research.

Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004

R Core Team. (2021). R: A language and environment for statistical computing (4.2.1)

[Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/

R Core Team. (2023). R: A Language and Environment for Statistical Computing (4.3.1)

[Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/

Raiche, G., & Magis, D. (2022). nFactors: Parallel Analysis and Other Non Graphical

Solutions to the Cattell Scree Test [Computer software]. https://CRAN.R-

project.org/package=nFactors
 38

Raîche, G., Walls, T. A., Magis, D., Riopel, M., & Blais, J.-G. (2013). Non-graphical

solutions for Cattell’s scree test. Methodology: European Journal of Research Methods for the

Behavioral and Social Sciences, 9, 23. https://doi.org/10.1027/1614-2241/a000051

Rammstedt, B., & Beierlein, C. (2014). Can’t we make it any shorter? The limits of

personality assessment and ways to overcome them. Journal of Individual Differences, 35(4),

212–220. https://doi.org/10.1027/1614-0001/a000141

Revelle, W. (2023). Psych: Procedures for psychological, psychometric, and personality

research. Northwestern University, Evanston, Illinois. https://CRAN.R-

project.org/package=psych

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of

Statistical Software, 48, 1–36. https://doi.org/10.18637/jss.v048.i02

Russell, J. A. (1983). Pancultural aspects of the human conceptual organization of

emotions. Journal of Personality and Social Psychology, 45(6), 1281–1288.

https://doi.org/10.1037/0022-3514.45.6.1281

Scherer, K. R. (2005). What are emotions? And how can they be measured? Social

Science Information, 44(4), 695–729. https://doi.org/10.1177/0539018405058216

Schultze, M. (2023). stuart: Subtests Using Algorithmic Rummaging Techniques

(0.10.2) [R Package].

Schultze, M., & Eid, M. (2018). Identifying Measurement Invariant Item Sets in Cross-

Cultural Settings Using an Automated Item Selection

Procedure. Methodology, 14(4), 177–188. https://doi.org/10.1027/1614-2241/a000155

Sellbom, M., & Tellegen, A. (2019). Factor Analysis in Psychological Assessment

Research: Common Pitfalls and Recommendations. Psychological Assessment, 31(2), 1428–

1441. https://doi.org/10.1037/pas0000623
 39

Shaver, P., Schwartz, J., Kirson, D., & O’Connor, C. (1987). Emotion knowledge: Further

exploration of a prototype approach. Journal of Personality and Social Psychology, 52(6),

1061–1086. https://doi.org/10.1037/0022-3514.52.6.1061

Shiota, M. N., Neufeld, S. L., Danvers, A. F., Osborne, E. A., Sng, O., & Yee, C. I. (2014).

Positive emotion differentiation: A functional approach. Social and Personality Psychology

Compass, 8(3), 104–117. https://doi.org/10.1111/spc3.12092

Steenkamp, J. E. M., & Baumgartner, H. (1998). Assessing Measurement Invariance in

Cross-National Consumer Research. Journal of Consumer Research, 25(1), Article 1.

https://doi.org/10.1086/209528

Sutherland, C. A. M., Oldmeadow, J. A., Santos, I. M., Towler, J., Burt, D. M., & Young,

A. W. (2013). Social inferences from faces: Ambient images generate a three-dimensional

model. Cognition, 127(1), 105–118. https://doi.org/10.1016/j.cognition.2012.12.001

van der Eijk, C., & Rose, J. (2015). Risky Business: Factor Analysis of Survey Data –

Assessing the Probability of Incorrect Dimensionalisation. PLoS ONE, 10(3), Article 3.

https://doi.org/10.1371/journal.pone.0118900

Van Goozen, S., & Frijda, N. H. (1993). Emotion words used in six European countries.

European Journal of Social Psychology, 23(1), 89–95.

https://doi.org/10.1002/ejsp.2420230108

Waldman, I. D., King, C. D., Poore, H. E., Luningham, J. M., Zinbarg, R. M., Krueger, R.

F., Markon, K. E., Bornovalova, M., Chmielewski, M., Conway, C., Dretsch, M., Eaton, N. R.,

Forbes, M. K., Forbush, K., Naragon-Gainey, K., Greene, A. L., Haltigan, J. D., Ivanova, M.,

Joyner, K., … Zald, D. (2023). Recommendations for Adjudicating Among Alternative Structural

Models of Psychopathology. Clinical Psychological Science, 11(4), 616–640.

https://doi.org/10.1177/21677026221144256
 40

Wang, G., Seibert, S. E., & Boles, T. L. (2011). Synthesizing what we know and looking

ahead: A meta-analytical review of 30 years of emotional labor research. In Research on

Emotion in Organizations (Vol. 7, pp. 15–43). https://doi.org/10.1108/S1746-

9791(2011)0000007006

Wickham, H., Chang, W., & Wickham, M. H. (2016). Package ‘ggplot2.’ Create Elegant

Data Visualisations Using the Grammar of Graphics. Version, 2(1), 1–189.

Wierzbicka, A. (1992). Defining emotion concepts. Cognitive Science, 16(4), 539–581.

https://doi.org/10.1016/0364-0213(92)90031-O
 41

Supplementary Materials for:

The Expression Regulation Scale (ERS): Validation of three emotion dimensions for

expressive norms with close and distant others in private and public contexts

Conal Monaghan1, Yiyun Shou1,2,3, Paige Mewton1, Anika Quayle1, Amy Dawel*1

1
School of Medicine and Psychology, The Australian National University.

2
Lloyd’s Register Foundation Institute for the Public Understanding of Risk, National

University of Singapore, Singapore.

3
Saw Swee Hock School of Public Health, National University

Author Note

This research was funded by an ANU College of Health and Medicine TRANSFORM

Career Development Fellowship to AD. We have no conflicts of interest to disclose. The

dataset is available at <OSF link when published>. *Correspondence concerning this article

should be addressed to Amy Dawel, School of Medicine and Psychology, The Australian

National University, Canberra, ACT 2600, Australia. Email: amy.dawel@anu.edu.au

 42

Supplement A — ACO Heuristics and Final Emotions for Each Domain

Table S1

ACO Heuristics and Final Emotions for Each Domain

Domain Emotion Heuristic 10-item selection Final 8-item ERS

Harmonious
Admiration Admiration Admiration
Amusement
Awe
Calm
Compassion X Compassion Compassion
Curiosity
Delight Delight Delight
Desire
Elation
Enthusiasm
Excitement X Excitement Excitement
Gratitude
Happiness X Happiness Happiness
Hope X Hope Hope
Interest
Joy
Love
Pleasure X Pleasure Pleasure
Pride X Pride Pride
Relaxation Relaxation Admiration
Relief X
Surprise X Surprise
Sympathy X
Triumph
Vulnerable
Anxiety
Confusion
Depression
Despair X Despair Despair
Disappointment X Disappointment
Distress X Distress Distress
Embarrassment X Embarrassment Embarrassment
Fear X Fear Fear
Gloom
Grief
Guilt X Guilt Guilt
Hopelessness
Hurt X Hurt Hurt
Pain
 43

Panic
Regret
Remorse
Sadness X Sadness Sadness
Shame
Shock
Sorrow
Stress Stress
Terror
Unhappiness Unhappiness Unhappiness
Worry
Pity
Disharmonious
Anger X Anger Anger
Annoyance
Boredom Boredom Boredom
Contempt Contempt
Disgust X Disgust Disgust
Envy
Frustration X Frustration
Fury Fury Fury
Hatred Hatred Hatred
Irritation X Irritation Irritation
Jealousy X Jealousy Jealousy
Rage
Resentment Resentment Resentment
Note. X indicates this emotion started with an initial pheromone (heuristic) of 15.
 44

Supplement B — Scree Plots from EFA

Figure B

Scree Plots from EFA

Scree plot for Private / Close Scree plot for Private / Distant
with parallel analysis (200 iterations) with parallel analysis (200 iterations)

24.0
23.5 21.0
23.0 20.5
22.5 20.0
22.0 19.5
21.5 19.0
21.0 18.5
20.5 18.0
20.0 actual 17.5 actual
19.5
19.0 17.0
simulated 16.5 simulated
18.5
18.0 16.0
17.5 15.5
17.0 15.0
16.5 14.5
16.0 14.0
15.5 13.5
15.0
14.5 13.0
14.0 12.5
13.5 12.0
13.0 11.5
Eigenvalue

Eigenvalue
12.5 11.0
12.0 10.5
11.5 10.0
11.0 9.5
10.5
10.0 9.0
9.5 8.5
9.0 8.0
8.5 7.5
8.0 7.0
7.5 6.5
7.0 6.0
6.5 5.5
6.0
5.5 5.0
5.0 4.5
4.5 4.0
4.0 3.5
3.5 3.0
3.0 2.5
2.5 2.0
2.0 1.5
1.5
1.0 1.0
0.5 0.5
0.0 0.0
-0.5 -0.5
-1.0 -1.0

1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364 1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364
Factor Number Factor Number
Maximum Likelihood Factoring Maximum Likelihood Factoring

Scree plot for Public / Close Scree plot for Public / Distant
with parallel analysis (200 iterations) with parallel analysis (200 iterations)
22.0
21.5
20.0 21.0
19.5 20.5
19.0 20.0
18.5 19.5
18.0 19.0
17.5 18.5
17.0 18.0
16.5 actual 17.5 actual
16.0 17.0
simulated simulated
15.5 16.5
15.0 16.0
14.5 15.5
14.0 15.0
13.5 14.5
13.0 14.0
12.5 13.5
13.0
12.0
12.5
11.5 12.0
11.0 11.5
Eigenvalue

Eigenvalue

10.5 11.0
10.0 10.5
9.5 10.0
9.0 9.5
8.5 9.0
8.0 8.5
7.5 8.0
7.0 7.5
6.5 7.0
6.0 6.5
5.5 6.0
5.0 5.5
4.5 5.0
4.0 4.5
3.5 4.0
3.5
3.0
3.0
2.5 2.5
2.0 2.0
1.5 1.5
1.0 1.0
0.5 0.5
0.0 0.0
-0.5 -0.5
-1.0 -1.0

1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364 1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364
Factor Number Factor Number
Maximum Likelihood Factoring Maximum Likelihood Factoring

Note. These figures were created using the Psych (Revelle, 2023) and GGplot2 (Wickham et
al., 2016) package for R (R Core Team, 2021).
 45

Supplement C — EFA Results for 64 Emptions

Table C

EFA Variance Explained and Inter-Factor Correlations for 64 Emotions

Proportion of variance Factor

Model
explained correlations
Total Explained
Structure Scenario Factor 1 2 3
Variance Variance
Two-factor solution
Private-
close 1 .33 .56 - - NA
2 .26 .44 .21 - NA
Private-
distant 1 .32 .63 - - NA
2 .19 .37 .09 - NA
Public-
close 1 .31 .58 - - NA
2 .22 .42 -.07 - NA
Public-
distant 1 .32 .63 - - NA
2 .19 .37 .08 - NA
1
Three-factor solution
Private-
close 1 .24 .39 - - -
2 .26 .42 .23 - -
3 .12 .20 .73 .09 -
Private-
distant 1 .22 .42 - - -
2 .19 .35 .11 - -
3 .12 .23 .72 .04 -
Public-
close 1 .23 .40 - - -
2 .23 .40 -.04 - -
3 .11 .20 .68 -.08 -
Public-
distant 1 .21 .39 - - -
2 .19 .35 .09 - -
3 .14 .26 .71 .03 -
 46

Supplement D— Internal Consistencies for ERS Subscales

Table D - 1

Internal Consistencies for ERS Subscales in Exploratory Sample

Scenario Subscale Correlations

Disharmonious

Disharmonious
Harmonious

Vulnerable

Vulnerable
Estimate Subscale
Private-close
Alpha .95 .92 .93 Harmonious -.04 .01
Omega .95 .92 .93 Disharmonious - .67
Private-distant
Alpha .91 .93 .92 Harmonious .00 -.03
Omega .91 .93 .92 Disharmonious - .78
Public-close
Alpha .94 .93 .92 Harmonious -.14 -.16
Omega .94 .93 .92 Disharmonious - .76
Public-distant
Alpha .91 .93 .92 Harmonious -.03 -.01
Omega .91 .94 .92 Disharmonious - .79
Note. Internal consistencies for the combined vulnerable and disharmonious (negative
valence) factor were between .95 and .96.
 47

Table D - 2

Internal Consistencies for ERS Subscales in Confirmatory Sample

Scenario Subscale Correlations

Disharmonious

Disharmonious
Harmonious

Vulnerable

Vulnerable
Estimate Subscale
Private-close
Alpha .95 .92 .92 Harmonious -.07 .03
Omega .95 .92 .92 Disharmonious - .69
Private-distant
Alpha .91 .92 .91 Harmonious -.06 -.05
Omega .91 .92 .91 Disharmonious - .75
Public-close
Alpha .93 .92 .92 Harmonious -.05 -.02
Omega .93 .92 .92 Disharmonious - .75
Public-distant
Alpha .92 .91 .90 Harmonious .02 .10
Omega .92 .92 .90 Disharmonious - .78
Note. Internal consistencies for the combined vulnerable and disharmonious (negative
valence) factor were between .95 for all scenarios.
 48

Supplement E — Estimates of Central Tendency and Dispersion for ERS Subscales

Table E - 1

Estimates of Central Tendency and Dispersion for ERS Subscales in Exploratory Sample

Close- Close- Distant- Distant-

ERS subscale Total private public private public p-value
Harmonious
Median (IQR) 10 (0, 31) 16 (0, 40) 12 (0, 37) 9 (0, 27) 7 (0, 21) <0.001
Mean (SD) 17 (27) 25 (29) 20 (27) 13 (24) 9 (26) <0.001
Vulnerable
Median (IQR) -32 (-57, -11) -8 (-24, 0) -32 (-52, -13) -41 (-67, -24) -47 (-73, -28) <0.001
Mean (SD) -34 (32) -11 (30) -34 (28) -43 (29) -49 (29) <0.001
Disharmonious
Median (IQR) -41 (-69, -17) -15 (-37, 0) -41 (-67, -21) -53 (-76, -30) -58 (-80, -33) <0.001
Mean (SD) -42 (33) -20 (31) -43 (30) -51 (30) -55 (30) <0.001
Negative
Median (IQR) -37 (-61, -15) -12 (-29, 0) -37 (-57, -19) -45 (-70, -27) -52 (-76, -30) <0.001
Mean (SD) -38 (31) -15 (29) -38 (28) -47 (28) -52 (28) <0.001
Note. IQR = inter-quartile Range. Medians differences indicated with Kruskal-Wallis rank sum
test, mean differences estimated using repeated-measures ANOVA.

Table E - 2

Estimates of Central Tendency and Dispersion for ERS Subscales in Confirmatory Sample

Close- Close- Distant- Distant-

ERS subscale Total private public private public p-value
Harmonious
Median (IQR) 9 (0, 30) 13 (1, 39) 11 (0, 35) 8 (0, 27) 6 (-4, 25) <.001
Mean (SD) 17 (27) 24 (29) 19 (26) 14 (24) 10 (26) <.001
Vulnerable
Median (IQR) -31 (-58, -10) -8 (-24, 0) -27 (-51, -11) -44 (-66, -24) -50 (-71, -30) <.001
Mean (SD) -34 (33) -10 (29) -31 (29) -44 (29) -50 (28) <.001
Disharmonious
Median (IQR) -43 (-66, -18) -17 (-39, -2) -40 (-62, -19) -53 (-74, -29) -58 (-81, -36) <.001
Mean (SD) -42 (33) -20 (32) -40 (30) -51 (30) -56 (29) <.001
Negative
Median (IQR) -37 (-61, -16) -13 (-32, 0) -35 (-55, -16) -49 (-68, -27) -54 (-75, -33) <.001
Mean (SD) -38 (32) -15 (29) -36 (28) -47 (28) -53 (27) <.001
Note. IQR = inter-quartile Range. Medians differences indicated with Kruskal-Wallis rank sum
test, mean differences estimated using repeated-measures ANOVA.
 49

Supplement F — Associations with External Variables

Table F - 1

Associations with External Variables in Exploratory Sample

External variable ERS subscale

Scenario Harmonious Disharmonious Vulnerable Negative
ERQ-suppression
Private-close .079 -.084 -.050 -.069
Private-distant .120** -.108* -.102* -.109*
Public-close .062 -.051 -.037 -.046
Public-distant .123** -.056 -.042 -.052
ERQ-reappraisal
Private-close -.046 -.105* -.196*** -.156***
Private-distant -.060 -.078 -.170*** -.130**
Public-close -.072 -.006 -.095* -.052
Public-distant -.080 -.055 -.138** -.101*
Age
Private-close -.130** .056 .125** .094*
Private-distant -.126** -.048 .032 -.009
Public-close -.055 -.208*** -.068 -.147***
Public-distant -.058 -.227*** -.125** -.188***
Note. * p < .05. ** p < .01. *** p < .001.
 50

Table F - 2

Associations with External Variables in Confirmatory Sample

External variable ERS subscale

Scenario Harmonious Disharmonious Vulnerable Negative
ERQ-suppression
Private-close .080 -.103* -.026 -.070
Private-distant .101* -.143** -.092* -.124**
Public-close .112* -.056 -.061 -.062
Public-distant .102* -.063 -.022 -.045
ERQ-reappraisal
Private-close -.076 -.211*** -.195*** -.216***
Private-distant -.092* -.172*** -.212*** -.203***
Public-close -.102* -.092* -.142** -.123**
Public-distant -.081 -.092* -.136** -.120**
Age
Private-close -.008 .098* .154*** .132**
Private-distant .037 .011 .115** .066
Public-close -.027 -.116** -.043 -.086
Public-distant -.002 -.155*** -.071 -.120**
Note. * p < .05. ** p < .01. *** p < .001.
 51

Supplement G — Expression Regulation Scale (ERS)

The full survey used in this study is available at: <xxosf link>. This supplement presents

the general format for the final 24-item ERS, scoring key, and R code for scoring the ERS

subscales.

Main task instructions: The next few tasks ask how you think you should express

emotions with people who are very close to you (for example, family and close friends) and

people not so close to you (for example, colleagues and acquaintances) in private (for

example, in either your own home or someone else's) and in public (for example, in a

restaurant or park).

Example specific instructions for private-close scenario:1 For this task, we would like

to know how you think you should2 express emotions in private (for example, in either your

own home or someone else's) with people very close to you (for example, family and close

friends).

1
The general format for the specific instructions for each scenario is: For this task, we would like to
know how you think you should express emotions in <context> (for example, <define context>) with
<interactant> (for example, <define interactant>). Please take a moment, while the below clock counts down,
to imagine interacting in <context> with <interactant>. How should you express each of the following emotions
in <context> with <interactant>?
2
The ERS has been validated for people’s beliefs about how they should express emotions, to capture
display rule norms. However, the ERS wording could be adapted to capture expressive behaviour by replacing
“how you think you should express emotions” and “How should you express each…” with “how you express
emotions” and “How do you express each…” respectively. The wording could also be adapted to capture habitual
expression regulation, akin to that captured by the Emotion Regulation Questionnaire (ERQ; Gross & John 2003);
for example, by asking “When I feel the following emotions, I…”. However, the factor structure of the ERS is yet
to be tested for these alternative wordings and thus there will be a need to reassess the validity of adapted
versions of the scales.
 52

How should you express each of the following emotions in private with people very close to
you?

Please make your rating anywhere on the scale from -100 to +100, where:
-100 = express no emotion/hide my emotion completely
-50 = express less than I feel
0 = express it as I feel it
+50 = express more than I feel
+100 = express much more than I feel

Express no
emotion/ Express less Express it Express more Express much
hide my emotion than I feel as I feel it than I feel more than I feel
completely
-100 -50 0 50 100
Admiration
Compassion
Delight
Excitement
Happiness
Hope
Pleasure
Pride
Despair
Distress
Embarrassment
Fear
Guilt
Hurt
Sadness
Unhappiness
Anger
Boredom
Disgust
Fury
Hatred
Irritation
Jealousy
Resentment
 53

ERS scoring. The ERS is scored by averaging the ratings within each subscale, allowing for up

to one missing item per subscale.

Harmonious = mean(Admiration, Compassion, Delight, Excitement, Happiness, Hope,

Pleasure, Pride)

Vulnerable = mean(Despair, Distress, Embarrassment, Fear, Guilt, Hurt, Sadness,

Unhappiness

Disharmonious = mean(Anger, Boredom, Disgust, Fury, Hatred, Irritation, Jealousy,

Resentment)

The general R code for scoring the ERS is:

### Note: This code uses the dplyr package in R.

### Note: Variables are named in the standard format: prefix_Emotion, e.g., clspri_Admiration
### Note: Prefixes indicate the scenario type as follows: clspri = close-private; clspub = close-public; dispri =
distant-private; dispub = distant-public.

# Define lists of variables for each ERS subscale

har_cols <- c("Admiration", "Compassion", "Delight", "Excitement", "Happiness", "Hope", "Pleasure", "Pride")
vul_cols <- c("Despair", "Distress", "Embarrassment", "Fear", "Guilt", "Hurt", "Sadness", "Unhappiness")
dis_cols <- c("Anger", "Boredom", "Disgust", "Fury", "Hatred", "Irritation", "Jealousy", "Resentment")
neg_cols <- c("Despair", "Distress", "Embarrassment", "Fear", "Guilt", "Hurt", "Sadness", "Unhappiness",
"Anger", "Boredom", "Disgust", "Fury", "Hatred", "Irritation", "Jealousy", "Resentment")

# Function to calculate mean allowing for a maximum of one NA value

mean_oneNAmax <- function(x) {
if (sum(is.na(x)) > 1) {
return(NA)
} else {
return(mean(x, na.rm = TRUE))
}
}

# Calculate row means with the custom function

df <- df %>% # df is the dataframe
rowwise() %>%
mutate(
clspri_har = mean_oneNAmax(c_across(matches(paste0("clspri_", har_cols)))), # e.g., Calculates the
harmonious subscale score for the close-private scenario
clspri_vul = mean_oneNAmax(c_across(matches(paste0("clspri_", vul_cols)))),
clspri_dis = mean_oneNAmax(c_across(matches(paste0("clspri_", dis_cols)))),
clspri_neg = mean_oneNAmax(c_across(matches(paste0("clspri_", neg_cols)))),
clspub_har = mean_oneNAmax(c_across(matches(paste0("clspub_", har_cols)))),
clspub_vul = mean_oneNAmax(c_across(matches(paste0("clspub_", vul_cols)))),
clspub_dis = mean_oneNAmax(c_across(matches(paste0("clspub_", dis_cols)))),
clspub_neg = mean_oneNAmax(c_across(matches(paste0("clspub_", neg_cols)))),
dispri_har = mean_oneNAmax(c_across(matches(paste0("dispri_", har_cols)))),
 54

dispri_vul = mean_oneNAmax(c_across(matches(paste0("dispri_", vul_cols)))),

dispri_dis = mean_oneNAmax(c_across(matches(paste0("dispri_", dis_cols)))),
dispri_neg = mean_oneNAmax(c_across(matches(paste0("dispri_", neg_cols)))),
dispub_har = mean_oneNAmax(c_across(matches(paste0("dispub_", har_cols)))),
dispub_vul = mean_oneNAmax(c_across(matches(paste0("dispub_", vul_cols)))),
dispub_dis = mean_oneNAmax(c_across(matches(paste0("dispub_", dis_cols)))),
dispub_neg = mean_oneNAmax(c_across(matches(paste0("dispub_", neg_cols))))