The Journal of Systems and Software 219 (2025) 112225

Contents lists available at ScienceDirect

The Journal of Systems & Software

journal homepage: www.elsevier.com/locate/jss

Assessing gender bias in the software used in computer science and software
engineering education✩
Lyndsey O’Brien a , Tanjila Kanij b ,∗, John Grundy a
a
HumaniSE Lab, Faculty of Information Technology, Monash University, Victoria, Australia
b
Department of Computing Technologies, Swinburne University of technology, Australia

ARTICLE INFO ABSTRACT

Keywords: Women are underrepresented in Computer Science (CS)/ Software Engineering (SE) and other technology
Gender inclusivity related degrees. As undergraduates, they are also less likely to persist with CS/SE studies than men enrolled in
Education those same courses. Gender correlated differences in personal characteristics, behaviour, and preferences mean
Persona
that course design decisions may introduce unintended bias. To address this issue, we drew inspiration from the
GenderMag
GenderMag method. GenderMag uses personas with evidence-based gender differences in problem-solving traits
Computer science
Software engineering
to detect usability issues in software. In this paper we investigate the personal qualities of CS and SE students,
Software usability and how these influence their CS/SE learning journey. A series of persona development workshops were held
to gather an extensive and unique qualitative dataset capturing the prior experiences, preferences, learning
styles, motivations, goals, frustrations, and constraints of CS/SE students. Gender differences were used to
construct preliminary male and female student personas. These personas were used in cognitive walkthroughs
of software applications commonly used in education, and their performance compared to GenderMag’s Tim
and Abi. While the student personas were less effective and lacked specificity compared to Abi, they were
able to identify issues not detectable with GenderMag. Furthermore, the findings show the utility of persona
development workshops as a data collection method and introduce a comprehensive list of CS/SE student
qualities that may inspire future investigations.

1. Introduction of the Chief Scientist, 2020). However, CS itself is a wide area of study
that also encompasses subjects like artificial intelligence, databases,
The traditional belief of ‘‘male being strong and rational’’ and and software engineering (Yatsko and Suslow, 2016), and it is often
‘‘female being weak and emotional’’ has led to a gender disparity within a pathway for students into these more specific computing disciplines.
Science, Technology, Engineering and Mathematics (STEM) (Keller, While lower enrolment rates are a major reason for women’s under-
1995). There have, however, been advances towards gender parity representation in CS, they are not the only reason. In addition to
within many STEM fields (Saad, 2017). These changes are also re- lower enrolment rates, undergraduate women are also less likely to
alised in Australia, (15% enrolments in 2015 compared to 19% in complete their CS studies than men enrolled in these courses (De-
2021 in Engineering degrees) (Department of Industry, Science and partment of Industry, Science and Resources, 2022), a troubling trend
Resources, 2023). Enrolment to computing degrees has also seen some that has also been observed in both the United States (Barker et al.,
increment - 16% in 2015 compared to 21% in 2015 (Department of 2009) and Europe (Höhne and Zander, 2019). It has been suggested
Industry, Science and Resources, 2023). However, the figures indicate that this disparity is due, at least in part, to the presence of gender
that women continue to comprise only one-fifth of enrolments in bias within CS courses (Medel and Pournaghshband, 2017; Henwood,
computing related degrees compared to men (Department of Industry, 1999). While direct discrimination is an issue, comparison with STEM
Science and Resources, 2023). This figure comes from the STEM Equity fields where women are well represented reveals that alone it cannot
Monitor, an annual report on the enrolment and course completion explain their under-representation in CS (Cheryan et al., 2017). Re-
rates of women in Australian STEM degrees. Computer science (CS), search into this gender imbalance has also found the presence of mascu-
Information Systems (IS), and Information Technology (IT) degrees are line cultures (Cheryan et al., 2017), negative assumptions about the ca-
all broadly grouped as an information technology qualification (Office pabilities of female students (Cohoon, 2007; Moss-Racusin et al., 2012),

✩ Editor: Professor Laurence Duchien.

∗ Corresponding author.
E-mail address: tkanij@swin.edu.au (T. Kanij).

https://doi.org/10.1016/j.jss.2024.112225
Received 8 November 2023; Received in revised form 28 July 2024; Accepted 18 September 2024
Available online 27 September 2024
0164-1212/Crown Copyright © 2024 Published by Elsevier Inc. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-
nc/4.0/).
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

and biased course materials (Medel and Pournaghshband, 2017). How- 2. Literature review
ever, women’s sense of belonging in CS may also be impacted by
more subtle cues in the learning environment (Metaxa-Kakavouli et al., Despite the increased participation of women in many STEM fields,
2018). CS/SE and other technology related careers continue to be male dom-
As stated, Software Engineering (SE) is a domain of Computer inated (Murphy et al., 2019a; Department of Industry, Science and
Science (CS) and is used alternatively in the literature. It is difficult Resources, 2021). Efforts to address this issue are crucial, given ev-
to differentiate the findings in the existing literature between CS and idence of enhanced performance and innovation in gender diverse
SE education. Moreover, CS is considered a pathway for many SE organisations (Dai et al., 2019; Østergaard et al., 2011). In software
students. For these reasons, we refer to computer science and software development in particular, gender diverse teams have been associ-
engineering teaching and learning environment as CS/SE in this article.
ated with greater creativity and sounder decision making (Kohl and
The decision to undertake a CS/SE course may be influenced by
Prikladnicki, 2022a). The shortage of women in these fields stems in
gender correlated factors including personality, values, interests, and
part from their underrepresentation and lower course completion rates
computer self-efficacy (Beyer, 2014). Even in the absence of deliberate
in technology related degrees (Department of Industry, Science and
discrimination or unfairness, failing to account for gender differences
Resources, 2023, 2022). The broad themes of research on gender bias
in these traits may make CS/SE courses less appealing to women and
introduce implicit bias. While learning and interacting with new soft- within CS/SE education is divided into investigating reasons and form-
ware applications is an integral part of any CS/SE program, it is also an ing recommendations, gender bias within teamwork, gender expression
area where there are well-established gender differences (Burnett et al., and difference, self efficacy of students, and gender bias in CS/SE
2016b). These are often not considered by software developers who software. The following subsections present some key related research
tend to design applications to suit their own needs (Nunes et al., 2021), under each of these broad themes. It is noted that, there is a great body
and this may negatively impact the experience of female students. of research on gender in software engineering, Rodríguez-Pérez et al.
Gender Inclusiveness Magnifier (GenderMag) is a highly effective (2021), Silveira and Prikladnicki (2019), Murphy et al. (2019b), Kohl
tool for the detection of gender bias within software that is based and Prikladnicki (2022b), however, since our focus is on education, we
on five ‘facets’, or research supported differences in cognitive styles present literature focusing on gender in the CS/SE education context.
of problem-solving (Burnett et al., 2016b; Vorvoreanu et al., 2019).
These facets are: information processing style, learning style, computer 2.1. Gender bias in CS/SE education
self-efficacy, attitude towards risk, and motivations. The GenderMag
method employs personas constructed using these facets to identify A recent mapping of literature on ‘‘female inclusiveness’’ in CS/SE
inclusivity issues in software applications (Burnett et al., 2016b). Per- education reveals that a gender imbalance is still prevalent in this
sonas are fictitious but representative personal profiles that support domain, although it is a knowledge and skill based profession, rather
the consideration of different user groups (Adlin and Pruitt, 2010; Mi- than a physical or labour intensive one (Kovaleva et al., 2024). There
askiewicz and Kozar, 2011). GenderMag follows the InclusiveMag ap- has been an increase in research on this topic since 2015, with a
proach, a three-stage method for the development of personas tailored
decline in 2020–21 with a possible reason of Covid-19 pandemic. Some
to a specific diversity dimension (Mendez et al., 2019).
significant findings of the systematic mapping study were - (1) a global
While the GenderMag personas have already been applied in an ed-
imbalance in research on this topic, with the USA leading most of the
ucation context (Shekhar and Marsden, 2018; Chatterjee et al., 2022),
research, (2) less research focusing on the persistence of women in the
the facets they employ are based on evidence of gender differences
in the general population. CS/SE students are a unique group whose domain, and (3) a significant body of recommendations for educators.
gender expression may not be entirely consistent with this framework. Research into gender bias in education has traditionally investigated
Additionally, previous work looking at education software examined acts of direct discrimination (Cheryan et al., 2017), implicit beliefs
only a single Learning Management System (LMS) (Shekhar and Mars- that can lead some teachers to underestimate the competency of their
den, 2018). Using the InclusiveMag method it may be possible to female students (Cohoon, 2007; Moss-Racusin et al., 2012), or the
develop representations of students that are more effective in detecting use of gendered language or materials that reinforce negative stereo-
gender bias in CS courses, and that can be applied to a broader range types and create an unwelcoming learning environment (Medel and
of education software. Pournaghshband, 2017).
The present study investigated the characteristics of students that Medel and Pournaghshband Medel and Pournaghshband (2017)
affect their experience in CS/SE education, as well as the extent to examined the instructional materials used in CS education to iden-
which their personal problem-solving traits align with the GenderMag tify gender differences in the representations, imagery, and language
framework. A series of workshops were conducted where students were used. They found case study examples of gender bias including the
asked to create and test personas representing themselves and their disproportionate assignment of negative roles to characters with female
peers, and complete a survey assessing their personal cognitive styles. names, persistence of a playboy centrefold picture as the standard stock
Thematic analysis of the deconstructed workshop data was used to image representing image processing, and perpetuation of stereotypes
identify gender related traits and personal qualities. These traits were through the misuse of gendered pronouns. While their examples pro-
used to construct gendered CS/SE student personas that were employed vide a compelling negative portrait of the portrayal of women within
in cognitive walkthroughs of software applications commonly used in CS education materials, only a limited number were included. It has
CS/SE education. Their performance in the detection of gender bias was
been demonstrated, however, that the creation of an unwelcoming
then compared to the existing GenderMag personas.
learning environment can negatively affect the experience of women
The key contributions of this paper are: (1) A comprehensive list of
in CS (Metaxa-Kakavouli et al., 2018).
traits and personal qualities that affect the study experience of CS/SE
Metaxa-Kakavouli et al.’s (2018) study on the relationship between
students, (2) evidence of gender differences in the CS/SE student pop-
ulation that are not captured in the existing GenderMag personas, and website design and ambient belonging tested the hypothesis that a
(3) an efficient and effective approach for collecting a rich qualitative perception that they do not belong decreases women’s willingness to
dataset to use in the scoping stage of the InclusiveMag method. enrol in CS courses. When presented with an introductory CS course
The rest of the article is organised as follows; Section 2 presents webpage designed to evoke masculine stereotypes, potential female
a review of the relevant literature, Section 3 describes the problem students had lower confidence in their abilities, less sense of belonging,
statement and research questions under investigation, Section 4 elabo- and a reduced intention to enrol in the course. While it is difficult
rates research methodology, Section 5 illustrates our findings, Section 6 to imagine that the choice of colours and pictures can have such
discusses these findings, Section 7 describes some of the limitations of a profound influence, the effect of masculine imagery on women’s
our research, and finally Section 8 concludes the article. sense of belonging has also been observed offline in the design of CS

2
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

classrooms (Cheryan et al., 2009). Unfortunately, the implicit feeling of low prediction models (Al-Taharwa, 2020). In large software engineer-
these students that they do not belong may be shared by some of their ing projects, Laura and Marie found that female students were less
teachers (Cohoon, 2007). likely to assign themselves to technical roles and consequently received
In an extensive survey across CS and SE programs in the US, Cohoon lower peer review scores for their contributions to team projects (Heels
(2007) found a strong link between faculty attitudes towards diversity and Devlin, 2019).
and women’s confidence in their abilities and comfort asking questions Apart from team projects, gender has also been studied in pair
in class. Surprisingly, half of professors in some faculties believed that programming settings within SE education. Choi et al. found that in
efforts to increase the number of women would lead to a less capable terms of quality of programming, there was no significant difference
student cohort. While the study also showed that, even in 2007, this between same and mixed gender pairs, however communication and
belief was only held by a minority of teachers, negative assumptions compatibility was better in the same gender pairs (Choi, 2015). Simi-
about women’s competence have also been revealed in more recent larly, Gómez et al. did not find any significant difference in productivity
research within the broader STEM sector, and these assumptions may between same and mixed gender pairs (Gómez et al., 2017). Productiv-
have harmful implications for their career prospects (Moss-Racusin ity, however, was determined by lines of code written within a fixed
et al., 2012). timeframe. The authors did find that productivity varied a lot in mixed
When presenting science faculty professors with the applications pair groups. In a randomised controlled trial with a large sample of
of otherwise identical students randomly assigned male or female undergraduate computer science students, Jarrat et al. investigated the
names, Moss-Racusin et al. (2012) found that students believed to be influence of gender of the partner in a pair and found that having
female were considered both less competent and less hireable than a female partner was associated with higher confidence in the out-
male students. This bias was present regardless of the professor’s gender come (Jarratt et al., 2019). The authors also noted that female students
and resulted in them suggesting lower starting salaries and less career had lower perceptions of their competence relative to their partners.
mentoring for students with feminine names. While not examining
assumptions about competence directly, Wang and Redmiles (2019) 2.3. Gender expression and CS/SE education
found a similar discrepancy in the hiring preferences of software engi-
neers when presented with equally qualified men and women seeking
While the term gender has often been used interchangeably with bi-
leadership positions. Whether or not this sort of incidental discrimi-
ological sex in academic research and viewed binarily (Anna Lindqvist
nation can explain the underrepresentation of women in some STEM
and Renström, 2021), this fails to account for the true variability in
fields has been investigated by comparing them to fields with a more
human gender expression and may obscure interesting findings related
even gender balance (Cheryan et al., 2017).
to people who do not exist at either end of the masculine/feminine
From a series of studies with children and adolescents, Master et al.
spectrum (Bittner and Goodyear-Grant, 2017). The present paper is
concluded that beliefs that ‘‘girls are less interested in computing and
concerned with gender expression, which relates both to how the
engineering’’ are formed early and cause gender disparity in later
individual sees themselves along that spectrum and their presentation
stages (Master et al., 2021). While it might be presumed that the
and behaviour in relation to social norms (Anna Lindqvist and Ren-
presence of bias and discrimination provides a clear explanation for the
ström, 2021). While no person will demonstrate all the characteristics
underrepresentation of women in CS courses, comparison with other
typically associated with either masculinity or femininity, there are dif-
STEM fields reveals that the situation is more complex. While inves-
ferences in how individuals experience CS studies that can be correlated
tigating potential reasons for the differing gender distributions across
with their gender identity (Beyer, 2014).
STEM fields, Cheryan et al. (2017) found no evidence that formal gen-
In a survey of first year students in the US, Beyer (2014) found
der discrimination is any more prevalent in CS than in courses where
significant gender differences in student’s confidence in their computer
women are well represented. The authors conducted a broad literature
skills, interest in pursuing CS studies, and the values and motivations
review to identify factors that influence the participation of women in
that shaped their career goals. Men were actually slightly more likely
STEM courses before exploring the contribution of those factors to the
to hold negative beliefs about computer use, but the effect size was
discrepancies in women’s representation across disciplines. Masculine
small, and they were still more likely to undertake a CS course than
culture and stereotypes, less experience with CS in early education, and
the women surveyed. While there was no relationship between gender
lower self-efficacy were all identified as potential contributing factors.
In contrast, while discrimination was present, it could not explain and academic performance, lower confidence in their computer skills
the greater underrepresentation of women in CS courses. It should be was evident even among the women who chose to undertake CS studies.
noted that the authors were looking at enrolment rates only and did
not examine whether the discrimination identified negatively impacted 2.4. Gender difference and self efficacy of students
educational experiences or student retention rates. Nevertheless, their
findings suggest that there may be alternative explanations for the low Gender differences in CS student’s sense of self-efficacy are well
participation of women in CS. documented in the literature (Beyer, 2014; Cheryan et al., 2017).
While investigating the effects of self-regulated learning techniques
2.2. Gender bias in CS/SE teamwork in a beginner CS course, Lishinski et al. (2016) also found that self-
efficacy is positively correlated with academic performance. Perhaps
Investigating role selection in a final year student team projects, more interestingly however, they observed gender differences in the
Nguyen-Duc and Jaccheri found that female students participated in malleability of the student’s sense of self-efficacy over time. While
project management and requirement engineering tasks more than ar- men’s view of their abilities continued to change in response to feed-
chitecture design or Scrum methods (Nguyen-Duc and Jaccheri, 2023). back, women’s self-efficacy was fixed earlier in the course, leaving
The authors also observed that the female students tended to engage them more affected by early challenges. Given the relationship between
in more light weight programming tasks over complex technical roles. self-efficacy and performance outcomes, this study highlights possible
However, they noted that these task assignments were more influenced issues with the presence of steep learning curves at the beginning of
by knowledge, and the previous experience and commitment of team CS courses. It suggests that gender inclusive education should avoid
members, than the gender identity of the students. Al-Taharwa, on the overwhelming students early when women may be disproportionately
other hand, experimented with student teams with female leadership affected by negative experiences and feedback. One of the challenges
and regular membership of the team, and concluded that female-led to this is the necessity of introducing new and potentially challenging
teams followed good software engineering practices, however, achieved software programs to students early in their CS studies.

3
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

2.5. Gender bias in software in English over their local language, the incorporation of cultural icons
and motifs improved the experience of many users. Feedback comments
To select software that meets the needs of all students, it is impor- from 14% of the participants indicated that seeing elements of their
tant to recognise that most software is not designed with inclusivity own culture made them feel a greater sense of belonging. This provides
in mind. In a systematic mapping study reviewing current approaches an interesting contrast to the work of Metaxa-Kakavouli et al. with
for fostering gender inclusivity in software design, Nunes et al. (2021) a masculine website design (Metaxa-Kakavouli et al., 2018). Building
found three recurring issues- (1) creators tend to assume that software, inclusivity into a user interface could involve the addition of elements
and the common methods for developing it, are gender neutral even that students can relate to, not just the removal of imagery that could
where there is no gender diversity in the development team, (2) gender make them feel excluded.
is looked at binarily and without consideration of how to address any In addition to LMSs, CS/SE students are exposed to a broad range
gender bias that is identified, and (3) there is a lack of information and of software packages including programming environments, database
guiding methods available to help teams navigate gender inclusivity management systems, and project management software. Selecting
issues. In view of these findings, it is unsurprising that the authors which software to use in a CS course is a unique challenge as technology
also found that developers tend to design according to their own is constantly changing, and the tools used are often the same as those
preferences and requirements, failing to account for the needs of a used in industry (Parker, 2010). While gender differences in software
diverse potential user base. This bias may create usability issues that use have been thoroughly investigated in the literature (Schlesinger
disproportionately affect users who are demographically different from et al., 2017), as evidenced above, they have often been overlooked
the developers. If these inclusivity issues are to be addressed, they must in the selection and design of education software. As a further ex-
first be detected. This can be done without the involvement of end ample, Parker (2010) published a guide for educators to use when
users using expert-based testing methods like cognitive walkthroughs selecting software for use in information technology courses. While the
or heuristic evaluations (Jaspers, 2009). author included ease of use and learnability as factors for consideration,
Nielsen (1993) describes five elements of software usability: learn- there was no discussion of the differing needs of individual students.
ability, efficiency, memorability, low error rate, and subjective satisfac-
tion. Subjective satisfaction is an aspect where individual differences 2.7. The GenderMag approach
in personal characteristics, some of which will be gender correlated,
GenderMag is a research supported tool for identifying gender
could be particularly impactful. The Cognitive walkthrough is a us-
bias in software using personas and a custom cognitive walkthrough
ability inspection method able to capture this internal experience of
method (Burnett et al., 2016b). Burnett et al. assembled five facets
users (Mahatody et al., 2010), that has been found to be particularly
(information processing style, learning style, computer self-efficacy,
sensitive to issues related to learnability (Farzandipour et al., 2022;
attitude towards risk, and motivations) of software use that are known
Khajouei et al., 2016), another crucial element in the education context.
to correlate with the user’s gender when developing a new tool for
the detection of gender bias. Women are generally more risk averse,
2.6. Education software
more likely to adopt a comprehensive information processing style,
and less likely to tinker with new software features. As with CS/SE
While learning and using new software systems is an integral part
education, they also tend to have lower self-efficacy than men. These
of any CS/SE program, much of the work related to their design and
facets are represented in the GenderMag personas, ‘Tim’, ‘Pat’ and ‘Abi’.
selection has not considered student diversity. While Oliwa (2021)
A persona is a descriptive profile of a fictional person used to represent
sought the opinions of students, teachers, and administrative staff on
a class of potential users while conducting usability testing (Adlin
the required functions of an online learning platform, they did not
and Pruitt, 2010). These personas are used in a systematic cognitive
consider demographic differences within those groups. In a case study
walkthrough process to identify usability problems in problem-solving
looking at the needs and expectations of students, Şahin and Yur- software interfaces.
dugül (2022) found that a competitive environment, including elements The GenderMag method has been shown to be highly accurate in
like leaderboards, is a desirable feature of online learning platforms. its identification of software usability issues (Burnett et al., 2016b),
However, their study also examined the needs of students collectively, and applying it to increase software inclusivity can lead to design
and did not consider how personal characteristics like a user’s age, improvements that benefit all users, not just women (Vorvoreanu et al.,
cultural background, or gender could affect the suitability of certain 2019). Shekhar and Marsden (2018) employed the GenderMag method
features. This is problematic considering there is research that suggests to investigate gender bias in LMSs with information technology and
more competitive settings may actually decrease the educational per- software engineering students and professionals. While they modified
formance of women (Ors et al., 2013). While Lim et al. (2020) found the demographic traits of the GenderMag personas, they used the pre-
gender differences in the features used and frequency of interactions set facet values. Their study showed that the method was effective
with a LMS when surveying university students, the reasons for those for detecting usability issues, and that the female persona ‘Abi’, in
differences and how they might be mitigated through better software particular, could help participants consider the needs of users different
design, were left unexamined. from themselves.
In contrast, while not directly considering demographic differ- In the education context, GenderMag has already been used in the
ences, Kolekar et al. (2018), proposed an addition to an existing LMS assessment of both online courseware (Chatterjee et al., 2022) and a
that would allow it to adapt to student’s learning styles. Based on usage LMS (Shekhar and Marsden, 2018), and there is growing support for
data, students are categorised into one of eight groups. The system then the use of personas and other user experience (UX) tools in STEM
presents different user interfaces and learning materials based on the course development (Minichiello et al., 2018). However, GenderMag
presumed needs and preferences of that user group. While the authors inquiries are limited to five facets based on gender differences observed
found that the system improved the performance of students, their in the general population (Burnett et al., 2016b). While limiting the
sample was small, and detailed results, including the significance of the number and nature of facets in this way makes the method more fea-
difference found, were not reported in the paper. That said, it provides sible (Mendez et al., 2019), it also narrows its scope. At present, there
an interesting example of how education software could be made more have been no investigations into whether these facets can accurately
inclusive. capture the experiences of CS/SE students when learning new software.
In a similar small study, Yalamu et al. (2020) created a LMS CS/SE students are a unique group. They show notable personality
prototype that incorporated aspects of Papua New Guinea indigenous differences even when compared with students in other STEM disci-
culture. While most of their participants preferred to view the interface plines (Benest et al., 2003; Larson et al., 2010). Beyer (2014) found

4
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Fig. 1. Methodology overview.

that personal qualities like low conscientiousness and low openness • RQ3: Can personas developed by students in collaborative work-
to experience are predictors of whether a student will take a CS/SE shops be adapted to detect gender related usability issues in
course regardless of gender. Low openness refers to avoiding changes or software?
resisting new ideas, and low conscientiousness refers to not preferring • RQ3: Are custom student personas more effective at detecting
structure, schedule or organisation (McCrae et al., 2010). While there usability issues in CS/SE education software than the GenderMag
are gender correlated differences that affect the experience of CS/SE personas?
students (Beyer, 2014; Lau and Yuen, 2010), like personality, these may
not entirely mirror traits seen in the general population. This means 4. Methodology
that there may be relevant usability issues that are not detectable with
the existing GenderMag facets. The study was conducted in three phases (Fig. 2). In the first,
a series of persona development workshops were held to investigate
Mendez et al. (2019) introduced InclusiveMag as a method for
the personal traits of CS/SE students (RQ1) and assess their problem-
developing new inclusivity tools, similar to GenderMag, that support
solving traits under the GenderMag framework (RQ2). The second
underserved populations. Their method has three stages: scope, derive,
phase was a thematic analysis of the workshop data to see if gendered
and apply. In the first stage researchers set the domain and conduct
CS/SE student personas could be extracted from the participant re-
research to identify relevant facets. Stages two and three involve the
sponses (RQ3). In the final phase these personas were used in a series
creation and use of personas based on those facets. In the present paper of cognitive walkthroughs of education software, and their performance
we take a narrow focus, examining gender differences found specifically compared to GenderMag’s Tim and Abi (RQ4). (see Fig. 1).
within the CS/SE student population.
4.1. Recruitment
3. Problem analysis
Monash University students over the age of 18 currently studying
While the utility of the GenderMag personas has been thoroughly a CS/SE related course were recruited through convenience sampling.
This means that the participant recruitment was based primarily on
investigated (Vorvoreanu et al., 2019; Chatterjee et al., 2022; Burnett
availability and interest over a more purposive approach (Denscombe,
et al., 2016a), the applicability of their facets to CS/SE students as
2010). The study was advertised through social media, student clubs
a distinct population has not. To directly address the lower course
and societies, and leaflets distributed around the campus. Potential
completion rates of female students, it is necessary to investigate the
participants were also invited to participate in person or contacted via
qualities and experiences that make this group unique. Employing
email. While convenience sampling reduces the generalisability of the
personas that capture these qualities in cognitive walkthroughs of
findings (Etikan et al., 2016), the limited time available, and scheduling
education software may provide a more accurate depiction of the requirements of the study design, made it the most appropriate option
difficulties and frustrations faced by these students. Furthermore, prior for the project.
work in education was limited to LMSs (Shekhar and Marsden, 2018) or
online courseware (Chatterjee et al., 2022). Consideration of a broader 4.2. Persona creation workshops
range of applications, including those developed for use in industry,
will improve our understanding of the usability issues encountered by Students were invited to attend a one-hour workshop designed for
CS/SE students during their degree. groups of three to five participants. Although multiple participants
The study has been designed to address the following research attended the workshops at the same time, they were not particularly
questions: collaborative in nature. A workshop approach was adopted to allow
for the collection of a larger number of participant responses within the
• RQ1: What are the facets of CS/SE students that affect the way time constraints of the project and collate a more representative data
they experience CS/SE education? set (Denscombe, 2010). Allowing each participant to create individual
• RQ2: Do CS/SE students express gender differences in problem responses ensured that the perspective of less extroverted students
solving consistent with the GenderMag framework? would still be captured in our results.

5
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Table 1
Summary of participant enrolments.
Course AQF level No. participants
Computer Science Bachelor 5
Data Science Masters 2
Information Technology Bachelor 4
Information Technology Masters 1
Information Technology Unknown 1
Other Bachelor 1 Engineering (Specialising in Software)
1 Business (Currently taking an IT subject)

The workshops were divided into three parts. In the first part, most important. Most participants chose to create a single persona
the participants created one or more CS/SE student personas. This based on themselves.
approach was selected over a traditional interview method to allow Pilot Workshop Outcomes
persona development to be guided by CS/SE students themselves. In a pilot workshop, two participants wrote a group description
These tasks made no reference to the GenderMag facets and gave in their response instead of an individual persona. To limit this in
participants significant freedom over the content and structure of their future workshops, age, gender, skills, and experience were added to
answers. This limited the influence of the researcher’s familiarity with the instructions as examples of ‘‘Who are they?’’. This encouraged the
GenderMag on participant responses, and increased the likelihood of participants to think of the personas as individuals and provide more
identifying novel insights into gender differences that are not captured personalised responses.
under the GenderMag framework. In the second task, participants were
instructed to investigate an online tutorial from the perspective of
the personas they created. Finally, participants were introduced to 4.2.3. Task 2:persona testing
GenderMag to complete their third task. An online survey rating their Participants selected one of two introductory courses in MATLAB
personal cognitive styles according to the five facets. or SQL for the testing task. They were asked to reflect on the following
questions as they investigated the tutorial:
4.2.1. Participants
A series of eight workshops were held with groups of one to four What is your persona thinking as they go through the steps?
participants. While the initial intention was to conduct all workshops
Do they get stuck at all?
with groups of three to five, difficulties with scheduling and participant
retention meant that four were held with only a single participant. A Is there anything that bothers them?
total of 15 participants from diverse cultural backgrounds attended. Of
those participants, 14 disclosed their gender (seven male, seven female) Participants were told they would use their personas for this task
and most were aged 18–21. A summary of the participant’s course before they began creating them. This was to provide context for
enrolments can be seen in Table 1, with the majority (73%) undertaking the participants and improve the relevance of their responses to their
degrees in CS/SE or Information Technology. All participants were educational experiences and use of software. MATLAB and SQL were
on-campus students, with 14 of the 15 participants studying full-time. selected because they are widely used for different units within SE.
However, since not all students complete all units, and some partici-
4.2.2. Task 1:student personas pants may not be familiar with these languages, the courses selected
Participants were asked to create one or more CS/SE student per- started at an introductory level.
sonas based on themselves or their peers. Human factors, commonly
present in personas across different domains and identified by Karolita
4.2.4. Task 3:GenderMag facet survey
et al. (2023) in their persona taxonomy, were used as examples to
Their last task was to complete a survey, available through Open
encourage the students to create realistic and detailed student de-
scriptions. These factors included personal characteristics, skills and Educational Resources Commons (Open Educational Resources (OER)
experience, and group characteristics. Commons, 2021), where they rated their cognitive styles across the
Participants were provided with the following prompts when begin- five GenderMag facets. Scores were used to evaluate their similarity
ning the task: to the three GenderMag personas. The ‘Tim’ persona has facet values
most commonly seen in men, ‘Abi’ those seen in women, and ‘Pat’ an
Who are they? amalgamation of the two (Burnett et al., 2016b). In general, people not
showing extreme values are referred to as ‘‘Pat’’.
What are their goals?
The five facets are information processing style, learning style,
What motivates them? computer self-efficacy, attitude towards risk, and motivations (Fig. 2).
What are their constraints? Individual facet scores range from −10 to +10. A very negative score on
a facet indicates a cognitive style similar to the Tim persona, while very
Is there anything that could make learning or using new software difficult
high scores suggest similarities with Abi. Total scores range between
for them?
−50 and +50. Average scores by gender were compared to evaluate
They were given freedom over the format of their responses, as well the extent to which the sample’s problem-solving traits are consistent
as whether they included a picture, but were asked to add a one-line with the GenderMag framework. Since the authors did not develop
summary at the end. Karolita et al. (2023) found that ‘taglines’, or one- this survey, adopting it from the Open Educational Resources Com-
line statements, are a commonly included feature of personas used in mons (Open Educational Resources (OER) Commons, 2021), we used
the software development domain. Summaries were requested in this the same scoring mechanism (resulting in positive (+) and negative (-))
case to gain insight into which characteristics the students considered provided by the source.

6
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Fig. 2. The five GenderMag facets. .

Source: Adapted from the Cognitive Styles Survey (Open Educational Resources
(OER) Commons, 2021)

4.3. Analysis code list. This refined code list was used to recode the complete dataset
before further analysis. For example the female persona description
4.3.1. Deconstruction and thematic analysis of participant responses - ‘‘work in (G)oogle’’, male persona attribute - ‘‘have a successful
A thematic analysis of the participant’s written responses was used career in AI’’ and an unspecified persona attribute ‘‘getting into... IT
to identify and categorise personal qualities that affect student’s experi- companies’’ were coded under ‘‘opportunity’’ which was merged with
ences in CS/SE education. Thematic analysis is a flexible approach that other codes such as ‘‘industry’’, ‘‘lifestyle’’ and ‘‘personal fulfillment’’ to
aids in the discovery and reporting of relevant patterns within a quali- form the group ‘‘Career’’, indicated in Fig. 4.
tative dataset (Clarke and Braun, 2017; Braun and Clarke, 2006). Braun When reviewing the use of UX design methods in STEM education
and Clarke (2006) describe a six-phase model of thematic analysis that research, Minichiello et al. (2018) found that in most papers, manual
begins with the researcher familiarising themselves with the data before qualitative clustering methods like affinity diagramming were used
developing the initial codes. to organise data during the persona development process. Affinity
Familiarisation and Initial Coding
diagramming is also strongly recommended by Adlin and Pruitt in their
Memos were taken to record the researcher’s thoughts and rea-
guide to building personas because of its speed and ease of use (Adlin
soning as initial impressions of the individual participants, created
and Pruitt, 2010).
personas, and complete dataset were developed. Preliminary codes
were appended to each passage of text produced during the persona
and test activities. In light of the additional context provided by the 4.3.2. Persona reconstruction
association between the two tasks, and records of participant comments For each attribute and its associated codes (‘student qualities’),
within the workshops, a latent coding process was chosen over a relevant text passages in the dataset were collected and grouped by
semantic approach. Latent coding attempts to ascertain meaning in gender. Comparisons were made based on the presence/absence of
data that goes beyond literal description or summary of participant traits, the student qualities represented, and the language used by the
words (Braun and Clarke, 2006). participants. Notable differences were used to create preliminary CS/SE
Searching for Patterns student personas. Since our target is to understand the female and
The initial codes were categorised based on emerging patterns and male facets displayed by CS/SE students to detect potential bias within
the workshop prompts. The data was restructured using the recurring education software, we developed ‘‘Abi’’ and ‘‘Tim’’ personas only.
attributes: gender, age, culture, group identity, skills/experience, back-
ground, preferences, learning style, motivations, goals, frustrations, and
4.4. Cognitive walkthroughs
constraints. Fig. 3 presents a snippet of coding. These attributes were
used to create affinity diagrams grouping and connecting related text
segments. Associated and heavily overlapping attributes, like motiva- While Minichiello et al. (2018) found that assessment of new per-
tions and goals, were merged during this process (Fig. 4). Colours sonas is not always reported in usability research, they nevertheless de-
were used to denote the gender of the persona each extract was taken scribed it as the important last step in the persona development process.
from (blue, green and yellow were used to denote female, male and The preliminary CS/SE student personas were assessed by employing
unspecified or non-binary, respectively). These diagrams were used to them in cognitive walkthroughs and comparing their performance to
identify recurring traits within the attributes and iteratively refine the GenderMag’s Tim and Abi.

7
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Fig. 3. Example of coding.

Four desktop or web-based software applications were chosen for persona would have formed the subgoal and whether the persona will
evaluation from the following categories: database management, pro- know what they need to do at each step (Burnett et al., 2016b). While
cedural programming, project management and learning management. these questions are highly relevant when assessing usability in industry,
These applications were selected because they are used for multiple they make less sense in an education context where students are often
units taught in CS/SE. The selected applications cover a breadth of provided with very specific instructions. In light of this, the GenderMag
application types in CS, especially majoring SE students will encounter walkthrough was adapted to evaluate tasks appropriate for students
over the course of their degree. The selection, outlined in Table 2, learning a new software. The structure of this walkthrough is presented
includes an application designed for use in education, and others drawn below:
from industry. It includes both text-based and more visual tools, as
well as variety in the complexity of the user interfaces. This was to For each instruction/set of instructions consider:
expand upon the prior work examining a single LMS (Shekhar and
Marsden, 2018), and explore the experience of student users when 1. Would the persona have trouble understanding this instruction or
learning diverse application types. For each application, two multi- step?
stage tasks were selected to investigate its learnability and ease of
2. Would the persona encounter any frustrations or difficulties when
use.
taking this step?
The GenderMag cognitive walkthrough requires the evaluator to
answer a series of questions for each subgoal and subsequent actions 3. Would the persona have any trouble knowing that they have
a persona will take (Burnett et al., 2016b). This includes whether the completed this step successfully?

8
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Fig. 4. Affinity diagram: motivations and goals.

Table 2
Applications selected for cognitive walkthroughs.
Application Category Domain Presentation Complexity
SQL Developer Database Management Industry Text-Based High
BlueJ Procedural Programming Education Visual Low
IntelliJ Procedural Programming Industry Text-Based High
Trello Project Management Industry Visual Low
Canvas Learning Management Education Visual Low

The second question aims to capture the usability element of subjective The testing task responses provided examples of behaviour, preferences
satisfaction as defined by Nielsen (1993). The walkthroughs looked not and frustrations that supplemented the persona descriptions during the
only at whether the fictional student could successfully complete the analysis. Hereafter a ‘persona’ created by the participants refers to both
tasks, but also whether they would find the application pleasant to use. the task 1 description and content of any corresponding test task.

5. Results 5.1.2. GenderMag scale

All 15 participants rated themselves using the GenderMag Cognitive
5.1. Workshop responses Styles survey. Their total scores were all in the −25 to +25 range
indicating that none of the participants were particularly Tim or Abi
5.1.1. Personas like in their gender presentation. As the participant scores were not
A total of 19 personas were created. Eighteen were created by normally distributed, and only a small sample size was available,
individual participants, and one was created by two female participants Mann–Whitney U tests were used to compare the scores of the male
in collaboration. Of the 19 personas, 17 included gender. Eight were and female participants. The results found no significant differences
male, eight were female, and one was non-binary. Two were members
between the total scores (U = 18, n1 = n2, = 7, p > 0.05 two-tailed),
of the LGBTQ+ community. A summary of the participants and their
or the scores of any individual facets. A summary of the mean values
personas can be seen in Table 3. Fifteen personas included age (M =
by gender can be seen in Tables 4 and 5, below. Very positive values
22.67, SD = 4.20), and seven included cultural background. In each
indicate trait expressions more commonly seen in women, and very
case where age and cultural background were included, these matched
negative values those seen in men.
the personal demographics of the participant who created the persona.
All but two personas included a one-line summary. Demographic
information, most commonly gender, was included in over half of these. 5.2. Thematic analysis
Of the seven male summaries, four referred to personal constraints
or difficulties that affected their studies, and six included goals or The analysis revealed 18 traits that capture the full range of qualities
motivations. In contrast, goals or motivations were only included in present in the written responses. These are outlined in Table 6. Traits or
one female summary, while personal constraints were included in five. qualities where the participants in our sample displayed notable gender
There were 17 responses to the testing task. Two male participants differences are denoted with ‘‘(*f)’’ and ‘‘(*m)’’ signs - to indicate
chose to go through the coding tutorials with two different personas. majority appearance in female and male personas, respectively.

9
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Table 3
Persona and participant demographic information.
No Persona Participant Persona Participant Persona Participant
gender gender age age cultural cultural
background background
1 Male 18–21 Indian Indian
2 Female 26–30 Chinese
3 Male Male 18 18–21 Caucasian Caucasian
4 Female Female 30 26–30 Chinese
5 Female Female 20 18–21
6 Male 22–25 Australian
7 Non-Binary 22–25 Australian
8 Male Male 19 18–21
9 Female Female 23 22–25 Sri Lankan Sri Lankan
10 Female Female 23 22–25 Sri Lankan
11 Female Female 20 18–21 Cambodian
12 Male Male 20 18–21 Indian Indian
13 Male Male 19 18–21 Australian
14 Female Male 19 18–21 Australian
15 Male Male 24 22–25 Indian Indian
16* Female Female 23 22–25 Sri Lankan Sri Lankan
17 Male Male 30 22–25 Indian
18 Female Female 20 18–21 Malaysian
19 Male Male 22 18–21 Malaysian

* Created in collaboration by the participants who created Personas 9 and 10.

Table 4 5.2.2. Motivations and goals

Mean total scores by gender.
Money and Lifestyle. Money or lifestyle aspirations were motivating
Gender Total score factors for six of the student personas (four male, one female, one
N Min Max Mean Std ungendered). Financial difficulties motivated two of the male personas.
Men 7 −18 17 1.29 14.48 They wanted to earn enough money to support themselves, or their
Women 7 −16 9 −3.00 9.68 families. Others hoped that their CS studies would lead to job oppor-
tunities with higher pay, increased flexibility and freedoms, or a better
work life balance.
The personas described variable learning styles, but no gender
patterns were identified within this attribute. Notably, statements sug- Male Persona: ‘‘(1) money (2) good working environments (3) flexible
gesting a preference for a slower learning pace, including concerns with work timings (4) work at home options’’
being overwhelmed with too much information at once, were present Ungendered Persona: ‘‘Their goal is to get a job that provides better
in a third of the personas, but this did not appear to be gender related. pay, more freedom and Independence and provides them with a good
work-life balance’’.
5.2.1. Skills, experience and background
Existing CS Skills. Most of the responses (15) provided some descrip- Interest/Passion The female personas were more likely to be moti-
tion of the personas level of experience in CS. Eight personas, four vated by passion. While four female and two male personas expressed
male and four female, had existing skills or knowledge. All of these goals or motivations based on interest in CS, the descriptions used in
experienced personas had some programming knowledge, and there the female personas were more emotive. Three used the words ‘‘passion
were examples of both male and female personas with skills in data for’’ or ‘‘passionate about’’, while the male personas only referenced
analysis or UI design. Male and female personas were also equally likely enjoyment or interest.
to be described as being inexperienced. Motivated by Others. Of the eight male personas, six included mo-
While there were no gender differences in the types of existing CS tivations related to other people. Two of these wanted to exceed or
skills, there were differences in the language used to describe them. match the success of their peers, two were motivated by family, and
While the words ‘‘experience’’ or ‘‘background’’ were used in three of two aimed to improve other’s lives through their future work. The
the four experienced male personas, female personas only referenced non-binary persona similarly had altruistic goals, wanting to promote
‘‘skills’’ or ‘‘knowledge’’. While two of the male personas were described online safety for LGBTQ+ people. In contrast, only one of the female
as being ‘‘talented’’ or ‘‘good with’’, the only measure of skill included personas was primarily motivated by others. They were taking a CS
in a female persona was ‘‘fundamental’’ course because of family expectations.
Recognition or Accomplishment. Similarly, a wish to be exceptional
or have recognition for their talents was common in male personas (3),
Male Persona Example: ‘‘Talented UI designer and data analyst, some
but almost entirely absent from the female personas.
experience from high school as well as personal use of different pro-
grams’’
Male persona: ‘‘Wants to aim to be a cut above the rest. . . Motivated by
Female Persona Example: ‘‘Skills: fundamental knowledge of Python, having something to be proud about, good progress spurs more effort.
Java, HTML, MATLAB and CSS. . . ’’ Want to be recognised for their efforts in some capacity’’.

Communication and Teamwork. The ability to work with others was While many female personas included goals related to career suc-
the most referenced non-CS related skill. More than half of the female cess, none included the desire to make a unique contribution in CS or
personas (5) included communication, teamwork, or people skills. In be more successful than their peers.
contrast, these types of skills were not included in any of the male Social Goals. While other people were not primary motivations in
personas. This greater interest in collaboration and social connections the female personas, many described personal social goals that were
was also reflected in the goals of some of the female personas. not present in any of the male personas. Half (4) aspired to be more

10
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Table 5
Mean facet scores by gender.
Gender Information processing Learning style Computer Self-Efficacy Attitude towards risk Motivations
Min Max Mean Min Max Mean Min Max Mean Min Max Mean Min Max Mean
Men 4 10 6.43 −6 10 2.29 −9 5 −3.86 −9 4 −1.57 −9 10 −2.00
Women −9 8 2.71 −10 9 3.29 −10 8 −4.71 −10 10 −3.43 −10 8 −0.86

extroverted, participate in student projects, or build their personal or 5.2.4. Frustrations and constraints
professional connections. The constraints and frustrations of most female personas focused
Academic Goals. Academic goals were present in both male and only on personal limitations, while the male persona’s constraints were
female personas but these tended to be more explicit, and were more equally likely to be based on personal limitations and external factors
likely to be repeated multiple times, within the female personas. In outside of their control.
one female persona, high academic achievement was their first listed Low Confidence in Abilities. In some female personas, the descriptions
motivation, goal, and constraint. seemed to show a lack of confidence in the student’s ability to complete
their coursework. While there were also examples of male personas who
Female persona: ‘‘Worrying too much about academic results which
were struggling, none seemed to imply that the student could not learn
restricts her from trying new things. Eg. studying electives from other
if given additional time or resources.
faculties’’

5.2.3. Preferences Male Persona: ‘‘doesn’t always pick things up immediately, requires
Presentation and Attention. Almost half of the personas (9) included access to resources that make it easy to look back on and fix miscon-
aesthetic preferences for content presentation or UI designs. These ceptions or that help you apply your learning’’. Female Persona:: ‘‘She
included the use of colour and interesting fonts, length of text passages, tried so hard, but it’s not working’’. ‘‘is a helplessful (sic) student’’
and general creativity and character of the overall design. Male and
female personas were equally likely to be bothered by a bland or boring Personas were not just affected by their own perceptions, but also
presentation, but in female personas this was more often linked to by how they were seen by others. Two female personas were concerned
motivation and attention. with being perceived negatively by their peers because of their gender.
This was expressed as being ‘‘looked down on’’ or ‘‘considered as having
‘‘Believes that a nicer UI would help in keeping student’s interest on the low potential’’. In one case this was a result of personal exposure
page’’ ‘‘Rather boring site - might lose interest quickly’’ to negative stereotypes in STEM. Similarly, one of the male personas
described being negatively impacted by their teacher’s impressions.
A preference for visual learning, with the inclusion of video or picture
content, was also found in two of the female personas.
Content and Structure. The length and structure of text passages was ‘‘and his professors sideline him as they think he’s not smart enough to
another element frequently linked to attention and motivation. Both complete the degree’’.
male and female personas showed a preference for shorter text passages
and examples, or the inclusion of summaries and overviews. In female Emotions and Mental Health. Descriptions of emotional difficulties
persona’s this was expressed in their criticisms of the online tutorials were present in four of the personas. Three of those personas were
used in the second task. male and one was non-binary. Two personas had an existing mental
health condition (one male, one non-binary). The others were stressed
Female Persona Extracts: ‘‘no overview showing all components of SQL or overwhelmed by assessment or heavy workloads.
& how they relate to each other’’ ‘‘no summary given after each topic’’ Time Management and Motivation. Six personas (four female, two
‘‘While the content’s way of explanation is pretty straightforward, a lot male) had difficulties with time management. There were examples of
of reading content becomes monotonous at one point’’. both male and female personas who struggled to meet assignment dead-
lines or keep up with their course content. Reasons for this included
While examples were seen to be useful, they could be viewed
unmanageable workloads, problems with motivation and, in the case
negatively when not sufficiently realistic. One of the career-oriented
male personas criticised a lack of connection between the examples of two female personas, struggles with competing commitments and
available and their future work. work-life balance.

‘‘Real world examples are present but still not similar to what would be ‘‘Sometimes failing to balance work/professional/personal life while
seen in career’’. being a student’’ ‘‘Has other commitments that can take up a lot of time
so her attention is often split on various goals’’.
Inclusion of practical exercises was viewed positively in many per-
sonas, but both the male personas and real-life participants had mixed Financial or Resource Constraints. Descriptions of personal financial
reactions to the quizzes present in the online tutorials. While they were struggles or a lack of access to required resources were present in half
viewed as being useful, they were also a source of stress. of the male personas. Two of these were unable to access software they
needed because of OS or hardware limitations.
Male Persona: ‘‘If this quiz were assessed - would be under a lot of stress
and pressure making some of his answers incorrect as he cannot think
5.3. Personas
straight’’.

Software Usability. Preferences related to usability were found in Gender differences identified in the sample were used to construct
seven of the personas (Four male, two female, one ungendered). There the preliminary personas shown in Fig. 5 below (Hereafter CSTim and
were preferences for clear icons and layouts, simple menu structures, CSAbi). While individual students are likely to display a range of these
and navigation that does not require students to take an excessive qualities, and may behave in more ‘‘Pat’’ style, we considered the
number of steps to reach their goal. extremes of the traits found in our sample. As such only ‘‘Tim’’ and
This wish for software that is easier to use was also reflected ‘‘Abi’’ personas were created. Since our main motivation was to detect
in many personas’ frustrations. A lack of accessibility options, like bias in education software, two personas with the extreme qualities of
tutorials, and poorly designed interfaces were common concerns. each spectrum were sufficient for the congnitive walkthroughs.

11
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Table 6
Traits and qualities of CS/SE students.
Attribute Traits Student qualities
Level of Experience
Inexperienced in CS
Not Sufficiently Prepared by Prior CS Studies
Skills, Experience and Background Prior Skills/Knowledge in CS
Non-CS Related Skills
Communication and Teamwork (*f)
Other Skills
Harmful Gender Related Experience
Stereotypes (*f)
Harassment
Content Presentation
Appealing Presentation
Attention (*f)
Visual Learning (*f)

Preferences Content
Structure and Segmentation
Amount of Text
Practical Exercises(*m)
Software Usability(*m)
Clarity
Navigation
Independence
Prefers Support
Works Alone
Attitude Towards Learning
Enjoys Learning
Learns Everything They Can
Learns Only What’s Necessary
Learning Style and Behaviour
Ease of Learning
Easily Picks Up New Content
Has Difficulties Learning
Perseverance Through Difficulties
Learning Pace
Works Efficiently
Slower Learning Pace
Career
Money and Lifestyle (*m)
Interest/Passion
Recognition or Accomplishment (*m)
Goals and Motivations
Study
Academic Achievement(*f)
Skill Development
Social(*f)
Other People(*m)
Family
Peers
Altruistic Goals
Software(*m)
Accessibility
Complexity and Design
Teaching Approach
Pacing of Content

Frustrations and Constraints Content

Lacks Connection to Future Career
Lacks Clarity
Poor Presentation(*f)
Personal Barriers
Struggles with Emotions or Mental Health (*m)
Financial or Resource Constraints (*m)
Time Management and Motivation (*f)
Low Confidence in Abilities (*f)
Interpersonal
Lack of Support
Gender Stereotypes(*f)
Social Difficulties

12
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Fig. 5. Preliminary CS/SE student personas.

5.4. Cognitive walkthroughs directly accessible to users (Fig. 6) found with CSTim. The greying out
of this checkbox may mislead users that the option is not currently
Usability issues were found in all eight cognitive walkthrough tasks. available. They actually need to find and check a separate, less clearly
The number of issues detected did not appear to be related to the labelled, checkbox to add the AUTO_INCREMENT keyword to their
application’s complexity, or whether it is primarily visual or text based. script.
An overview of the results for each persona can be seen in Table 7.
Using GenderMag’s Abi revealed the most issues, followed by the
Most of the issues detected were minor, causing frustration or
two CS/SE student personas. Only two of CSTim’s facets were relevant
delays, but not preventing the persona from completing their task.
However, there were four notable exceptions. An unintuitive process to software usability: frustration with complex UI designs and naviga-
for deleting project boards in Trello detected by three personas, a tion, and preference for simple menus and limited steps. In contrast
hidden link to recorded sessions in Canvas found by two personas, five of CSAbi’s facets were raised during the walkthroughs. These
locating student groups for joining in Canvas found by two personas included preference for visual learning, difficulty maintaining attention
and an ‘‘Auto Increment’’ checkbox in the MySQL Workbench not when content is not presented in an appealing way, struggles with

13
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Table 7
Usability issues detected (Yes:Y, No:N).
Application Task Tim Abi CSTim CSAbi
Create a new database N N N Y
MySQL
Create a table using the MySQL workbench N N Y N
Learn to use basic features Y N N N
BlueJ
Debug a program N Y* Y Y*
Prepare a new project N Y N Y
IntelliJ
Learn code completion N Y Y N
Create and share a simple board N N N N
Trello
Delete an existing Trello Board N Y* Y* Y*
Find recorded sessions N Y* N Y*
Canvas
Joining an existing group (when group is full/not full) N Y* Y* N

*Used to denote that the same issue was detected by two or more personas.

Fig. 6. MySQL auto-increment.

time management, communication and teamwork skills, and low self- may not be applicable for CS education software. Our findings indicate
confidence. However, this persona detected only three unique issues, that some of gender differences of CS/SE students are not captured
and both were minor. by the GenderMag personas, and this could influence their experience
There was overlap between issues raised by CSAbi’s low self with CS education software. For example CSTim’s facets of ‘‘disliking
-confidence and the GenderMag dimension of computer self-efficacy, complex UI design and navigation’’ and ‘‘preference for simple menu
and all five GenderMag facets were raised when conducting walk-
structures’’ helped to identify issues such as - a hidden link to recorded
throughs with the Abi persona.
sessions in Canvas and potential struggle with deleting a Trello board.
Previous research applying GenderMag to review Canvas (Chatterjee
6. Discussion
et al., 2022), mostly focused on the content of the courses, however
To understand the facets of CS/SE students that influence their our findings indicate that together with GenderMag’s personas, CSTim
learning experience, we adopted a workshop approach to develop and CSAbi were also helpful to find issues in the education software.
CS/SE student personas. From the personas developed by the CS/SE There was an extensive list of qualities found in the personas created
students themselves, we could identify a wide range of facets. Con- by CS/SE students. Only a subset of those, that were present in multiple
sidering only those that were described in multiple personas, we de- personas, were used for CSTim and CSAbi persona development. This
veloped CSTim and CSAbi personas with clearly differentiating facets. extensive list of all qualities needs further investigation if we are
Our research findings indicate that workshops can be very useful for
to understand CS/SE student characteristics, develop a more gender
developing personas for use in InclusiveMag framework scoping.
inclusive learning and teaching environment, and refine the personas
The usability evaluation with the GenderMag and newly developed
to better detect bias within CS education software and/or learning
CS/SE personas revealed that, although the GenderMag Abi persona is
very effective, CSTim and CSAbi could detect some issues not found by contents, as a whole.
the GenderMag personas. Since GenderMag is developed with problem In the following subsections we discuss our findings for each RQs
solving software in mind, all the facets of the GenderMag personas stated in Section 3.

14
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

6.1. RQ1: The facets of CS/SE students 6.2. RQ2: Consistency with the GenderMag facets

Thematic analysis of the workshop responses revealed a comprehen- The survey scores of the students in our sample did not display the
sive list of personal qualities relevant to the study experience of CS/SE gender differences predicted by the GenderMag framework. Of note are
students. Personas were split into gender groups and facets extracted the male participant’s tendencies towards comprehensive information
where a quality was present only in one group or expressed in at least processing and the female participant’s high computer self-efficacy.
half of the personas in a group. Non-binary students were insufficiently While these may be explained by the small sample size and expected
represented in the data, so only male and female student personas were individual differences in gender expression, it is also possible that these
reconstructed based on the results. are areas where CS/SE student traits differ from those of the general
Within our sample, gender differences were found in preferences, population.
motivations, goals, frustrations, and constraints. With the exception In their survey of college students Beyer (2014) found that computer
of self-confidence, they do not appear to be captured in the Gender- self-efficacy is a predictor of intention to undertake CS studies irrespec-
Mag personas, and there is some mixed support for them within the tive of gender. This suggests that women who choose to undertake CS
literature. courses will rate themselves higher on this facet than those who do not.
The social skills and goals expressed exclusively in the represen- It is interesting, however, that the high computer self-efficacy ratings of
tations of female CS/SE students, seem to suggest they may be more the female participants were not reflected in the persona descriptions
comfortable and adept in group work situations than male CS/SE or our own observations within the workshops. This may suggest that
students. There is also some support for this in studies of cooperative lower self-confidence in female CS/SE students is related to aspects of
behaviour (Vugt et al., 2007) and social skills (Groves, 2005). However, their coursework other than software use.
in a large-scale meta-analysis Balliet et al. (2011) found that any gender Another consideration could be CS students evaluating their self-
differences in cooperative behaviour may be context dependent. Men efficacy differently, resulting from a better idea of ‘‘how much to
were more cooperative while working in a single gender group while learn’’ than other people. Although a comparison with other profession
women were more cooperative when working with men. It is possible or discipline is not available in the literature, self-efficacy has been
that social aspects were included in the female personas not because an important aspect of research within software engineering (Ribeiro
they are more skilled in these areas, but because these traits are viewed et al., 2023; Hazzan and Seger Guttmann, 2010; Tsai and Cheng, 2010),
as being more important in women (MacNell et al., 2015). indicating a potential difference with other disciplines.
Similarly, research in product design suggests that the idea that
women place greater importance on visual aesthetics may be a stereo- 6.3. RQ3: Applicability of CS/SE personas in usability evaluation
type that is not reflected in their actual behaviour (Dai et al., 2023).
Nevertheless, in the present study, preferences for visual learning and CSTim and CSAbi were useful in cognitive walkthrough tasks and
appealing presentation were indeed exhibited during the persona test- could reveal usability issues similar to GenderMag Tim and Abi per-
ing task. While the coding tutorials were not undertaken in a real sonas. CSTim identified one usability issue that was not detected by any
classroom, these responses may still provide credible insights into the other personas. Several facets were identified from the workshops that
preferences and frustrations of students when being presented with were used to develop CSTim and CSAbi personas. In the ten cognitive
new content. In the cognitive walkthroughs, unappealing UI design walkthrough tasks we carried out, two facets of CSTim and five facets
was an issue detected in two of the four software applications. If the of CSAbi were useful. These outcome are promising and indicate that,
gender differences found in the sample are reflective of the wider depending on the nature of the cognitive walkthrough tasks, different
CS/SE student population, this could negatively affect the education facets of CSTim and CSAbi personas can be helpful in detecting different
experience of female students. usability issues.
They may also be affected by low self-confidence. The content
of the one-line summaries, and nature of the constraints included in 6.4. RQ4: Performance of the CS/SE student personas compared to Gen-
the personas, suggest that female students tend to blame themselves derMag
when facing difficulties in their studies. This is consistent with the
GenderMag computer self-efficacy facet and supported by research in- While reasonable facets could be extracted from the workshop
vestigating gender differences in self-compassion (Yarnell et al., 2019), responses, the cognitive walkthroughs revealed that only a subset of
however, may seem to contradict findings that women generally have these are valuable in software usability testing. CSAbi and CSTim both
a more external locus of control than men (Sherman et al., 1997). found issues not detectable with the GenderMag facets, however, the
Locus of control is a spectrum of thinking between believing that GenderMag Abi persona still appears to be more effective at detecting
outcomes are caused by one’s own characteristics and behaviours (in- gender bias in software, even when applying a walkthrough adapted
ternal), or by outside factors (external) (Rotter, 1966). This is, however, for use in education research.
domain dependent (Sherman et al., 1997), and in the education con- However, the preliminary personas we introduced are broad in
text Ghazvini and Khajehpour (2011) found that girls did have a greater scope, and may be able to supplement the GenderMag personas. The
tendency towards internal locus of control than boys. They also found student qualities revealed in this study are diverse, wide-ranging, and
gender differences in time management and self-testing behaviours highlight the personal qualities our student participants considered
comparable to other facets present in our personas. most relevant to their study experience. There was also very little
At first glance, the male CS/SE student persona’s greater concerns overlap between these qualities and the GenderMag facets.
with software usability and navigation seem to conflict with the fact
that most software is designed in a way that is most suitable for 7. Threats to validity
men (Nunes et al., 2021), and appear to be a departure from the Gen-
derMag Tim persona’s tinkering behaviour (Burnett et al., 2016b). This 7.1. Construct validity
finding may be an anomaly stemming from our relatively small sample
size and highlight the limitations of trying to draw clear conclusions Examples were needed in the workshop presentation to ensure that
based on what has been included or excluded from our participant’s all participants understood what was required of them. Despite efforts
responses. Conversely, male CS/SE students may have higher expecta- made to ensure these examples were general and not directly related to
tions for the software they use than the general population and differ education experiences, we cannot be certain that they did not influence
from the Tim persona in one or more of their key characteristics. participant responses. For example, during the pilot workshop colour

15
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

was used as an example of UI design that may be perceived differently 8. Conclusion

by people of different genders. This may have influenced one of the par-
ticipants who said certain colours make them feel uncomfortable. This This paper investigated tools for identifying gender bias within
example was removed from the presentation in subsequent workshops CS/SE education software. A series of workshops were held to identify
to avoid further biasing participant responses. the personal qualities of students that influence their experience in
CS courses and gender differences across these qualities were used
7.2. External validity to construct personas for use in usability testing. The performance
of these personas was then compared to GenderMag’s Tim and Abi.
Generalisability is a major concern of this type of research. We Although the effectiveness of the GenderMag personas is well supported
recruited participants from one large Australian University due to by existing research, their facets are based on the general population.
our approved human research ethics protocol. The study was adver- The participants in our small sample were not well represented by
tised with social media messages, via different university clubs, an the GenderMag facets, and more research is needed to investigate the
on-campus leaflet distribution, and snowballing. However due to the applicability of that framework to CS/SE students as a unique group.
nature of participation requirements we could only find a small number In the future, we plan to replicate the research study, using different
of students with the time and interest to participate. Although a rich set methods for creating CS/SE student personas, to confirm or refute the
of male and female CS student attributes were identified in this study, facets founds.
the findings need to be verified with larger sample size recruited from The CS/SE student personas did not perform as well as Abi, how-
more organisations. ever, they were both able to identify usability issues not detectable with
Another key threat to external validity is the sample bias. As a the GenderMag facets. Additionally, the research produced a detailed
requirement of the approved human research ethics protocol, all our list of student qualities that may assist with the identification of bias
advertisements clearly stated our motivation to address the issue of within the broader CS/SE education environment. Further research
‘‘gender bias’’ in the CS education environment. As such those who are investigating gender differences across these qualities in a larger and
interested in the topic, have awareness, or have any lived experience more diverse sample may allow for the development of personas and
may have opted for participation. This is also evident from the fact methods that can be applied to identify bias in other learning materials,
that an almost equal number of male and female students participated not just software. Supporting the consideration of gender inclusivity in
in the study, as this does not reflect the gender distribution of the this way could improve women’s course completion rates and foster
population. To reduce the impact of any ‘‘sample bias’’, we did not greater diversity within the CS field.
collect any information on their views on the topic, and instead the
participants were requested to develop personas containing their or the CRediT authorship contribution statement
peers’ attributes as they see them. The instructions of the workshop
were crafted in such a way, that no ‘‘gender’’ related concepts were Lyndsey O’Brien: Conceptualization, Data curation, Formal anal-
used, so that the participants can think about the attributes of students ysis, Investigation, Methodology, Project administration, Resources,
of any gender without relating those to ‘‘gender bias’’. Writing – original draft, Visualization. Tanjila Kanij: Conceptualiza-
tion, Formal analysis, Investigation, Methodology, Project administra-
7.3. Internal validity tion, Resources, Software, Supervision, Validation, Writing – review
& editing. John Grundy: Supervision, Validation, Writing – review &
A major motivation of this research was to better understand the editing.
attributes and characteristics of male and female CS students. We used
persona development workshops with CS students themselves for two Declaration of competing interest
reasons - (1) personas are a powerful tool for understanding end users,
and (2) our research is inspired by the success of the GenderMag The authors declare that they have no known competing finan-
framework which uses persona as one of its essential elements. We cial interests or personal relationships that could have appeared to
found several characteristics from the persona development workshops influence the work reported in this paper.
and some significant differences between males and females. The find-
ings were compiled in CSTim and CSAbi personas which were able Data availability
to find some bias related issues in sample CS/SE software. Although
these workshops were very helpful in identifying facets of CS students, Data will be made available on request.
we cannot overlook the fact that the personas were developed by
CS students themselves. The accuracy/consistency of these facets are Acknowledgements
highly dependent on the ability of the participants to engage in ‘‘self-
assessment’’ and ‘‘role play’’. As such the findings need to be confirmed We thank all the students who participated in the workshops. Kanij
with a replication study. and Grundy are supported by ARC Laureate Fellowship FL190100035.
While most participants provided detailed and personal responses
to the tasks, study participants were offered a gift card to thank them References
for their time, and one participant indicated this was their primary
motivation for participating. Conducting workshops in groups may Adlin, T., Pruitt, J., 2010. The Essential Persona Lifecycle: Your Guide to Building
have also limited the openness of participant responses, particularly and Using Personas. Morgan Kaufmann, Burlington, Massachusetts, USA, http:
//dx.doi.org/10.1016/C2009-0-62475-2.
when thinking about personal limitations and negative experiences.
Al-Taharwa, I., 2020. Female student participation in software engineering projects:
Furthermore, while almost all participants created a single persona Opportunities to model project evaluation and to improve early prediction of
based on themselves, secondary personas developed based on friends or teamwork failure. J. Educ. Pract. 11 (35), 30–36.
classmates may be affected by the participant’s own assumptions and Anna Lindqvist, M.G.S., Renström, E.A., 2021. What is gender, anyway: a review
biases. Any of these factors may have influenced the validity of individ- of the options for operationalising gender. Psychol. Sexual. 12 (4), 332–344.
http://dx.doi.org/10.1080/19419899.2020.1729844.
ual answers. As such qualities and gender differences highlighted in our Balliet, D., Li, N.P., Macfarlan, S.J., Van Vugt, M., 2011. Sex differences in cooperation:
results were only considered when present in the responses of multiple a meta-analytic review of social dilemmas.. Psychol. Bull. 137 (6), 881, DOI:
students. 10.1037/a0025354.

16
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Barker, L.J., McDowell, C., Kalahar, K., 2009. Exploring factors that influence computer Heels, L., Devlin, M., 2019. Investigating the role choice of female students in a software
science introductory course students to persist in the major. ACM Sigcse Bull. 41 engineering team project. In: Proceedings of the 3rd Conference on Computing
(1), 153–157. http://dx.doi.org/10.1145/1539024.1508923. Education Practice. CEP ’19, Association for Computing Machinery, New York, NY,
Benest, I., Carter, J., Chandler, J., 2003. Identifying the dominant personality and USA, http://dx.doi.org/10.1145/3294016.3294028.
attitudinal traits in computer science students. In: Proc. Int. Conf. Eng. Educ. 2003. Henwood, F., 1999. Exceptional women? Gender and technology in U.K. higher
Available: https://api.semanticscholar.org/CorpusID:14144701. education. IEEE Technol. Soc. Mag. 18 (4), 21–27. http://dx.doi.org/10.1109/44.
Beyer, S., 2014. Why are women underrepresented in Computer Science? Gender 808845.
differences in stereotypes, self-efficacy, values, and interests and predictors of Höhne, E., Zander, L., 2019. Belonging uncertainty as predictor of dropout intentions
future CS course-taking and grades. Comp. Sci. Educ. 24 (2–3), 153–192. http: among first-semester students of the computer sciences. Z. Erziehungswiss 22, no.
//dx.doi.org/10.1080/08993408.2014.963363. 5 (5), 1099–1119. http://dx.doi.org/10.1007/s11618-019-00907-y.
Bittner, A., Goodyear-Grant, E., 2017. Sex isn’t gender: Reforming concepts and Jarratt, L., Bowman, N., Culver, K., Segre, A., 2019. A large-scale experimental study
measurements in the study of public opinion. Political Behav. 39 (4), 1019–1041. of gender and pair composition in pair programming. In: Proceedings of the 2019
http://dx.doi.org/10.1007/s11109-017-9391-y. ACM Conference on innovation and technology in computer science education.
Braun, V., Clarke, V., 2006. Using thematic analysis in psychology. Qualit. Res. Psychol. ITiCSE ’19, ACM, pp. 176–181. http://dx.doi.org/10.1145/3304221.3319782.
3 (2), 77–101. http://dx.doi.org/10.1191/1478088706qp063oa. Jaspers, M.W., 2009. A comparison of usability methods for testing interactive health
technologies: Methodological aspects and empirical evidence. Int. J. Med. Inform.
Burnett, M., Peters, A., Hill, C., Elarief, N., 2016a. Finding gender-inclusiveness software
78 (5), 340–353. http://dx.doi.org/10.1016/j.ijmedinf.2008.10.002, https://www.
issues with GenderMag: A field investigation. In: Proceedings 2016 CHI Conf. on
sciencedirect.com/science/article/pii/S1386505608001809.
Human Factors in Comput. Syst.. CHI ’16, pp. 2586–2598. http://dx.doi.org/10.
Karolita, D., Grundy, J., Kanij, T., Obie, H., McIntosh, J., 2023. What’s in a persona?
1145/2858036.2858274.
A preliminary taxonomy from persona use in requirements engineering. In: Intern.
Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., Peters, A.,
Confer. ENASE 2023. pp. 39–51. http://dx.doi.org/10.5220/0011708500003464.
Jernigan, W., 2016b. GenderMag: A method for evaluating software’s gender
Keller, E.F., 1995. Reflections on Gender and Science. Yale University Press.
inclusiveness. Interact. Comput. 28 (6), 760–787. http://dx.doi.org/10.1093/iwc/
Khajouei, R., Zahiri Esfahani, M., Jahani, Y., 2016. Comparison of heuristic and
iwv046.
cognitive walkthrough usability evaluation methods for evaluating health infor-
Chatterjee, A., Letaw, L., Garcia, R., Reddy, D.U., Choudhuri, R., Kumar, S.S., Mor-
mation systems. J. Am. Med. Inform. Assoc. 24 (e1), e55–e60. http://dx.doi.org/
reale, P., Sarma, A., Burnett, M., 2022. Inclusivity bugs in online courseware:
10.1093/jamia/ocw100, arXiv:https://academic.oup.com/jamia/article-pdf/24/e1/
A field study. In: ICER ’22 -Volume 1. pp. 356–372. http://dx.doi.org/10.1145/
e55/34149263/ocw100.pdf.
3501385.3543973.
Kohl, K., Prikladnicki, R., 2022a. Benefits and difficulties of gender diversity on
Cheryan, S., Plaut, V.C., Davies, P.G., Steele, C.M., 2009. Ambient belonging: How software development teams: A qualitative study. In: Proc. SBES ’22. pp. 21–30.
stereotypical cues impact gender participation in computer science. J. Personal. http://dx.doi.org/10.1145/3555228.3555253.
Soc. Psychol. 97 (6), 1045–1060. http://dx.doi.org/10.1037/a0016239. Kohl, K., Prikladnicki, R., 2022b. Benefits and difficulties of gender diversity on
Cheryan, S., Ziegler, S.A., Montoya, A.K., Jiang, L., 2017. Why are some STEM fields software development teams: A qualitative study. In: Proceedings of the XXXVI
more gender balanced than others? Psychol. Bull. 143 (1), 1–35. http://dx.doi.org/ Brazilian Symposium on Software Engineering. SBES ’22, Association for Computing
10.1037/bul0000052. Machinery, New York, NY, USA, pp. 21–30. http://dx.doi.org/10.1145/3555228.
Choi, K.S., 2015. A comparative analysis of different gender pair combinations in pair 3555253.
programming. Behav. Inf. Technol. 34, 825–837, https://api.semanticscholar.org/ Kolekar, S.V., Pai, R.M., Pai M.M., M., 2018. Adaptive user interface for
CorpusID:11680590. moodle based E-learning system using learning styles. Procedia Comput.
Clarke, V., Braun, V., 2017. Thematic analysis. J. Positive Psychol. 12 (3), 297–298. Sci. 135, 606–615. http://dx.doi.org/10.1016/j.procs.2018.08.226, https://www.
http://dx.doi.org/10.1080/17439760.2016.1262613. sciencedirect.com/science/article/pii/S1877050918315229 The 3rd International
Cohoon, J.M., 2007. Gendered experiences of computing graduate programs. SIGCSE Conference on Computer Science and Computational Intelligence (ICCSCI 2018)
Bull. 39 (1), 546–550. http://dx.doi.org/10.1145/1227504.1227494. : Empowering Smart Technology in Digital Era for a Better Life.
Dai, Y., Byun, G., Ding, F., 2019. The direct and indirect impact of gender diversity in Kovaleva, Y., Kasurinen, J., Kindsiko, E., Happonen, A., 2024. State-of-the-art review on
new venture teams on innovation performance. Entrepreneurship Theory Pract. 43 current approaches to female inclusiveness in software engineering and computer
(3), 505–528. http://dx.doi.org/10.1177/1042258718807696. science in higher education. IEEE Access 12, 1360–1373. http://dx.doi.org/10.
Dai, X., Lin, Y., Liang, J., Yang, C., 2023. Appearance for females, functionality for 1109/ACCESS.2023.3346767.
males? The false lay belief about gender difference in product preference. J. Larson, L.M., Wu, T.F., Bailey, D.C., Gasser, C.E., Bonitz, V.S., Borgen, F.H., 2010. The
Consum. Res. ucad054, http://dx.doi.org/10.1093/jcr/ucad054. role of personality in the selection of a major: With and without vocational self-
Denscombe, M., 2010. The Good Research Guide For Small-Scale Social Research efficacy and interests. J. Vocat. Behav. 76 (2), 211–222, https://www.sciencedirect.
Projects, fourth ed. McGraw-Hill International (UK) Ltd, Maidenhead, England. com/science/article/pii/S0001879109001341.
Department of Industry, Science and Resources, 2021. STEM Equity Monitor: Data Lau, W.W.F., Yuen, A.H.K., 2010. Gender differences in learning styles: Nurturing a
Highlights 2021. Commonwealth Government of Australia, [Online]. Available gender and style sensitive computer science classroom. Austral. J. Educat. Technol.
https://www.industry.gov.au/sites/default/files/May%202021/document/stem- 26 (7), DOI: 10.14742/ajet.1036.
equity-monitor-highlights-report-2021_0.pdf. Lim, K., Nam, Y.O., Eom, S., Jang, Y., Kim, D., Kim, M.H., 2020. Structural gender dif-
ferences in LMS use patterns among college students. Sustainability 12 (11), http://
Department of Industry, Science and Resources, 2022. STEM Equity Monitor Data
dx.doi.org/10.3390/su12114465, https://www.mdpi.com/2071-1050/12/11/4465.
Report 2022. Commonwealth Government of Australia, [Online]. Available
Lishinski, A., Yadav, A., Good, J., Enbody, R., 2016. Learning to program: Gender
https://www.industry.gov.au/sites/default/files/2022-09/stem-equity-monitor-
differences and interactive effects of students’ motivation, goals, and self-efficacy
data-report-2022.pdf.
on performance. In: Proc. ACM ICER ’16. pp. 211–220. http://dx.doi.org/10.1145/
Department of Industry, Science and Resources, 2023. University Enrolment and Com-
2960310.2960329.
pletion in STEM and Other Fields. Commonwealth Government of Australia, [On-
MacNell, L., Driscoll, A., Hunt, A.N., 2015. What’s in a name: Exposing gender
line]. Available https://www.industry.gov.au/publications/stem-equity-monitor/
bias in student ratings of teaching. Innovat. Higher Educ. 40, 291–303, DOI:
higher-education-data/university-enrolment-and-completion-stem-and-other-fields.
10.1007/s10755-014-9313-4.
Etikan, I., Musa, S.A., Alkassim, R.S., 2016. Comparison of convenience sampling and
Mahatody, T., Sagar, M., Kolski, C., 2010. State of the art on the cognitive walkthrough
purposive sampling. Am. J. Theoret. Appl. Statist. 5 (1), 1–4. http://dx.doi.org/10.
method, its variants and evolutions. Int. J. Human–Comput. Interact. 26 (8),
11648/j.ajtas.20160501.11.
741–785. http://dx.doi.org/10.1080/10447311003781409.
Farzandipour, M., Nabovati, E., Sadeqi Jabali, M., 2022. Comparison of usability Master, A., Meltzoff, A.N., Cheryan, S., 2021. Gender stereotypes about interests
evaluation methods for a health information system: heuristic evaluation versus start early and cause gender disparities in computer science and engineer-
cognitive walkthrough method. BMC Med. Inform. Decis. Mak. 22 (1), 1–11. ing. Proc. Natl. Acad. Sci. 118 (48), e2100030118. http://dx.doi.org/10.1073/
http://dx.doi.org/10.1186/s12911-022-01905-7. pnas.2100030118, arXiv:https://www.pnas.org/doi/pdf/10.1073/pnas.2100030118
Ghazvini, S.D., Khajehpour, M., 2011. Gender differences in factors affecting academic https://www.pnas.org/doi/abs/10.1073/pnas.2100030118.
performance of high school students. Procedia-Soc. Behav. Sci. 15, 1040–1045. McCrae, R.R., Costa, P.T., Psychological Assessment Resources, I., 2010. NEO invento-
http://dx.doi.org/10.1016/j.sbspro.2011.03.236. ries for the NEO personality inventory-3 (NEO-PI-3), NEO five-factor inventory-3
Gómez, O.S., Solari, M., Pardo, C., Ledezma-Carrizalez, A., 2017. A controlled ex- (NEO-FFI-3), NEO personality inventory-revised (neo PI-R) : professional manual.
periment on productivity of pair programming gender combinations: Preliminary PAR https://cir.nii.ac.jp/crid/1130282272331328256.
results. ISBN: 978-987-3806-98-8, pp. 197–210. Medel, P., Pournaghshband, V., 2017. Eliminating gender bias in computer science
Groves, K.S., 2005. Gender differences in social and emotional skills and charismatic education materials. In: Proc 2017 ACM SIGCSE Tech.. pp. 411–416. http://dx.doi.
leadership. J. Leadership Organ. Stud. 11 (3), 30–46. http://dx.doi.org/10.1177/ org/10.1145/3017680.3017794.
107179190501100303. Mendez, C., Letaw, L., Burnett, M., Stumpf, S., Sarma, A., Hilderbrand, C., 2019. From
Hazzan, Seger Guttmann, T., 2010. Recruiting software practitioners: The importance GenderMag to InclusiveMag: An inclusive design meta-method. In: 2019 IEEE Symp.
of self-efficacy. CROSSTALK J. Defense Softw. Eng. 23, 8–12. VL/HCC. pp. 97–106. http://dx.doi.org/10.1109/VLHCC.2019.8818889.

17
L. O’Brien et al. The Journal of Systems & Software 219 (2025) 112225

Metaxa-Kakavouli, D., Wang, K., Landay, J.A., Hancock, J., 2018. Gender-inclusive Schlesinger, A., Edwards, W.K., Grinter, R.E., 2017. Intersectional HCI: Engaging
design: Sense of belonging and bias in web interfaces. In: Proc. 2018 CHI Conf. identity through gender, race, and class. In: Proc. CHI ’17. pp. 5412–5427. http:
on Human Factors in Comput. Syst. pp. 1–6. http://dx.doi.org/10.1145/3173574. //dx.doi.org/10.1145/3025453.3025766.
3174188. Shekhar, A., Marsden, N., 2018. Cognitive walkthrough of a learning management
Miaskiewicz, T., Kozar, K.A., 2011. Personas and user-centered design: How can system with gendered personas. In: GenderIT ’18. pp. 191–198. http://dx.doi.org/
personas benefit product design processes? Des. Stud. 32 (5), 417–430. http: 10.1145/3196839.3196869.
//dx.doi.org/10.1016/j.destud.2011.03.003. Sherman, A.C., Higgs, G.E., Williams, R.L., 1997. Gender differences in the locus of
Minichiello, A., Hood, J.R., Harkness, D.S., 2018. Bringing user experience design to control construct. Psychol. Health 12 (2), 239–248. http://dx.doi.org/10.1080/
bear on STEM education: A narrative literature review. J. STEM Educ. Res. 1, 7–33. 08870449708407402.
http://dx.doi.org/10.1007/s41979-018-0005-3. Silveira, K., Prikladnicki, R., 2019. A systematic mapping study of diversity in software
Moss-Racusin, C.A., Dovidio, J.F., Brescoll, V.L., Graham, M.J., Handelsman, J., 2012. engineering: A perspective from the agile methodologies. pp. 7–10. http://dx.doi.
Science faculty’s subtle gender biases favor male students. Proc Nat. Acad. Sciences org/10.1109/CHASE.2019.00010.
109 (41), 16474–16479, DOI: 10.1073/pnas.1211286109. Tsai, M.-T., Cheng, N.-C., 2010. Programmer perceptions of knowledge-sharing behavior
Murphy, A., Kelly, B., Bergmann, K., Khaletskyy, K., O’Connor, R.V., Clarke, P.M., under social cognitive theory. Expert Syst. Appl. 37 (12), 8479–8485. http:
2019a. Examining unequal gender distribution in software engineering. In: EuroSPI //dx.doi.org/10.1016/j.eswa.2010.05.029, https://www.sciencedirect.com/science/
2019. pp. 659–671. http://dx.doi.org/10.1007/978-3-030-28005-5_51. article/pii/S0957417410004409.
Murphy, A., Kelly, B., Bergmann, K., Khaletskyy, K., O’Connor, R., Clarke, P., Vorvoreanu, M., Zhang, L., Huang, Y.-H., Hilderbrand, C., Steine-Hanson, Z., Bur-
2019b. Examining unequal gender distribution in software engineering. ISBN: nett, M., 2019. From gender biases to gender-inclusive design: An empirical
978-3-030-28004-8, pp. 659–671. http://dx.doi.org/10.1007/978-3-030-28005-5_ investigation. In: Proc. 2019 CHI Conf. on Human Factors in Comput. Syst.. pp.
51. 1–14. http://dx.doi.org/10.1145/3290605.3300283.
Nguyen-Duc, A., Jaccheri, L., 2023. Gender equality in software engineering education Vugt, M.V., Cremer, D.d., Janssen, D.P., 2007. Gender differences in cooperation
– a study of female participation in customer-driven projects. In: Gupta, V., and competition: The male-warrior hypothesis. Psychol. Sci. 18 (1), 19–23. http:
Rubalcaba, L., Gupta, C., Hanne, T. (Eds.), Sustainability in Software Engineering //dx.doi.org/10.1111/j.1467-9280.2007.01842.x.
and Business Information Management. Springer International Publishing, Cham, Wang, Y., Redmiles, D., 2019. Implicit gender biases in professional software devel-
pp. 39–49. opment: An empirical study. In: 2019 IEEE/ACM 41st International Conference
Nielsen, J., 1993. Usability engineering, first ed. AP Professional, Cambridge, England, on Software Engineering: Software Engineering in Society. ICSE-SEIS, pp. 1–10.
http://dx.doi.org/10.1016/C2009-0-21512-1. http://dx.doi.org/10.1109/ICSE-SEIS.2019.00009.
Nunes, I., Moreira, A., Araujo, J., 2021. Conceptual modeling of gender-inclusive Yalamu, P., Chua, C., Doube, W., 2020. Does indigenous culture affect one’s view of
requirements. In: International Conference on Conceptual Modeling. Springer, pp. an LMS interface: A png and Pacific islands students’ perspective. In: Proceedings
395–409. http://dx.doi.org/10.1007/978-3-030-89022-3_31. of the 31st Australian Conference on Human-Computer-Interaction. OzCHI ’19,
Office of the Chief Scientist, 2020. Australia’s STEM Workforce: Science, Tech- Association for Computing Machinery, New York, NY, USA, pp. 302–306. http:
nology, Engineering and Mathematics. Commonwealth Government of Aus- //dx.doi.org/10.1145/3369457.3369483.
tralia, [Online]. Available https://www.chiefscientist.gov.au/news-and-media/ Yarnell, L.M., Neff, K.D., Davidson, O.A., Mullarkey, M., 2019. Gender differences in
2020-australias-stem-workforce-report. self-compassion: Examining the role of gender role orientation. Mindfulness 10,
Oliwa, R., 2021. The process of designing the functionalities of an online learning 1136–1152. http://dx.doi.org/10.1007/s12671-018-1066-1.
platform–a case study. Teach. English Technol. 21 (3), 101–120, Available: https: Yatsko, A., Suslow, W., 2016. Areas of computer science. In: Insight Into Theoretical
//eric.ed.gov/?id=EJ1314776. and Applied Informatics. Walter de Gruyter GmbH, Germany.
Open Educational Resources (OER) Commons, 2021. Cognitive styles. [Online.]
Available: https://oercommons.org/courses/interactive-tool-cognitive-styles-slider-
3/view.
Ors, E., Palomino, F., Peyrache, E., 2013. Performance gender gap: Does competition Lyndsey O’Brien is studying Master of Information Technology in Monash University.
matter? J. Labor Econ. 31 (3), 443–499. http://dx.doi.org/10.1086/669331. She is most interested in, and enjoys the challenge of, the qualitative research,
Østergaard, C.R., Timmermans, B., Kristinsson, K., 2011. Does a different view create particularly thematic analysis. Lyndsey is a dedicated student who has achieved the
something new? The effect of employee diversity on innovation. Res. Policy 40 (3), top mark in multiple subjects, including Software Engineering. She puts her full focus
500–509. http://dx.doi.org/10.1016/j.respol.2010.11.004. into learning as much as she can and always working to a high standard.
Parker, K.R., 2010. Selecting software tools for IS/IT curricula. Educ. Inf. Technol. 15,
255–275. http://dx.doi.org/10.1007/s10639-010-9126-8.
Dr. Tanjila Kanij is Lecturer at Swinburne University of Technology. She has completed
Ribeiro, D., Lima, R., França, C., Souza, A., Silva, I., Pinto, G., 2023. Understanding
her Ph.D. from Swinburne University of Technology in 2013. Her research interests in-
self-efficacy in software engineering industry: An interview study. In: Proceedings
clude understanding di erent human aspects of software engineering and understanding
of the 27th International Conference on Evaluation and Assessment in Software
how diverse user requirements can be better captured and modelled to make software
Engineering. EASE ’23, Association for Computing Machinery, New York, NY, USA,
solutions more widely adaptable. Her research focus also includes gender diversity
pp. 101–110. http://dx.doi.org/10.1145/3593434.3593467.
within computer science education environment and software engineering workplace.
Rodríguez-Pérez, G., Nadri, R., Nagappan, M., 2021. Perceived diversity in software
Contact her as tkanij@swin.edu.au.
engineering: a systematic literature review. Empirical Softw. Eng. 26 (5), http:
//dx.doi.org/10.1007/s10664-021-09992-2.
Rotter, J.B., 1966. Generalized Expectancies for Internal Versus External Control Professor John Grundy received the B.Sc. (Hons), M.Sc., and Ph.D. degrees in
of Reinforcement. In: Psychological Monographs: General and Applied, vol. 80, computer science from the University of Auckland, New Zealand. He is an Australian
(1), American Psychological Association, pp. 1–28. http://dx.doi.org/10.1037/ Laureate fellow and a professor of software engineering at Monash University, Mel-
h0092976. bourne, Australia. He is an associate editor of the IEEE Transactions on Software
Saad, L., 2017. Gallup vault: A sea change in support for working women. https://news. Engineering, the Automated Software Engineering Journal, and IEEE Software. His
gallup.com/vault/214328/gallup-vault-sea-change-support-working-women.aspx. current interests include domain-specific visual languages, model-driven engineering,
Şahin, M., Yurdugül, H., 2022. Learners’ needs in online learning environments and large-scale systems engineering, and software engineering education. More details about
third generation learning management systems (LMS 3.0). Technol. Knowl. Learn. his research can be found at https://sites.google.com/site/johncgrundy/. Contact him
27, 33–48. http://dx.doi.org/10.1007/s10758-020-09479-x. at john.grundy@monash.edu.

18