Background

Sense of Belonging is an important predictor of student success, retention and well-being. Historically, Sense of Belonging at JMU has been measured using the 6-item Bollen and Hoyle (1990) Perceived Cohesion Scale.

  1. I feel a sense of belonging to JMU
  2. I am happy to be at JMU
  3. I see myself as a part of the JMU community
  4. I am enthusiastic about JMU
  5. JMU is one of the best schools in the nation
  6. I feel that I am a member of the JMU community

Benefits

Drawbacks

Objectives

The objective of this study was to compare three different methods of measuring sense of belonging to elucidate gaps in current practices and inform changes to sense of belonging data collection in the future.

Methods

Students were recruited to take a survey through the SCOM Research Pool in April-May 2025. Students responded to the six items measured on Likert-scale Perceived Cohesion Scale (Bollen & Hoyle, 1990). Scores on the Perceived Cohesion Scale could range from 6 to 54.

Students also responded to an open ended question: Overall, do you feel like you belong at JMU? Please explain why or why not?

743 students started the survey; of those responses, 637 were used for analysis (85.7%). Responses were removed if the person did not complete the survey or did not provide an actual response to the open-ended question of interest (e.g., wrote “asdf”, “xxxx”), or did not provide an explanation (e.g., just wrote “yes” or “no”).

Analysis

Sentiment analysis of open-ended question

We used the analyzeSentiment() function from the SentimentAnalysis package in R to assess the emotional tone of textual data. This function evaluates the sentiment of each text entry by referencing established sentiment dictionaries and algorithms, assigning scores that indicate whether the content expresses a positive, negative, or neutral sentiment. Scores range from -1 (most negative) to 1 (most positive), with 0 indicating neutral responses. These scores were then used to quantify and compare sentiment trends across the dataset. A categorical variable was created from the sentiment value with values less than 0 assigned as Negative, 0 assigned to Neutral and values greater than 0 assigned to Positive.

Manual coding of open-ended question

Responses were categorized as positive, neutral or negative after reading through them. The neutral category included responses that were ambivalent or that did not have a clear sentiment.

Sentiment and Perceived Cohesion Comparison

Sentiment scores were compared to scaled Perceived Cohesion Scale scores (-1 to 1 range) through correlations and examining score distribution using density plots. Categories (negative, neutral and positive) were created from the scaled PCS scores using the same methodology as sentiment categorization.

Sentiment and Manual Coding Comparison Sense of belonging categorization was compared between PCS categories, sentiment categories and manually coded categories.

Results

Correlations Between Sentiment and PCS

Correlation between Sentiment and PCS
Numeric Sentiment Value PCS Total
Numeric Sentiment Value 1.00 0.17
PCS Total 0.17 1.00

There is a weak correlation between student responses on the Perceived Cohesion Scale and the numeric sentiment value assigned to their free response.

Sentiment scores range from -1 to 1 with higher scores indicating more positive sentiment. The Perceived Cohesion Scale data were scaled to be the same range of values as sentimental values (-1 to 1). The most negative PCS value was assigned a scaled value of -1, while the most positive PCS value was assigned a scaled value of 1. This allows us to directly compare the distribution of these two variables.

This first plot shows the value distributions for numeric sentiment and scale PCS, as well as the frequency of that value appearing in the data. The PCS scale shows greater variance compared to the numeric sentiment which is largely concentrated between 0 and 0.5.

The second plot presents the same information, but slightly differently. We can see the range of response for the PCS is much wider than the range of sentiment values.

Because there is a larger right (positive) skew in the PCS data compared to the sentiment data, the mean is significantly higher.

Measure Mean
PCS Score 0.46
Sentiment Value 0.22

Sentiment Category and Perceived Cohesion Scale

Mean values on the Perceived Cohesion Scale were different between the sentiment categories assigned based on the numeric sentiment value (Negative < 0, Neutral = 0 and Positive >0).

Mean PSC Scores
Sentiment Mean
Negative 34.15
Neutral 39.71
Positive 42.75

There were significant differences in PCS scores between responses labeled as positive and negative, and responses labeled positive and neutral.

Tukey Post-Hoc Values
Comparison Mean Difference Lower CI Upper CI P Value
Neutral-Negative 5.55 -0.86 11.96 0.10
Positive-Negative 8.60 2.59 14.61 0.00
Positive-Neutral 3.05 0.45 5.65 0.02

Comparing Categorical Methods

Perceived Cohesion scaled values were coded into a categorical variable (Negative < 0, Neutral = 0 and Positive >0).

Distribution of Categorical Responses by Coding Method
Sentiment Manual Code PCS Category Text Sentiment
Negative 36 (5.9%) 86 (14%) 13 (2.1%)
Neutral 76 (12.4%) 14 (2.3%) 78 (12.7%)
Positive 501 (81.7%) 513 (83.7%) 522 (85.2%)

The following table specifically shows cross-tabluations between manual coding the text sentiment categorization.

Manual Codes and Text Sentiment Categorization
Text Sentiment Negative Neutral Positive
Negative 3 (8.3%) 7 (9.2%) 3 (0.6%)
Neutral 6 (16.7%) 14 (18.4%) 58 (11.6%)
Positive 27 (75%) 55 (72.4%) 440 (87.8%)

Below are examples of statements that were categorized as ‘Positive’ during the sentiment analysis, but were coded as negative during the manual process.

Responses with Incongruent Categories
No. I just don’t think it is the right atmosphere for me.
I feel I don’t belong at JMU because I have tried making friends and haven’t made any that stick and when I am at events, I feel like I don’t belong to any group.
No. The campus is beautiful and has a lot to offer but I just don’t believe it is the spot for me.
No, just because living on campus isnt for me
I don’t feel a sense of belonging at JMU. A lot of the culture at this school seems centered around the fraternities and sororities. As an alternative, I could join one of the clubs. While there is a variety of clubs, I have not felt a connection to any so far. This school appeals well to its target audience. I consider myself excluded from that target audience.
no, I haven’t found a group yet. I have friends but they also have other friends. As in, im not their favorite or just an option to hang out with. I want to be more involved but can’t find any groups.
I dont think I belong here because of the people I feel like I am different here. I am a much more southern person and JMU has not felt like that to me.
I feel like I don’t belong because I don’t have many friends on campus and it feels like no one has my back. It can get really lonely.
I don’t feel like I belong at JMU because it feels like I don’t have people who have my back, I don’t have many friends and it gets lonely.
To be honest, I don’t really feel like I belong. I’ve noticed that people don’t usually approach me unless I make the first move, which can be exhausting. Coming from a different background and being multiracial, it sometimes feels like I don’t fully fit in, and that makes it harder to connect. I’ve chosen to stay true to who I am rather than trying to blend in, but that can make things feel even more isolating. Maybe it’s just because I’m a freshman and still trying to find my people but right now, it’s tough.
No, I haven’t been able to make a single friend here every time it seems like I’ve maybe made a friend all they want are classwork answers or help on an activity and then after the semester it’s like we’ve never talked at all and I see them in their stories and instagram posts having fun with their friends and I just wonder what I did wrong to have made absolutely zero friends. It’s my birthday today and I have not even one friend on jmu campus to congratulate me or celebrate with I’ve never felt more alone in my entire life here.
I do not feel like I belong at JMU, but it is a good school.
I feel that there are some people that I am friends with but mostly I am alone.
Personally, I do not have a very strong sense of belonging at JMU. It has been exceptionally hard for me to connect with people here, whereas I did not struggle this much in high school and community college. I feel like maybe it has something to do with the major I chose (Biology), as I do not feel such a strong bonding energy with other STEM majors, who I feel I am very different from.
No, coming to JMU all I heard about was the community. While most people are friendly, a lot of people are only close with groups they came in here with or in their Greek life groups.
Not always because I don’t feel like I have very many friends or that I am liked by others which doesnt make me feel very comfortable
no I just moved to the state
I’m not white And I do not party
I feel like I don’t belong because I don’t fit in with the rest of the more party focused people at the school.
I don’t feel like I belong here at JMU because I am not interested in organizations. I am just trying to get into my program and graduate with my degree.
I don’t feel like I belong at JMU because I am just here to get my degree so I can start working.
Kind of, its just alot of factors I dont like about it.
Not entirely. The majority of this school is white and because of this I feel somewhat left out in certain spaces. It feels weird being the only person of color in a class and to not be able to talk to everyone I meet about certain things
No, I don’t feel like I fit in. I tried joining things, but I was always rejected, so I lost hope. The people I consider friends aren’t friends. They always talk badly about me and make fun of me in front of me. I have a kind heart, so it wouldn’t make sense to talk back because I don’t wanna hurt anyone’s feelings. I was told I can’t join other orgs because the people I’m friends with told me that they wanted their own thing and didn’t want me to take it away from them.
Feel like I don’t belong can’t seem to find my people
Not really, I don’t have a particular sense of belonging, but I do have people I have become close with for sure. But often times I feel the general JMU population is very different from me.
Honestly, no because I dont like the concept that JMU is like the number one party school in all of virginia and that really sit right with me. But also if you dont drink or if your not part of a fraternity you are kind of a odd ball at JMU.
eh sometimes, it really depends because its been up and down

These statements should in fact be coded as negative.

We can also look at how manual coding compares to the categories derived from the scaled PCS variable. There are fewer responses coded as the positive per the PCS that were manually coded as negative.

Manual Codes and PCS Categorization
PCS Category Negative Neutral Positive
Negative 26 (72.2%) 25 (32.9%) 35 (7%)
Neutral 1 (2.8%) 6 (7.9%) 7 (1.4%)
Positive 9 (25%) 45 (59.2%) 459 (91.6%)

Here are the statements that were mis-categorized by the PCS category.

Responses with Incongruent Categories
no, I haven’t found a group yet. I have friends but they also have other friends. As in, im not their favorite or just an option to hang out with. I want to be more involved but can’t find any groups.
Not always because I don’t feel like I have very many friends or that I am liked by others which doesnt make me feel very comfortable
I don’t feel like I belong here at JMU because I am not interested in organizations. I am just trying to get into my program and graduate with my degree.
Kind of, its just alot of factors I dont like about it.
No, I don’t feel like I fit in. I tried joining things, but I was always rejected, so I lost hope. The people I consider friends aren’t friends. They always talk badly about me and make fun of me in front of me. I have a kind heart, so it wouldn’t make sense to talk back because I don’t wanna hurt anyone’s feelings. I was told I can’t join other orgs because the people I’m friends with told me that they wanted their own thing and didn’t want me to take it away from them.
Feel like I don’t belong can’t seem to find my people
eh sometimes, it really depends because its been up and down

Categorization Agreement

The following section explores sentiment value and PCS scores in the subset of participants with sentiment category agreement across the three methods (n = 408, 66.5%).

Similar to what we saw in the full sample, PCS scores skew more heavily to the right, while sentiment values are clustered between 0 and 0.5.

Again, this results in a substantially higher mean for the PCS.

Measure Mean
PCS Score 0.63
Sentiment Value 0.27

Implications & Next Steps

Sentiment analysis is not a perfect substitute for Sense of Belonging scales or manual qualitative analysis of open ended questions. However, the dramatic differences between PCS and sentiment values (even in the subset with congruent sentiment categorization) suggests the PCS is not capturing a complete picture of sense of belonging. This suggests, in order to more fully understand sense of belonging, we may need explore alternative or supplemental strategies to obtain this information.

Thematic analysis of the open ended responses analyzed for this study revealed three areas for further exploration:

  1. Academic confidence
  2. Social connections (e.g., friendships)
  3. Institution connections (e.g., organizations, clubs)

We plan to conduct another study measuring these three constructs and exploring their relationship with academic and general wellbeing outcomes.

Appendix: Demographic Breakdowns

The small N’s among demographic subgroups limit our ability to draw conclusions. Because smaller N’s produce less reliable estimates, confidence intervals are included in the visualization to show the range of uncertainty with each estimate for subgroups. Interpret estimates with wide confidence intervals and small N’s with caution.