09 Nov Student Engagement: Attempting Better Measures of a Subjective Experience
with Ben Keeler
In some of our previous blog posts we have discussed topics such as School Climate, Social-Emotional Learning, Digital Literacy, and Blended Learning. In future posts we plan on delving into the debates surrounding 21st Century learning, Parent-Teacher Relationships, Teamwork/Collaboration and Academic Motivation. Today’s post is about Student Engagement.
The first step in measurement is always to agree on a clear definition. After sorting through the best academic literature on the topic, we’ve arrived at this one as a starting point:
Students are engaged with learning when they are involved in their work, persist despite challenges and obstacles, and take visible delight in accomplishing their work. Student engagement also refers to a student’s willingness, need, desire and compulsion to participate in, and be successful at, the learning process.
It is not always easy to notice students who are engaged. It’s often easier to point out the ones who aren’t. Students texting during class, interrupting the teacher or refusing to take notes are some easy signs of disengagement. This poses some problems for measurement, and as a result, many studies have fallen back on measuring engagement indirectly through more traditional metrics.
For instance, in a research article by Leah Taylor and Jim Parsons at Arizona State University, they point out that traditional strategies for measuring student engagement “have predominantly focused on quantitative data such as attendance, standardized test scores, and truancy or graduation rates. The majority of these measures track levels of achievement (outcomes such as high scores, full attendance for the year) but not levels of student engagement in learning (interest, time on task, enjoyment in learning).”
Attempting to improve on this issue, The Regional Educational Laboratory at the Institute for Education Sciences (IES) suggests utilizing “classroom observations of one to four hours occurring two to five times a year” as well as a 121-item student self-report questionnaire taken a handful of times a year.
We wanted to take a much different approach, especially since we’re in the business of providing better measurement for schools while minimizing disruption to the daily routine. We couldn’t exactly convince a school to have us come in and do classroom observations for them. We also think a 121-item questionnaire in one sitting would be trying on an adult, let alone a teenager.
So we’re starting with self-report data, much like the IES questionnaire, sent multiple times throughout the year, but instead of 121 questions at once, we’re sending a couple a week, right to mobile devices. We understand that academic engagement can come and go, therefore our data collection is constant so that we can track the ebb and flow of responses over time.
Some sample questions from our Student Engagement module include:
- Today I was interested in the things I learned in school.
- Today, being at school was important for achieving my future goals.
- Today, I liked it when school subjects made me think hard and figure things out.
By asking questions such as these more than once over the course of a year, we end up being able to report back to schools very nuanced graphs showing student engagement over time. This is literally impossible to do with a once- or twice-yearly survey of student engagement, such as the IES questionnaire mentioned above.
Yes, engagement is a subjective experience, something that’s sometimes hard to articulate, but it’s much more accurately measured if we ask an individual these questions multiple times. We can find both an average of all of these questions (which will provide a better measurement than a one-time survey could) but also a nuanced look at the changes in student engagement throughout the year. This is where we need to head if we ever want to illuminate what’s really going on in students’ lives day-to-day in schools.