Improving Flipped Academic Advising: Designing Materials for the Technology-Based Higher Education Landscape
Abstract
This study investigated best practices in creating and providing advising materials to students in an asynchronous online environment (i.e., flipped advising). Specifically, we examined the effects of creating advising tutorials in different modalities (i.e., text vs. video), and providing learning support via a pre-tutorial before engaging with the flipped advising content (i.e., pre-tutorial vs. no pre-tutorial). We found a significant interaction of these factors such that participants who watched video tutorials benefitted from having a pre-tutorial, but those who read text tutorials were hindered by the pre-tutorial. These results suggest that pre-tutorials warrant additional investigation, especially in the context of technology-driven flipped academic advising.
Many higher education instructors use flipped learning frameworks to incorporate technology into their coursework; however, flipped learning can also apply to academic advising (Leonard, 2008; McIntosh et al., 2020; Steele, 2018). In the flipped classroom, direct instruction is delivered to the individual space (e.g., at home, outside of class) rather than the group space (i.e., lecture hall). The individual space created by flipped learning—where first contact with new material occurs—integrates well with the use of technology (i.e., instructional video tutorials, interactive games, etc.; Talbert, 2017). Students can interact with technologically aided lessons on their own time, then spend course time on more meaningful interactions (e.g., group conversations, active learning techniques, etc.) alongside the instructor. These flipped classroom principles also apply to academic advising—students can interact with structured advising material at home that is designed to prepare them to have a more meaningful, personalized, and productive conversation during the advising appointment.
To implement flipped advising effectively, Steele (2018) recommended using a Learning Management System (LMS) to create modules (text, video, etc.) on course registration, institutional policies, and campus technologies. Doing so provides a structured environment where students can find information before an in-person advisor meeting. When flipped advising is done well, students find it to be more effective than traditional in-person advising (Amini et al., 2018). However, this finding prompts a question: what does it mean to design flipped advising practices well? This study investigates how best to design materials for the individual learning space to prepare students for their in-person advising sessions. Specifically, we investigated the effect of:
-
modality of the materials (text vs. video).
-
provision of scaffolding materials that equip students to learn the advising content from each modality (inclusion of learning tips vs. no learning tips before viewing the instructional content).
Thus, our research questions were focused on academic advising materials. Specifically:
-
Would students learn better from video or text?
-
Would prefacing materials (video or text) with pre-tutorials improve student learning?
We focused on Gen Z—the current generation of learners who are situated in an increasingly technology-based learning world. The answers to these questions will help advisors optimize student learning and align with Gen Z learning preferences.
Flipped Advising: Which Modality Works Best?
Technology’s rapid advancement in recent years has created new methods of interaction for academic advisors (e.g., email, social media), including flipped advising frameworks. Gen Z students prefer e-resources like text and especially video (Nicholas, 2020; Seemiller & Grace, 2016; Twenge, 2017). Still, there is no single agreed upon best approach; what is newest is not always best. Because learning is affected by different modalities in a flipped advising framework and given that Gen Z’s stated general preference was for video materials (Kosterelioglu, 2016; Lloyd & Robertson, 2012), we compared advisees’ learning from online text-based tutorials vs. video-based tutorials.
How Can Advisors Design the Most Effective Materials for Flipped Advising?
Because both NACADA: The Global Community for Academic Advising (Crookston, 2009) and the Council for Advancement of Standards (CAS) consider academic advising a “teaching and learning activity” (Steele, 2018, p. 61), it is important to consider how best to support advisees’ learning in a flipped format. Flipped advising relies both on advisees interacting with assigned material prior to their advising session and the conversations that occur directly with the academic advisor. However, if students do not effectively learn from their assigned materials prior to their advising session, advisors cannot optimally assist them. When students do not know how to optimize their self-paced learning, they may fail to learn or even be discouraged from interacting with them at all. Indeed, though Gen Z undergraduates have been learning in varying educational contexts for most of their lives, many do not know about efficient learning and study techniques (McGuire, 2015). For example, many students use inefficient study techniques, such as re-reading their notes or textbook without deeply processing the content (Putnam et al., 2016). Thus, teaching students how to optimize their learning from different types of materials (e.g., texts, videos) before they engage with those materials may positively impact their learning (Clarke et al., 2005; Mayer et al., 2002; Mayer & Pilegard, 2014).
Implementing pre-trainings during flipped advising may be one way to support student learning in this environment. Pre-trainings provide background/conceptual knowledge before a targeted lesson, which facilitates students’ information processing and can prevent cognitive overload (i.e., when a novice learner is overwhelmed with information; Paas & Sweller, 2014). Pre-training has already been cited as an effective means of reducing cognitive overload for a variety of topics (e.g., math and physics problems, complex systems, etc.; Mayer & Pilegard, 2014). However, our study is different. We examined whether providing students with information about how to optimize their learning from the medium itself, rather than from the topic, would benefit students. This idea was born from research on developing students’ metacognitive learning strategies, which are beneficial for learning but need to be specifically taught to students (Stanton et al., 2021; Tanner, 2012).
Instead of pre-training, where the goal is to provide prior knowledge about a topic to reduce the potential of cognitive overload, we propose the concept of a pre-tutorial, which provides medium-specific interaction instructions to optimize learning. That is, students are provided with instructions about how best to learn from text versus video media before engaging with material. This approach is similar to how 3D movies tell viewers when to put on their glasses, or how the “Let’s All Go to the Lobby” jingle reminds the audience of concessions; likewise, pre-tutorials instruct participants to optimize their medium-specific experience. While a movie viewer may think they know how to optimize their viewing experience, it is not until instructions are given and followed that this is possible; the same may be true of learning.
Consider notetaking. While many students may believe they are taking notes on learning material effectively, research suggests that most students do not take complete notes (Kiewra et al., 2018). Thus, students may benefit from a pre-tutorial on notetaking grounded in evidence-based research whose script instructs them to: take complete notes (Kiewra et al., 2018), take handwritten notes instead of typed notes (Morehead et al., 2019; Mueller & Oppenheimer, 2014), and take more notes rather than less (Johnstone & Su, 1994). Because students who take complete notes regularly achieve more (Kiewra, 1985), it stands to reason that instructing students in how to engage in effective notetaking may improve their learning.
Notetaking is just one element of learning, but pre-tutorials can be applied to learning from any medium, like text or video. There are other specific and evidence-based instructions that students may benefit from, such as watching videos on laptops or desktops rather than mobile devices (Rigby et al., 2016) or using the functions of the video player (e.g., pause, rewind, etc.) to ensure information is not missed (Tuncer et al., 2020). Thus, we additionally investigated the potential benefits of providing students with these format-specific, evidence-based tips for optimizing their self-directed learning. Ideally, providing such tips in a pre-tutorial would boost learning and better prepare advisees for their in-person academic advising meetings.
Improving Student Perceptions of Academic Advising
The flipped advising framework does not seek to replace the academic advisor with technology; rather, it should enhance the in-person advising session. An instructional video cannot converse with students, draw connections between breadth and major coursework, or determine the best solution for an advisee. Thus, advisees must feel motivated to engage with their academic advisors and not rely entirely on at-home learning materials. Trust is vital. The more advisees trust their advisors, the more they will interact with them (Petress, 1996). One factor that influences trust is the perception of competence (Ohrt, 2018), which can be communicated through provision of thoughtful advising materials within a flipped framework. Given this idea, we chose to investigate whether the material’s format and provision of pre-tutorials altered student perceptions of their academic advisor. We additionally examined student perceptions of their own ability to complete the advising tasks taught in the tutorial, as well as engagement with the materials.
The Present Study
While much thought has been devoted to designing support materials for flipped classrooms (Bergmann & Sams, 2016; Carbaugh & Doubet, 2016), there has been little discussion about how academic advisors should produce such materials for their advisees (McKenna et al., 2014). Thus, we investigated not only the effect of the modality of the instructional materials, but also whether scaffolding the presentation of those materials with pre-tutorials benefited students.
As a first step, we tested if advising video tutorials improved scores on an eight-question, multiple-choice academic advising post-quiz compared to more traditional, text-based (but still online) tutorials. Based on the research in favor of video’s potential for improving and enriching learning (Kosterelioglu, 2016; Lloyd & Robertson, 2012), we hypothesized that students who interacted with instructional video tutorials rather than text tutorials would earn higher scores on the post-tutorial quiz (Research Question #1).
Considering that pre-training has been shown to improve learning across a wide variety of subjects (e.g., Clarke et al., 2005; Mayer et al., 2002), we also investigated whether prefacing academic advising instructional tutorials with pre-tutorials would improve scores on the post-tutorial quiz. We therefore additionally hypothesized that scaffolding instructional tutorials (i.e., text or video) with pre-tutorials would positively impact students’ recall of how to perform a required advising-related task compared to when they did not receive such pre-tutorials (Research Question #2). We further hypothesized that there would be an interaction such that students would benefit the most from video tutorials with a pre-tutorial compared to all other conditions, given the finding that students’ attention to videos typically wanes quickly without intervention like our proposed pre-tutorial (Wammes & Smilek, 2017).
Finally, because the purpose of advising resources is to facilitate better in-person interactions, it is important to investigate if differing formats and pre-tutorials result in improvements in trust towards academic advisors, as well as increases in self-confidence to accomplish similar tasks in the future. As such, we collected data using an exploratory survey that asked participants how their assigned condition might impact perceptions of their academic advisor, their level of engagement, and their confidence to accomplish future advising-related tasks. The entire project was pre-registered on Open Science Framework and IRB approval at the first author’s home institution was obtained (IRB Protocol: HS 20-0908).
Method
Participants
Participants were recruited in two stages through the university’s psychology participant pool. First, after collecting 40 participants, we conducted a preliminary effect size analysis on the interaction. This effect size was used in a power analysis set to 80% to determine the number of additional participants the experiment required, which was determined to be 35 per each of four groups—a total of 140 participants. Based on this power analysis, we recruited 142 students, all born between 1995–2010. Our minimum participant age was 18, the maximum age was 26, and the average age was 18.97. Additional demographic information about the participants’ gender, first-generation student status, and racial and ethnic backgrounds appears in Table 1. Participants earned partial course credit for their participation.
Design
This study utilized a 2 (tutorial format: text vs. video) × 2 (scaffolding: pre-tutorial vs. no pre-tutorial) fully between-subjects design, with all participants randomly assigned into one of the four conditions. Before watching the tutorial, participants in the pre-tutorial condition were instructed how to optimize their learning via utilization of various relevant techniques. For example, participants in the text tutorial condition were instructed how to take effective notes from written materials (e.g., Mueller & Oppenheimer, 2014); while participants in the video tutorial condition were instructed to utilize the unique features of the video to ensure they could fully absorb the material (Tuncer et al., 2020). Participants in the no pre-tutorial condition proceeded directly to the tutorial in their assigned format.
Materials and Measures
Materials
The study utilized two video and two text tutorials designed to teach students how to create a term plan (i.e., a students’ proposed course schedule, which is submitted to an academic advisor through an online portal for approval). This approach made sense as term plans are essential to student success. The first video tutorial (video, no pre-tutorial condition) was created using QuickTime’s audiovisual screen recording feature, hereafter referred to as the screencast. The screencast overviewed how to create and submit a term plan using a faux student profile. Students co-scripted and voiced this narrative, which advising professionals checked for accuracy on Adobe Premiere. It was 3:07 minutes in duration.
The second video (video, pre-tutorial condition) was nearly identical to the first, but included a pre-tutorial that provided pointers about how students could optimize their learning from the video (Rigby et al., 2016; Tuncer et al., 2020). For instance, the video pre-tutorial recommended that participants take advantage of the functions of the video player (e.g., pause, rewind, etc.) to generate a complete set of notes (Tuncer et al., 2020). This pre-tutorial incorporated stock footage from www.videoblocks.com to match the narrative voiceover (i.e., the footage shown matches the voiceover description). Figure 1 shows an example screenshot from one of the videos. The pre-tutorial video and instructional video tutorial were edited together using Adobe Premiere, resulting in a duration of 5:01 minutes.
Citation: NACADA Journal 44, 2; 10.12930/NACADA-23-18
Two text tutorials, with text identical to these video voiceovers, provided overviews of creating a term plan. These text-only tutorials included still images from the video, a choice made because the home institution currently uses this format for numerous “how-to” guides. The first text tutorial (text, no pre-tutorial condition) included an introduction, a conclusion, 10 instructional steps (513 words), and 11 still-images. Five pilot participants found that the text materials took approximately 2 minutes to read. The second text tutorial (text, pre-tutorial condition) covered the same content but was prefaced by a pre-tutorial intended to improve students’ learning from the written text. The pre-tutorial was 294 words and provided students with 11 evidence-based tips designed to optimize learning, such as taking notes with a pen and piece of paper rather than on a computer (Mueller & Oppenheimer, 2014). The same five pilot participants read these instructions with the integrated pre-tutorial and found that it took about 3:30 minutes to read.
Measures
We collected demographic information on participants’ gender, age, racial background, ethnicity, current major, family income, first-generation status, and highest grade of school completed. This data ensured that the participants were representative of campus demographics, that our viewers had limited experience with term plan creation, and that they all identified as Gen Z (i.e., born between 1995–2010). Out of all participants, 81 had previously created a term plan, whereas 61 had not.
In addition, an eight-question multiple-choice quiz that overviewed the tutorial’s content was utilized to collect data on participant learning from the tutorial. The questions asked participants to identify scenarios related to creating a term plan (e.g., “If you do not create and submit a term plan … ;” Answer choices: 1 = Your advisor will contact you, 2 = You will be charged a delinquent fee, 3 = A term plan hold will prevent you from registering, or 4 = None of the above; solution: 3). See Appendix A for all other questions and answers. The total number of questions correct out of eight became the dependent measure of learning.
Data on student perceptions of academic advisors based on the study’s materials and on participants’ confidence to complete additional tasks related to their major were also collected for exploratory analyses using an internally developed 15-question survey. This survey asked students to answer on a scale of 1 = Strongly Disagree to 5 = Strongly Agree with statements such as: “If my academic advisor provided similar instructional tutorials to help me, I would perceive them as competent,” and “I understand the role my academic advisor plays in the term plan creation process.” See Appendix B for all other exploratory survey questions.
Procedure
Students completed this study remotely in Qualtrics using their individual computers or laptops. After providing their electronic informed consent, participants read instructions that explained they would be participating in a study about improving instructional tutorials for academic advising and that they would later be tested on its content. They then completed the demographics questionnaire and were given directions about how to create and submit a term plan in one of the following formats:
-
Text with still images of the process (text, no pre-tutorial).
-
Text with still images prefaced by a text pre-tutorial (text, pre-tutorial).
-
A video tutorial (video, no pre-tutorial).
-
A video prefaced by an embedded video pre-tutorial (video, pre-tutorial).
Then, participants completed a 5-minute distractor task (i.e., an interactive memory delay game, Bubble Shooter) to incorporate a brief delay between the learning and testing phase. Participants were then given the eight-question multiple-choice quiz to assess their knowledge of creating a term plan (i.e., learning from the tutorial). Next, participants completed the 15-question exploratory survey. Finally, participants were debriefed on the experiment and thanked for their participation. The duration of the experiment was approximately 30 minutes. Figure 2 shows a flow chart of the experimental design.
Citation: NACADA Journal 44, 2; 10.12930/NACADA-23-18
Results
Learning from the Tutorial
A 2 (format: text vs. video) × 2 (scaffolding: pre-tutorial vs. no pre-tutorial) fully between-subjects ANOVA was conducted. The analysis revealed no main effect of format on final quiz score, F(1, 138) = 0.27, MSE = 2.20, p = .61, ŋp2 = .002. However, there was a significant main effect of scaffolding such that participants who received pre-tutorials performed better on the final knowledge quiz (M = 5.24, SD = 1.65) than participants who did not receive pre-tutorials (M = 4.61, SD = 1.56), F(1, 138) = 6.24, MSE = 2.20, p = .01, ŋp2 = .04. Most importantly, there was a significant interaction effect such that participants who received video tutorials benefitted from having a pre-tutorial, but those who received text tutorials were hindered by the pre-tutorial, F(1, 138) = 11.67, MSE = 2.20, p = .001, ŋp2 = .08 (see Figure 3). Note that this significant interaction remained when participants, who created a term plan before, were removed from the analysis, suggesting that familiarity with the concept of making a term plan does not impact the interaction.
Citation: NACADA Journal 44, 2; 10.12930/NACADA-23-18
Exploratory Analyses
We also collected data using an internally developed, exploratory fifteen-question multiple-choice survey. (Note: because this was an exploratory survey that did not use several items to measure constructs, we opted not to conduct reliability analyses.) However, only three questions (one per exploratory analysis) were analyzed because they are the most relevant to our initial hypotheses. These questions addressed:
-
participants’ perception of their academic advisor if said advisor were to provide materials like the ones experienced in the study to their advisees;
-
level of engagement with their assigned condition;
-
confidence to accomplish other administrative tasks if presented with similar instructional tutorials in the future.
First, we investigated if instructional tutorial format impacts participants’ perception of their academic advisor’s competence via a 2 (format: text vs. video) × 2 (scaffolding: pre-tutorial vs. no pre-tutorial) between-subjects ANOVA. The analysis revealed no significant main or interaction effects (all p > .05).
Second, we investigated whether format impacted participants’ engagement with their tutorial. A 2 (format: text vs. video) × 2 (scaffolding: pre-tutorial vs. no pre-tutorial) between-subjects ANOVA was run to determine how engaging participants perceived their assigned tutorials to be. The analysis revealed that participants perceived text tutorials as more engaging (M = 3.64, SD = 1.04) than videos (M = 3.13, SD = 1.16), F(1,138) = 8.08, MSE = 1.18, p = .005, ŋp2= .06. There was no significant main effect of pre-tutorial on perceived engagement, F(1,138) = 2.31, MSE = 1.18, p = .131. The analysis did, however, yield a significant interaction effect such that participants who did not receive a pre-tutorial prior to their video tutorial were much less engaged than those who did receive a pre-tutorial, but a similar effect was not found for providing pre-tutorials before text materials, F(1,138) = 3.96, MSE = 1.18, p = .049, ŋp2= .028 (See Figure 4a).
Citation: NACADA Journal 44, 2; 10.12930/NACADA-23-18
Lastly, to investigate if access to similar instructional tutorials would impact one’s confidence completing other administrative tasks required to finish their major, we conducted a 2 (format: text vs. video) × 2 (scaffolding: pre-tutorial vs. no pre-tutorial) between-subjects ANOVA. The analysis revealed no significant main effect of format on confidence to complete similar tasks, F(1,138) = 0.50, MSE = 0.57, p = .48, ŋp2= .004. The analysis also revealed no significant main effect of pre-tutorials on participant confidence, F(1,138) = 3.06, MSE = 0.571.74, p = .082, ŋp2= .022. There was, however, a significant interaction effect such that participants who received video tutorials had higher confidence when they received a pre-tutorial compared to when they did not, but those who received text tutorials were similarly confident with or without a pre-tutorial, F(1, 138) = 4.85, MSE = 0.57, p = 0.03, ŋp2= .034 (See Figure 4b).
Discussion
This research was a crucial first step in examining how to improve student learning from flipped academic advising materials, a topic not yet extensively explored (Amini et al., 2018; McKenna et al., 2014; Steele, 2018). In a single study of a flipped advising model with Gen Z undergraduates who represent a large, generalized advising caseload, we explored the role of 1) format of instructional tutorials (text vs. video) and 2) pre-tutorials (i.e., brief interventions that provided explicit instructions for engaging with a particular tutorial format) in improving learning from academic advising materials. Based on prior work about the engagement of video materials (e.g., Kosterelioglu, 2016; Lloyd & Robertson, 2012) and benefit of pre-training to support learning of complex material (e.g., Clarke et al., 2005; Mayer et al., 2002), we hypothesized that: 1) students who watched video tutorials would learn more than those who read text tutorials; 2) students who received pre-tutorials (our version of pre-training) would learn more than those who did not; and 3) students who received pre-tutorials before a video tutorial would learn most of all. We found partial support for these hypotheses.
Our first hypothesis was not supported. Students who interacted with instructional video tutorials rather than text tutorials did not earn higher scores on the post-quiz overall. Our second hypothesis, however, was supported: students who received pre-tutorials had significantly higher scores on the post-quiz than those who did not. However, and most interestingly, we observed a significant interaction such that participants who received a pre-tutorial before their instructional video tutorial outperformed those who did not receive one; yet participants who were given a text tutorial scored higher on the post-quiz when they did not receive a pre-tutorial.
This surprising finding contradicted our third hypothesis, and perhaps the modality effect explains this significant interaction. Derived from cognitive load theory, the modality effect contends that students learn better when instructions are spoken in a narrated animation (e.g., video) instead of visual text alone (Haavisto et al., 2023; Moreno & Mayer, 1999). Although the null results from our first hypothesis are inconsistent with the modality effect, given the crossover interaction, we speculate that the modality of the pre-tutorial affected post-quiz performance. Specifically, we propose that the audiovisual modality of video pre-tutorials (e.g., visuals, sound, etc.) and their immersive qualities (e.g., pause, rewind, etc.; Rigby et al., 2016) more effectively compelled participants to follow and remember the evidence-driven tips presented in the video pre-tutorials than text pre-tutorials. Perhaps with text tutorials, participants felt that they already knew how to effectively learn from such materials and disregarded the helpful information contained in the pre-tutorial, resulting in lower quiz scores. Additional investigation is needed to further understand this surprising result.
As a part of this study, we also collected exploratory data on participants’ perceived competence in their academic advisor, participants’ confidence to accomplish similar tasks when assigned similar tutorials, and their level of engagement with advising materials. Of most interest was the finding that students, despite reporting that they preferred their learning materials to be in video format, actually reported being more engaged with the text tutorials than with the video tutorials. Additionally, participants reported that they were more engaged with the video materials when they received an instructional pre-tutorial compared to when they did not, yet there was no difference in engagement for text materials regardless of whether they were provided with a pre-tutorial. These findings run counter to research that suggests the multimedia nature of video tutorials would be more engaging than text, especially for Generation Z students who reportedly prefer multimedia (Mayer & Pilegard, 2014; Nicholas, 2020; Seemiller & Grace, 2016). Though these results seem counter to prior literature and our own participants’ learning preference results, they make sense given the time period of the study. Because this study was conducted during the COVID-19 pandemic, we speculate that these counterintuitive findings may be the result of a technological fatigue caused by prolonged remote learning. Additional work is needed to determine whether this engagement effect persists in a post-pandemic, more in-person world.
When reflecting on their confidence to complete other academic advising tasks if given similar advising materials, participants thought they would feel similarly confident when given text materials (regardless of the inclusion of a pre-tutorial), but that they would feel more confident with video tutorials if also given a pre-tutorial. This finding is in line with what we expected, given that students are likely not well-informed about best practices that support learning from videos. Thus, more tips should be given to help students engage effectively with video content.
Finally, we did not find that tutorial format or the inclusion of pre-tutorials had a significant impact on participants’ perception of their academic advisor’s competence. Such null findings may be consistent with past academic advising scholarship, which notes that students’ perception of their academic advisors’ competence is formed over time and through many different encounters rather than during a singular instructional tutorial (McClellan, 2014). Additional follow-up work could examine whether the relationship with one’s academic advisor changes because of chosen advising materials.
Advising Implications
Our results demonstrate that it is imperative to consider modality when designing materials for the asynchronous component of flipped advising. Perhaps surprisingly, it may not always be beneficial to provide students with video tutorials (despite their often-stated preferences for them). Indeed, our results suggest that to design the most effective text materials, keep the text short and efficient, with no extraneous information. With video materials, advisors should remind students to take advantage of the technology’s capabilities (i.e., pause, rewind, etc.) to get the most out of the learning material. Though these tips might seem unnecessary, it is important to view them from a student’s perspective—especially a student who has not been explicitly taught how to learn effectively. Such reminders in the form of a pre-tutorial may be a small but powerful way to improve student engagement with some advising materials.
Limitations & Future Directions
Our study does include some caveats. First, the creation and submission of a term plan is not a universal academic advising process, so results may vary depending on an institution’s academic advising process. For example, while some academic advising processes are covered quickly in tutorials, others may take much longer when written. The length of any tutorial (whether video or text) can impact the advisees’ reaction to the material. Additionally, some topics may require individualized instruction and guidance (e.g., selecting appropriate coursework for a particular student) and would not be suitable for conversion into text or video format. This caveat highlights the importance of the group space of the flipped advising framework (i.e., face-to-face meetings between advisee and advisor), which provides advisees opportunities to ask questions and clarify concepts. Should an instructional tutorial underperform, regardless of format, the advisee can always meet with their academic advisor.
The quality of instructional video tutorials, pre-tutorials, and text tutorials are all dependent on the available software. Trained professionals created our videos; as such, our results may be quality dependent. Future work should test this idea by including a condition where participants watch a video in lower quality or one made by an amateur to see if there is a decrease in post-quiz performance, in trust in their academic advisor, and/or in confidence to accomplish future tasks.
The remote nature of the study presented additional caveats, such as the fact that participants could not be observed. As such, it is unclear how participants interacted with the video, what devices they used, or if they followed instructions. However, as advisors within a flipped framework cannot observe students to ensure they are interacting appropriately with their assigned tutorial, the study is congruent with real-life conditions. Future studies should investigate whether tutorial formats and pre-tutorials impact remote students differently than their in-person counterparts.
Though task type, production quality, and the remote nature of the project were caveats, this study still serves as an exciting departure point for improving instructional tutorials (text or video) for flipped advising. Future studies might examine how pre-tutorials and instructional tutorials in general can be manipulated to positively impact the cognitive mechanisms that make them work.
Conclusion
This study investigated how to create academic advising materials by examining the effects of tutorial format and inclusion of pre-tutorials on learning. Surprisingly, participants who watched videos benefitted from having a pre-tutorial, but those who read text were hindered by the pre-tutorial. Therefore, pre-tutorials demand more investigation, especially in the context of academic advising models reliant on technology. Using technology to improve learning in the individual space of the flipped advising framework can create space for more meaningful conversations with advisees by addressing queries before they arrive to the advising session. In this sense, the importance of the academic advisor cannot be overstated. While pre-tutorials are an important tool that can enhance the academic advising experience, they cannot function as a replacement for the academic advisor.
Screenshot from Video Pre-Tutorial — Example of Narrative Action Matching Visual Action
Note. The illustration above shows an example of how the instructional video tutorial can pair visual action (i.e., the screen showing a large computer) with narrative action (i.e., a voice-over explaining the benefits of utilizing larger screens).
Flow Chart of Experimental Design
Note. The illustration above shows how participants moved through the experiment.
Final Scores on the Learning Quiz as a Function of Condition
Note. Quiz scores were out of 8 possible points. Error bars represent 1 standard error.
Exploratory Analyses on Student Experiences
Note. Student engagement with the tutorials (Panel a) and student confidence to complete other administrative tasks related to their major (Panel b) as a function of tutorial format and presence of a pre-tutorial. Error bars represent 1 standard error.
Contributor Notes
Dr. Jaron D. Paschke is a film, media, and communications studies scholar. He currently serves as both a Disability Resource Specialist at Riverside City College and as an Adjunct Faculty in Chaffey College’s Cinema Studies department. Jaron has taught first-year seminars at the University of California, Riverside (UCR), Visual Media Studies at the University of Redlands and American Film History at Chaffey College. In 2017, Jaron earned an M.A. in Film Studies from Chapman University’s Dodge College of Film and Media Arts. During this time, Jaron worked in story development for International Creative Management (ICM), former studio presidents and Academy-Award winners such as Robert Zemeckis at Universal Studios. Prior to this, Jaron earned a B.A. in Visual Media Studies at the University of Redlands.
Dr. Annie S. Ditta is an award-winning Associate Professor of Teaching in the Psychology Department and chair of the Academy of Distinguished Teaching at The University of California, Riverside. Prior to her appointment at UC Riverside, she received her Ph.D. from UC Santa Cruz. Her research marries her two areas of interest: creative thinking and the scholarship of teaching and learning. Her work specifically investigates the complex interplay between memory and creativity, with particular focus on how people’s ability to both remember and forget helps them learn to produce novel ideas, avoid becoming fixated on old or unhelpful ones, and transfer learned ideas to novel situations. The ultimate goals of her research are threefold: 1) to help students develop their critical and creative thinking skills, 2) to increase motivation to learn, and 3) to design better methods of instruction for large lecture courses at the university level.
