Measuring Outcomes in SoTL

Conceptualizing Outcomes in SoTL – From ‘what works?’ to ‘what’s going on?’. In this interview Dr. Gary Poole, Professor Emeritus, University of British Columbia, suggests understanding what is going on in teaching and learning is as valuable as understanding what works. SoTL: What works? Interview with Dr. Gary Poole.

Consider a focus on process rather than outcomes. Learning from Processes vs Outcomes: an Interview with Dr. Gary Poole.

Why integrate student voices in SoTL? Center for Engaged Learning Elon University. This video argues for the transactional nature of SoTL and the importance of integrating student voices in to SoTL. It highlights how involvement in SoTL for student researchers in terms of understanding their own learning and developing their research and career skills.

SoTL Inquiry Methods Center for Engaged Learning Elon University. This website gives an overview of Inquiry Methods for SoTL. It includes a video where SoTL scholars in Arts and Humanities discuss the inquiry methods they use in their research.

What data do you need? A decision tree for consideration of potential data sources for SoTL work. The Office of the Cross Chair in The Scholarship of Teaching and Learning – Illinois State University.

What type of evidence supports the purpose of your study? A decision tree on direct vs indirect evidence in SoTL work. The Office of the Cross Chair in The Scholarship of Teaching and Learning – Illinois State University.

Sources, Types and Analysis of Data in SoTL. – an overview of data collection methods. The Office of the Cross Chair in The Scholarship of Teaching and Learning – Illinois State University.

Study Design and Data Analysis in SoTL – an overview of SoTL study designs that includes brief descriptions of exemplar projects. The Office of the Cross Chair in The Scholarship of Teaching and Learning – Illinois State University.

Instruments that can be used to Measure Student Outcomes

Shortlidge, E., & Brownell, S. (2016). How to Assess Your CURE: A Practical Guide for Instructors of Course-Based Undergraduate Research Experiences. Journal of Microbiology & Biology Education, 17(3), 399–408. https://doi.org/10.1128/jmbe.v17i3.1103. Permalink - https://search.lib.jmu.edu/permalink/01JMU_INST/lvvpvt/cdi_doaj_primary_oai_doaj_org_article_0944508e11b44982a98bc64ae8db0803

Jhangiani, R.S., Troisi, J.D., Fleck, B., Legg, A.M.& Hussey, H.D. (Ed.s). (2015) A compendium of scales for use in the scholarship of teaching and learning. Society for the Teaching of Psychology. https://teachpsych.org/Resources/Documents/ebooks/compscalesstl.pdf

SoTL data sources from online Learning

How to use analytics in Canvas as a research measure?
- What is New Analytics?
- New Analytics in Canvas allow you to view a range of data about student participation, for example, average weekly online participation for individual students and for the whole class. Review the information here to see what you can do with New Analytics: https://community.canvaslms.com/t5/Canvas-Basics-Guide/What-is-New-Analytics/ta-p/73
- How to view New Analytics
  - Enabling New Analytics in Canvas at JMU.pdf
  - Make sure to enable New Analytics from the Beginning: “New Analytics report data begins on the date that New Analytics was enabled in a course. For example, if a course enabled New Analytics on October 1 and downloaded a report on October 15, the report would only include data between October 1-15.”
- How to view analytics for individual students: https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-view-analytics-for-an-individual-student-in-New/ta-p/801
- How to view and download reports in New Analytics https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-view-and-download-reports-in-New-Analytics/ta-p/409936
- How to compare the course average data table with an assignment, section, or student filter in New Analytics?https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-compare-the-course-average-data-table-with-an/ta-p/1189
- How do I view weekly online activity analytics in a data table in New Analytics? https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-view-weekly-online-activity-analytics-in-a-data-table/ta-p/595

What can ZOOM tell us about student attendance and participation?
- Attendance reports on ZOOM https://sites.udel.edu/ats/conferencing/zoom/attendance/
- Recording ZOOM chat https://support.zoom.us/hc/en-us/articles/115004792763-Saving-in-meeting-chat
- Polling in Zoom https://support.zoom.us/hc/en-us/articles/213756303-Polling-for-meetings

What can WebEx tell us about student attendance and participation?
- Attendance reports in WebEx - https://ask.lib.jmu.edu/techsupport/faq/319893
- Polling in WebEx - https://ask.lib.jmu.edu/techsupport/faq/318613
- Tracking Attentiveness in WebEx Events. It is possible to track attentiveness in real time in WebEx Events only. You cannot download Attentiveness data. https://help.webex.com/en-us/yj9y4z/Track-Participant-Attention-in-Cisco-Webex-Events