Assessing Listening to the SONG of Life
7.2 Method
L-SONG Instrument
Likert-type statements were generated from sources used in teaching the course to create an empirical measure of student learning in the Listening to the SONG of Life course.[1] Fifteen items representing the four contexts in the SONG of Life are represented in the L-SONG instrument. Three to four items for each context are included for assessing internal reliability. The same six-point Likert-type scale[2] is used for all fifteen items. Items are grouped [3] by the four SONG contexts of self (see items 1 through 4 in Appendix B), others (5 through 7), nature (8 through 11), and Goddess-God-the Divine (12 through 15).[4]
Expert Validity of L-SONG
Expertise in an academic field is partly determined by credibility. McCroskey and Young’s three-decade summary of research on credibility finds competence (education and experience) the most important credibility factor. Accordingly, a credible expert evaluator for this study must have earned their PhD in Communication (educational competence) and published at least one academic journal article or book on a listening topic (experiential competence). Four experts in the field of listening were contacted through e-mail and asked to rate the appropriateness of the fifteen L-SONG items for inclusion in each of the SONG contexts.[5]
All experts agreed to participate. Experts used a six-point Likert scale[6] to rate each item on “appropriateness for inclusion” in a given listening context. Average mean scores for each SONG context are 5.0 for the listening to self context, 4.2 for other, 5.5 for nature, and 4.9 for the listening to Goddess-God-the Divine context. An average mean score of “4” or greater indicates experts agree that the items are appropriate for inclusion in a given listening context. All four contexts show average mean scores greater than “4,” indicating that experts agree that the items are appropriate for inclusion. In sum, the four SONG contexts demonstrate a degree of expert validity.
Design and Procedures
Using a pre-test and post-test design, the L-SONG instrument was made available to students in the Author’s Listening to the SONG of Life courses. Students were informed that the L-SONG instrument was not part of their academic grade and that the data would be used for student feedback and research. There was an option for students to opt out of including their data for research purposes and still receive feedback. The College of Arts and Letters Human Subject’s Committee at the Author’s home institution approved the research.
Students completed L-SONG on the first day of class before any instruction as a pre-test, and then a second time on the last day of regular class as a post-test. In principle, potential student learning would be indicated by positive gain scores, measured from pre- to post-test. The instructor left the room while students completed the L-SONG instrument, and a graduate student collected the surveys