Tuesday, November 07, 2006

SIG meeting 10/25/06

ARC Reference & Information Literary Special Interest Group Meeting
October 25, 2006 Assumption College - Emmanuel d'Alzon Library

The focus of this meeting about assessment was a presentation by Assumption librarians Carole Myles and Callie Curran Morrell on “Library Instruction & Program Assessment.” Assumption College reference staff began a process this past summer to try to standardize instruction and explore possibilities for assessment (measuring student learning) and evaluation (determining the effectiveness of teaching).

In terms of assessment, their presentation showed the progress they have made to date in using a pre-test/post-test system, which they have begun to analyze. Another strategy they use in advanced classes is to ask students two questions: Tell us something you learned; Is there something you still find confusing? Responses indicate to librarians things they want to emphasize in their sessions, and areas where students are not receiving what instruction is intended to convey. Qualitative analysis of these responses may be attempted with ATLAS software. Carole and Callie said they also hope to begin a collaboration with a faculty member that will allow them to review student bibliographies.

For evaluation, the librarians hope to measure efficiency and effectiveness through a combination of data collected on instruction sessions, through surveying students, by soliciting feedback from faculty, and using self-reflection.

The SIG attendees suggested ways to refine the pre-test/post-test questions as well as considerations about sample size and methods of obtaining feedback from students and faculty members. Other information literacy assessment tools including ICT and Project SAILS were mentioned. Librarians in this group of ARC libraries were not using any of these, but many had done or are about to do LIBQUAL surveys. WPI has an assessment cycle for faculty to evaluate student e-projects (which can be viewed online) including the reference lists. Several librarians are doing literature reviews on bibliography analysis—maybe we should assemble this list of citations to share it.



Christine Drew (WPI) opened the discussion of compliance with ARC governance structures. Following a brief discussion she volunteered to continue to act as she has been to organize and inform the members of the meetings. The next one is planned for spring break in March. Dale LaBonte (QCC) agreed to act as secretary for the SIG. Sending a representative to the ARC meetings was not settled.

Members felt that it would be beneficial to use funds leftover from WACL for to hire training consultants Decker Associates in Needham to deliver a session based on our needs as teaching librarians. Christine suggested doing so in conjunction with the CMRLS calendar of workshops so that as many librarians as are interested could attend.

Christine noted that the NERCOMP Annual Conference to be held in the DCU center on March 19-21, 2007 has a track for librarians. ARC librarians may want to take advantage of the Worcester location. NERCOMP—Northeast Regional Computing Program—is an affiliate of Educause.

While we were assembling, we got to watch and comment on two YouTube videos. The first was "Ray of Light", produced by the St Joseph County Public Library; we also saw a promo for Refworks produced by Arizona State University. At Assumption they have these running as students are arriving for library instruction session. Later in the session, Christine showed us some challenge exercises she offers as transition or sponge activities for students who finish an in-class project early, while they are waiting for other students to finish.

Next meeting is planned for WPI on Wednesday March 7, 2007 at 2:30.

Attendees:
Alice Baron (Anna Maria), Barbara Merolli (Anna Maria), Callie Curran Morrell (Assumption), Carole Myles (Assumption), Christine Drew (WPI), Dale LaBonte (QCC), Elizabeth Maisey (Assumption), Larry Spongberg (Assumption), Laura Robinson Hanlan (WPI), Mary Brunelle (Assumption), Rachel Shea (Clark)

Citation analysis bibliography

After discussing citation analysis as one possible tool for assessing student learning (at the meeting on 10/25), I wanted to post this bibliography of studies that have incorporated it as a methodology in their research. Please post any more that you may know of as comments.
-Callie Curran Morrell, Assumption College


Bibliography of Citation Analysis Research

Ackerson, L.G., and V.E. Young. “Evaluating the impact of library instruction methods on the quality of student research.” Research Strategies 12 (1994), p. 132-144.

Davis, Philip M. “The Effect of the web on undergraduate citation behavior: A 2000 Update.” College & Research Libraries (January 2002), p. 53-60.

Dykeman, Amy and Barbara King. “Term paper analysis: A proposal for evaluating bibliographic instruction.” Research Strategies 1 (Winter 1983) p. 18.

Gratch, B. “Toward a methodology for evaluation research paper bibliographies.” Research Strategies 3 (Fall 1985), p. 171-177.

Hovde, Karen. “Check the citation: library instruction and student paper bibliographies.” Research Strategies 17 (2000), p. 3-9.

King, D.N., and J.C. Ory. “Effects of library instruction on student research: a case study.” College and Research Libraries 42 (Jan. 1981). p.31-41.

Kohl, D.F. and L.A. Wilson. Effectiveness of course-integrated bibliographic instruction in improving coursework. RQ 27.2 (Winter 1986) p. 206-211.

Malone, D., and C. Videon. “Assessing undergraduate use of electronic resources: A quantitative analysis of works cited.” Research Strategies 15 (1997), p. 152-158.

Mohler, Beth. “Citation Analysis as an assessment tool.” Science and Technology Libraries 25.4 (2005), p. 57-64.

Robinson, Andrew M., and Karen Schlegl. “Student bibliographies improve when professors provide enforceable guidelines for citations.” portal: Libraries and the Academy 4.2 (2004), p. 275-290.

Ursin, Lara, Elizabeth Blakesley Lindsay, and Corey M. Johnson. “Assessing library instruction in the freshman seminar: a citation analysis study.” Reference Services Review 32.3 (2004) p. 284-292.

Young, Virginia E., and Linda G. Ackerson. “Evaluation of Student Research Paper Bibliographies: Refining Evaluation Criteria.” Research Strategies 13.2 (1995) p.80-93.