Assessment of Information Literacy Programs

From Immersion '06, July 31, 2006

Faculty Lead: Debra Gilchrist

For assessing a program, as opposed to student learning, Deb reframed the 5 questions from her “Assessment-as-Learning” sessions.

1. “What do you want the instruction program to be able to do? What does it do when it meets student needs and/or supports student learning?” (Outcome)
2. “What factors does the program demonstrate or exhibit if they are meeting the outcomes? What will be happening?" (Indicators)
3. “How will you gather your data or evidence?” (Assessment)
4. “How will you determine if you have achieved your outcomes?” (Criteria)
5. “How will the discussion or evaluation of the data occur? Who will be involved?” (Change)

It is important that the assessment leads into a cycle of continual improvement. Deb’s cycle of Outcomes/Indicators/Assessment/Criteria/Change reminded me of the Plan/Do/Check/Act cycle used in my organization. Deb recommends using ACRL’s “Characteristics of Programs of Information Literacy that Illustrate Best Practices” when establishing program outcomes.

Campus Cultures

From Immersion '06, July 30, 2006

Faculty Lead: Craig Gibson

A culture includes a combination of explicit knowledge (factual and recorded) and tacit knowledge (attitudes and beliefs). The “five core academic values of faculty” are 1) collegiality, 2) autonomy, 3) academic freedom, 4) specialized expertise, and 5) reason/scientific method.* They see librarians as “process, procedure oriented; rule-bound; service role rather than educational one; not seen as equals;” and “not aware of instruction that librarians already do.”

The campus culture also consists of a student culture, a library culture, and an administrative culture. In order to integrate information literacy into the curriculum, we need to “tap into all the cultures” and “bring them together.” Gibson quoted the “core changes strategies within the academy” from an article in The Journal of Higher Education (July/Aug. 2002) by Kezar and Eckel: “senior administrative support,” “collaborative leadership,” “robust design” (a collaborative and evolving blueprint of the future), “staff development,” and “visible actions” (actions that “show the culture is changing”). He emphasized robust design and visible actions.

* Barbara Walvoord. Academic Departments: How They Work, How They Change. ASHE-ERIC Higher Education Report, v. 27, no. 8. San Francisco: Josey Bass, 2000, pp.15-17.

So Whatsa Program?

At Immersion, July 30, 2006

Faculty Lead: Craig Gibson

Program Structure: From the Systems view, “structures consist of interrelated parts.” “Structures make sense, are visible to all the parties involved.” An IL Program structure is not linear, but a “spiral curriculum” (“Structures develop a progression of knowledge.”) An IL program structure consists of a horizontal scope (how many) and vertical scope (how long) that promotes the “Diffusion of Innovation.” Traditional Bibliographic Instruction helps the student finish the assignment but does not connect with the course. In an Information Literacy program, learning opportunities are embedded in the course. To get to an IL program, we can begin by leveraging the one-shot session. (See Clueless in Academe)

Leadership Orientations

At Immersion, July 30, 2006
Faculty Lead: Karen Williams

We began with a Leadership Orientations questionarre. Most in the room scored either as a Structural or Human Resources leader. According to Karen, these are the two most common frames in librarianship. A few scored as Symbolic, but only 2 scored as Political leaders. These four leadership frames: Structural, Human Resources, Symbolic, and Political come from Bolman and Deal's book that is used in the ACRL/Harvard Leadership Institute. Reframing is important because it broadens our perspective, helps manage complexity, and “provides a basis for effective and informed action.”

**It is crucial to create an Information Literacy Vision.**

There was then a discussion of what a leader is with views from several authors. Karen’s view of leaders:

1. Storytellers (Boyett & Boyett 1999)
2. “design learning processes” (Senge 1990)
3. “work with all in a community of leaders” (Barth 1992)
4. “take risks and are open to change” (Bennis 1997)
5. view mistakes and failure as learning opportunities (Bennis 1997).

Karen suggested we create leadership growth plans. She recommends Peter Senge’s The Fifth Discipline Fieldbook.

Our activity was an Opportunity Assessment: “Know your audience, focus on the change makers, and know their turf, use their issues.” It was also suggested that we draft short “sound bites” and a back-up plan.

Leadership Tools

From Senge’s Fifth Discipline Fieldbook (1994), the needed tools are: “personal mastery and self awareness, systems thinking, shared vision, mental models and team learning.”

There was a short discussion on change, including John Kotter’s (1996) process of creating change and human reaction to change. The session ended with a very brief overview of Force Field Analysis and the Ease/Impact Model.

Changing Paradigms

An Immersion event, July 29, 2006

Faculty Lead: Karen Williams

Bibliographic Instruction and Information Literacy are actually two different things, and most of what we do is still bibliographic instruction. According to Karen, the goal of bibliographic instruction is “to teach how to find information in the library” and the goal if information literacy is “to be an effective user of information in any format and place.” In an information literacy program model, information literacy is integrated into the curriculum. A librarian does not have to be present for information literacy instruction to take place. Changing from an “instruction paradigm” to a “learning paradigm,” as described by Robert B. Barr and John Tagg will allow information literacy instruction to be scaleable.

Assessment-as-Learning

During Immersion, Debra Gilchrist presented 3 workshops on assessment, 2 general sessions on assessment-as-learning and 1 on program assessment. My notes from the first 2 sessions are below. I will post my notes from the session on program assessment later.

Assessment as Learning Pt. 1 July 29, 2006
Faculty Lead: Debra Gilchrist
Deb’s 2 rules for assessment: 1) Work backwards from what you students to be able to do and 2) Assessment is about student learning.
Deb’s “5 questions for assessment design:

1. “What do you want students to be able to do?” (Outcome)
2. “What does the student need to know in order to do this well?” (Curriculum)
3. “What activity will facilitate learning?” (Pedagogy)
4. “How will the student demonstrate the learning?” (Assessment)
5. “How will I know the student has done this well?” (Criteria)

Deb’s outcome writing formula: Verb phrase + “in order to” + impact phrase
An example from Deb: "Develop topic relevant vocabulary in order to search databases with maximum flexibility and effectiveness."
Bloom’s Taxonomy is helpful in writing the verb phrase that describes what you want the student to be able to do. The impact phrase explains why you want the student to be able to do it.

Assessment-as-Learning Pt. 2 July 30, 2006
Faculty Lead: Debra Gilchrist
This session began with a brief review of the outcome formula. In the formula, the verb phrase is the skill and the “in order to” statement is the application or impact. Deb believes outcomes are fully measurable. Recall from the “good learning outcomes” slide that they are measurable/“judgeable.” Measurable outcomes are quantitative and judgment outcomes are qualitative. A judgeable outcome is where the instructor can see that the student has improved (I am thinking this means behavior wise). Assessment test like SAILS and ETS can provide a benchmark. The best assessments include a critical thinking element. Debra than shared some documents demonstrating IL assignments and outcomes that librarians have helped faculty develop and self-assessment students utilize in library workshops.

At Pierce College, the library provided a faculty workshop where they analyzed assignments with faculty. The faculty submitted assignments in advance and the librarians identified what IL concepts students needed to know in order to complete the assignment. They then worked with faculty to revise outcomes.

At Pierce College they assess only one out of five outcomes each year so that each out come will be assessed at least once over an eight year period. More important outcomes may be assessed more often than less important ones. At first they only assessed 1 in 10 sessions (10%), now they assess 1 in 4 (25%).

PS. Be sure to check out Pierce College's Library Instruction Program. Notice that Information Competency is on of the college's 5 core abilities. Pretty cool, huh?

Immersion Opening Plenary

Following is a summary of the first session at Immersion '06. My notes from other Immersion events will be following soon.

Opening Plenary June 29, 2006

Faculty Lead: Craig Gibson
After all the faculty introduced themselves, the program begin with discussion on the definition of information literacy that included the “7 Ages of Instruction,” a jig-saw reading activity on 5 articles that defined information literacy and Christine Bruce’s “Seven Faces of Information Literacy in Higher Education.”
For the next part of this session, we watched “The Deep Dive,” a fun Nightline clip about an innovative design company called IDEO. IDEO immerses itself into a design problem and uses play to foster innovation. Their five brainstorming rules are:

• “Defer judgment”
• “Encourage wild ideas”
• “Build on the ideas of others”
• “Stay focused on topic”
• “One conversation at a time”
• “Be visual”
• “Go for quantity”

Immersion '06

Simmons Conference Center
Originally uploaded by AmyGF.

Well, I got home from ACRL’s Institute for Information Literacy Immersion ‘06 in Boston a few days ago and I am dripping wet after being thoroughly immersed in Information Literacy for nearly a full week. It was an intense, but great experience. Not only did I learn a lot, but my perception of Information Literacy has changed and I’ve gotten so many ideas I want to try out.

I participated in the Program Track and our final product was an Action Plan to bring home. While my Action Plan still needs some work, it lays out some goals that I believe we can achieve within the next year.

Some pics from Immersion are available at www.flickr.com/groups/acrlimmersion2006/. Most of these pics are from the Teacher Track, but it still gives you good idea of what the whole program is like.

I’m still working on my summaries of events, but those will likely appear shortly.