What does information literacy look like in first-year college work? It can be highly disconcerting, especially for the faculty member confronted with students who discover and use online materials that are inappropriate for their projects. Angie was enrolled in my section of English 111, Composition, Research, and Rhetoric, a first-year composition course. For a research paper on poverty among college students, Angie relied heavily on an article she found online but which was first published in 1966 at the University of Strasbourg. The article, “On the Poverty of Student Life,” addresses the situation of European students generally, and I seriously doubt that Angie knew the context of student unrest overseas in the 1960’s. I almost talked her out of using it, but she insisted that it was a worthwhile source. Her annotations and marginal comments on the article demonstrate that she used the article to think deeply about her topic (see figure 1). She wrote comments on statements about student poverty, such as the assertion that “looked at economically, student life is a hard one.” As with this quote, her underlining shows that she focused on the timeless situation of students. Angie’s favorite statements include two on a student’s lack of awareness: “The student is blind to the obvious--that even his closed world is changing,” and “the student’s old-fashioned poverty, however, does put him at a potential advantage--if only he could see it.”¹ These quotes and her handwritten comments show that her personal reactions to the topic are at the forefront, as when she writes that “your life can become out of control and for all you know, you may feel as if you are on another planet.  It is all about economic survival.” Much of the page shows her double underlining in blue ink of words like “priorities” and “frustrations” indicating the emotional content of the topic for her and her peers. This article did, indeed, become a worthwhile source for her and shows her close engagement with her topic.

Figure 1

My goal for this course was to increase effective instruction on Internet research and improve the quality of Web-based sources used in student papers. I focused on developing a series of lessons in research methods, analytical reading, and critical thinking, all in the context of Internet research. Moreover, I wanted students to develop a better sense of the “art” of research, of finding good sources by sheer work, good judgment, and a bit of serendipitous luck. My composition course is offered at a small, rural, liberal arts college with a substantial enrollment of first-generation college students. In order to assess the results of course design, I used pre-and post-surveys to examine student attitudes and level of knowledge. At the end, I asked for student input about the sequence of assignments on Internet research and their perceptions of their own learning. Finally, I evaluated all drafts, research notes and textual annotations on their research, and final copies of their papers. Through these sources, students showed cognitive development in thinking about the processes of research, particularly in the area of several related cognitive functions: gathering, discovery, connectivity, and evaluation.  In summary:

Information Gathering: students appreciated a wide range of options and felt that the Internet allowed them more choices. They also expressed awareness of the pitfalls of searching the Web and cautioned against gathering too much information without focusing their topic or evaluating their results as they progressed in their research.

Discovery: students appreciated the exposure to new perspectives and areas of research--essentially the increased opportunity for serendipitous events.  

Connectivity: students liked the brainstorming aspect of Web searches and how they would return to their searching strategy as new information appeared. Students also made connections in terms of sequencing tasks and developed a sense of the connections emerging between their sources.

Evaluation: students want to read a lot of articles to find the “best ones.” They expressed awareness of practicing more caution in their choices and understood the need to look at sites for the publisher’s bias.²

Overall, student responses recognized the complexity of research, particularly Web research, and noted that it is time-consuming, unpredictable, but also worthwhile. Their personal motivations, particularly evident in their pre-writing work, were more obvious as well, and they demonstrated more thoughtful use of sources. 

In order to facilitate learning, students were required to annotate many of their research texts. Students were instructed to use a variety of markings and notations--from underlining to symbols in the margins to written notes. In class, we reviewed analytical reading and critical thinking terminology so that students looked for main ideas and supporting points. On their research, for example, they had to mark “important comparison” in the margin, write down their thought, or to write out a question. The students’ practice of annotating research texts reveals how their thought processes start with personal engagement with textual meanings. As I analyzed student comments on their sources and comments on their draft work, I focused on their reflective judgment, their ability to conduct higher order thinking. Although we do not want students to work without any sense of direction from us, if we limit their choices then we limit opportunities for creative thought. We should keep in mind that one popular approach, providing a pre-approved short list of Web sites for students, does not teach students how to conduct searches or make choices.³ However, without the scaffold of instruction about internet research, students make errors--particularly errors of omission. Should we be more concerned as teachers about correctness of sources or about successful inventiveness? This is an area of anxiety and disagreement. Yet consider Umberto Eco, who writes, “false beliefs and discoveries totally without credibility could then lead to the discovery of something true. . . . In the field of sciences, this mechanism is known as serendipity".⁴ This process is an important part of a student’s move towards becoming a creative, critical thinker. Angie’s article may not have been part of her paper except for the rather unpredictable results that we all get from Internet research. The concept of “serendipity” in a sense emerges from the activity of all four cognitive functions listed above. Serendipity can be useful in thinking about teaching first-year students who most likely have not been allowed much freedom in their academic lives.   

Gathering Information
By the end of class, my students’ papers showed a remarkable variety in the use of primary and secondary sources from Web sites sponsored by government agencies, the United Nations and various international agencies, and materials from university sites. Jack wanted to write about poverty in his home country, Canada, and relied on Web sites for information, in addition to articles in the library’s databases. His research took him to "The Status of Women in Canada” to read about policies for reducing women’s poverty; “Welfare Watch” to read about changes in Canadian welfare and a five-year report on the affects of cuts; the article “Canadian Women Among Poorest of the Poor;” and finally to a fact sheet from the Canadian Research Institute for the Advancement of Women. Like Angie, personal interests motivated Jack whose mother struggled in poverty. He writes in his research proposal:

There are many women who are on the borderline of poverty. These women try their best to escape lower class status and one option is by taking on two occupations. . . . A perfect example of this would be my mother [who] was facing extreme poverty. She managed to score a job as a waitress . . . but found the pay was lousy and needed another job to survive . . . For the last twenty years she has been working both jobs in order to support my family [and] now successfully provides for our entire family.

In this passage, Jack tries to put his mother’s struggles and successes into a larger context. He sees her not just as poor but places her on a poverty continuum. His pride in his mother is conceptualized, and he notes that this one “option” of working two jobs has serious drawbacks. As I found with other students, their acquisition of research skills on the computer coincided with development of personal approaches and evidence of high motivation and creative, critical thinking. 

Discovery
When students locate information that leads them to new ideas, the element of surprise in the process of making a discovery is part of creative thinking, which researcher Mathew Lipman argues works together with critical thinking as a student moves toward making sound and effective evaluations. He explains that teachers should:

preserve the element of surprise. The surprising fact opens the door to new discoveries and still further surprises. If critical thinking tends to eliminate surprise by seeing the perplexing event as something that happens quite as a matter of course, creative thinking tends to escalate surprise by seeing the surprising event as merely the first in a rapidly expanding series of surprises. For example, the Matruschka doll when opened by a child contains within itself a smaller doll that turns out also to be a Matruschka doll and so on.⁵

The personal facet of the student’s intellectual discovery may be hidden from the teacher. It was evident, however, in students’ notes. Allison worked on child labor issues, using multiple forms of research materials such as online photography collections and commentary, and she used Web sites on the history of child labor. In her work, she discovered a new term, “generational” (in reference to generational labor to repay debt). She carefully made a note to herself to remember it. A link to contemporary forms of slavery in Pakistan prompted her to write “check on this!” 

As Allison encountered new knowledge, she enjoyed exploring these new avenues. According to Perry, this inclination to learn places her in a middle phase of development, multiplicity. A student in the first phase, dualism, might look at new knowledge as inconvenient, since a dualist searches for the ‘right’ answers to a task. Multiplicity encompasses discovery, since students are open to learning new ways of thinking and are willing to consider diverse information and views. The final stage is one that we hope our students will reach; students comfortable with relativism and having a commitment to relativism have grasped that they must make judgments about evidence in terms of context and in a way that integrates objectivity and empathy.⁶ As students move through these phases, they must move out of established patterns and passive habits of learning and into discovery, with the ultimate goal of integrating valuable discoveries into their lives. Teachers can encourage students through methods that give time and support to student exploration. Students become “cognitive apprentices” in that they follow the teacher’s directions, and then strike out on their own. As Sue Berryman summarizes, “The Collins, Brown, and Newman model [of cognitive apprenticeship] includes a variety of methods that systematically encourage student exploration and independence. Teachers coach--offering hints, feedbacks, and reminders; provide "scaffolding"--support for students as they learn to carry out tasks; and "fade"--gradually handing over control of the learning process to the student.”⁷

Connections
Encouraging students to make connections is crucial to helping them deal with the wealth of materials that they discover. Students no longer rely solely, as in the past, on books that help the researcher sort and weigh evidence. Much of their online information is unsorted, particularly when they encounter primary data. Therefore, they need to be taught methods of bringing together disparate sources. Lipman defines connectivity as “relationship” and this term captures some of the personal implications that making connections has for students. He writes:

The world is made up of complexes (evidently there are no simples) such as molecules and chairs and people and ideas, and these complexes have connections with some things and not with others. The generic term for connections and disjunctions is relationships. Since the meaning of a complex lies in the relationships it has with other complexes, each relationship, when discovered or invented, is a meaning, and great orders or systems of relationships constitute great bodies of meaning.⁸

In a sense, these complexes are similar to the “nodes” of online searches. Everything is in relation to everything else through search mechanisms. We need students to learn to develop, or ‘invent’ connections, to use Lipman’s term, between various ideas and information without losing themselves in the search. We can encourage students to see themselves as independent, creative researchers.

Dave invented a connection when he used information from an Australian government Web site in a paper on Appalachia. As professional scholars, we would hesitate to use information from Australia to discuss Appalachia. Dave’s topic was health and welfare in Appalachia because he was from an Appalachian working-class community whose members suffered from many health issues. Not a strong writer, he was motivated to learn more about his topic because of his personal investment. This sense of interest and commitment lead him to creatively use this health Web site from Australia. On one page printed from the site (on health care for those over fifty), Dave made several notations about connections to his own topic. On another page where the author related family earnings to age “and other factors,” Dave wrote that one of these other factors was to “graduate from school” which is a serious issue in Appalachia. When the author of another “white paper” on this Web site noted that families with two working parents have higher incomes, Dave responded in the margin that in lower income families “the wife takes care of the kids,” a common practice in family-oriented mountain communities. Finally, Dave added a “star” and his marginal note “important” next to a section headed “Sources of income other than wages and salaries;" many students like Dave participate in farming (such as a plot of tobacco) as the family’s other source of income--which he no doubt was considering. In all these comments, Dave showed how he was thinking about his research--how the experiences of this collection of Australian professionals was helping him to think about his own community.

In pre-writing--annotations, draft work, and paper proposals--we see how students like Dave react to new information by connecting texts with their own lives. Their motivation comes from the initial, personal connection and sustains them through the boring work of a composition class. Online research draws their attention in part because they are exposed to a variety of ideas, perspectives, and styles. The Web provides a mechanism to invent connections that the more linearly-organized databases of professional journals just do not provide. Their research notes provide the teacher with the means to discover otherwise hidden student processes.

Evaluation
The complex task of evaluation involves applying reasoned criteria developed for the situation, but personal values cannot be ignored since students always start from their own experiences. Students must reach a stage of engagement in order to begin to fully evaluate information and arguments; otherwise, they will apply criteria indiscriminately. As Lipman argues, judgments should involve more than the application of logic; they are often the result of inquiry and “as for good judgments, we often attempt to explain these as a happy admixture of critical and creative judgings.”⁹ Students ideally negotiate personal criteria with new information in logical ways and in doing so they often face potentially emotional information.

Wanda’s study of wage gaps between men and women and minorities caught her personal interest as an African American undergraduate, and her research led her to several Web sites, including the Black America Web. Her annotative marks consist mainly of highlighting main points and bracketing important points, but these follow a clear pattern indicative of her thought process. She hones in on news-like phrasing; on an article printed from blackamericaweb.com news, she highlights a statement about pay disparities still affecting black women the most.  Even when she doesn’t seem to be reading an entire article, she shows that she is (after several blank pages) by marking a key phrase. On an analysis of a 1995 study on minorities in the workplace, she brackets a statement in the middle of the article about high levels of workplace discrimination towards minorities. Her interests, as indicated by research notations, led directly to her research proposal:

My research paper will show the degree of losses suffered by black women depending on how much education they have and the area they choose to live. I will show how this problem will affect young black women in the future who have not yet experienced the working world. I will also explain how this problem affects me as an individual and as a young black woman. I am committed to further evaluating this issue and coming up with a solution to fix it.

Her clear sense of purpose causes her to highlight, bracket and underline author Laura Dresser’s statement that the wages of college-educated women fell between 1973 and 1991. She also marks Dresser’s following question, “why did the wages of young black college graduate fall severely?” Wanda writes in the margin, “that’s a question that needs to be asked.” Her sense of purpose also compels her to find and read Title 29, “Guidelines on Discrimination Because Of Sex” which she finds through the U.S. Department of Labor Web site. She works her way through the legal language by labeling the article using critical thinking terms. She notes the “assumptions” of the commission and points out several instances of “cause and effect” in the section governing unlawful wage practices, such as avoiding paying women for overtime.

In her final paper, Wanda emphasizes that her research provides her with important information. “In summarizing all of my research and investigation to the reason why there is a wage gap among black and white women, I find very useful information, which could better help me today and in the future. The same information that I provided to my readers, I hope will also do the same.” She concludes, however, with practical advice encouraging political acumen in her readers: “The solution could be very simply, go to school and get an education. Then, choose a career or job that best fits the educational background that one has invested. As for discrimination, it is still part of society no matter how it is looked at.”

Sequencing Assignments
The integration of traditional and online research skills is essential to teaching a successful research course. In order to achieve results in student cognition, in knowledge base, and in motivation, course design should focus on this integration, with careful planning of lessons and assignments to build student skills and comprehension. My project demonstrates that courses are most successful when following these guidelines:

1. Lessons and assignments are sequenced, in a scaffolding fashion, so that each assignment builds on the last. A scaffold provides support for the student and is structured so that one level builds upon the other. For instance, instead of a single introduction to searching online, the introduction to search engines can be a starting point for progressively more complex assignments building on previous knowledge. Lessons are delivered at regularly scheduled intervals throughout the course, so that learning is evenly distributed.
2. Lessons balance the transfer of information about technology and research skills.
3. Critical thinking concepts, which provide an essential framework for research evaluation, are incorporated into class and practiced regularly through such means as textual annotation, concept mapping, and research logs.
4. Reading skills are adapted to the online environment; students learn to “read” search results, database information, and Web sites. Reading Web sites in particular requires that students read vertically, horizontally, and through multiple layers and that they recognize the key words that most Web sites use to guide the reader (such as “about us” which often leads to publication information). One student comment was typical about the course: “I learned how to read a site and how to read the hits.”

Requiring students to do intentional reading (textual notation and marginal comments) of each source also makes students aware of what they are collecting. A research log can help students maintain searches over several days and can also be used to log several types of searches. Instruction and practice in reading complex Web sites gives them some sense of how the two reading experiences differ. 

Research challenges students conceptually. Consider the complexity of the metaphorical language we use about areas of research: the “search” and the metasearch (a truer search?) sound mystical, a kind of archetypal quest (especially when combined with geographical terms like “mapping” and “logging”); the term keyword has a metaphysical promise; other terms invoke complex spatial images such as Web, network, and links; and finally “operators” (as in Boolean operators) echo with the lingering implication of agency. Given this profusion of rich terminology and complex skill building, students need to be grounded in regular, persistent sequencing of lessons and activities that encourage meta-cognition. My student Angie struggled with the research process, but the best aspect of her work was how she grappled with overarching ideas. She was thrilled to find graphs on the rates of college graduation by race and ethnicity and the rate of poverty by education published on a University of Maryland Web site. Her assertion of their importance in her final paper is quite direct: “Toward the end of this paper, you will find recent graphs showing racial differences of poverty and poverty based on levels of education. It is amazing how much of a difference a certain lifestyle or culture within a race can change the financial stability of ones’ [sic] own life setting.” As we teach research skills, fostering that sense of amazement at discovery develops students as thoughtful researchers capable of handling the serendipitous moment online.

Notes
Theresa Swann, Director of Instructional Technology at Tusculum College, was an integral part of this research project.  She provided training and classroom support, research support and continuous feedback on the project.
About VKP: In all, more than seventy faculty from twenty-two institutions participated in the Visible Knowledge Project over five years. Participating campuses included five research universities (Vanderbilt University, the University of Alabama, Georgetown University, the University of Southern California, Washington State University, and the Massachusetts Institute of Technology), four comprehensive public universities (Pennsylvania’s Millersville University, California State University (CSU)--Monterey Bay, CSU Sacramento, Ohio’s Youngstown State University, and participants from several four-year colleges in the City University of New York system, including City College, Lehman, and Baruch), and three community colleges (two from CUNY--Borough of Manhattan Community College and LaGuardia Community College, and California’s Cerritos College). In addition to campus-based teams, a number of independent scholars participated from a half dozen other institutions, such as Arizona State and Lehigh University.  The project began in June 2000 and concluded in October 2005.  We engaged in several methods for online collaboration to supplement our annual institutes, including an adaptation of the digital poster-tool created by Knowledge Media Lab (Carnegie Foundation), asynchronous discussion, and web-conferencing.  The VKP galleries and archives (http://crossroads.georgetown.edu/vkp/ ) provide a wealth of background information, including lists of participants, regular newsletters, and reports and essays by participants, as well as a number of related resources and meta-analyses. For this article, the author gratefully acknowledges the students whose work is cited here. All students whose work is included have granted the author permission to use the material.
1. U.N.E.F. Strasbourg, “On the Poverty of Student Life: Considered in Its Economic, Political,
Psychological, Sexual, and Particularly Intellectual Aspects, and a Modest Proposal for Its Remedy,” Nothingness.org, (first published by University of Strasbourg students as a pamphlet, 1966), http://library.nothingness.org/articles/SI/en/display/4 [return to text]
2. In a colleague’s more traditional section of the course, one without significant computer instruction, students responded to the survey in much less complex terms.  They reported that the Internet was either “easy,” “quick” and “reliable” or they found it to be “misleading” and time-consuming.  They answered either “yes” or “no” when asked if going online was helpful. [return to text]
3. David Ward and Sarah Reisinger, "Designing Internet Research Assignments: Building a Framework for Instructor Collaboration," Information Research 6, no. 1 (Oct. 2000) http://InformationR.net/ir/6-1/paper86.html. Ward and Reisinger identify three levels of research assignments: non-directive research, limited research, and the third level, a balanced approach.  In non-directive research, students are told specifically what source to use. [return to text]
4. Umberto Eco, Serendipities, trans. William Weaver (New York: Harvest, 1999), viii. [return to text]
5. Matthew Lipman, Thinking in Education, 2nd ed. (New York: Cambridge University Press, 2003), 275. [return to text]
6. See William G. Perry’s Forms of Ethical and Intellectual Development in the College Years: A Scheme, (Jossey-Bass, 1998). [return to text]
7. Sue Berryman, “Designing Effective Learning Environments: Cognitive Apprenticeship Models,” Institute on Education and The Economy 1 (Sept. 1991): 1-5, http://www.tc.edu/iee/BRIEFS/Brief01.htm [return to text]
8. Lipman, 23. [return to text]
9. Lipman, 141. [return to text]

Attachment	Size
Olsen_Engaging_Students.pdf	1.38 MB