[back to Part I]

Pedagogies of Adaptive Expertise
Conceptual finding: New media environments can make visible the intermediate thinking processes intrinsic to the development of expert-like abilities and dispositions in novice learners, and nurture abilities associated with adaptive expertise that allow practitioners (and learners) to make flexible use of knowledge in self-regulated ways. New media environments allow students to recognize layers of learning that lead to greater reflection and control over learning processes. New media pedagogies engage novice learners in foundational aspects of expert thinking, difficult to address in other ways. Students’ intellectual development in relationship to expert thinking traditionally has been invisible to higher education.

From the beginning of VKP, we were influenced by the burgeoning research on expertise. The 1999 publication of How People Learn (Bransford, et al.) made the previous thirty years of research on expert thinking available to a wider group of educators. That volume laid out basic differences between expert and novice thinking.⁸ We were especially interested in its broadened definition of the kinds of knowledge experts possess: formal and procedural knowledge (traditionally emphasized in college), as well as informal, experiential, and self-regulating knowledge. The salient characteristics of experts, who are “comfortable working at the edge of their competency” and who are particularly good at “progressive problem-solving” that allows them to move forward with uncertainty (Bereiter and Scardamalia),  struck us as particularly relevant, given our focus on new media pedagogies. We also came to see the importance of newer work on what is commonly referred to as “adaptive expertise,” traits linked to innovation and resilience in complex problem-solving.⁹

By focusing on expertise, we were not suggesting that all students should (or could) be turned into “experts.” What mattered to us were questions about how to contextualize the introduction of novice learners to structures of authentic learning in disciplines. What kind of values about learning and thinking should inform our designs? What was it about the affordances of new media environments that both enabled and compelled us to reevaluate the ways we cultivated “judgment in uncertainty?”¹⁰

Our findings about expertise and new media pedagogies disrupt in at least two ways the developmental assumptions that typically inform curricular and assessment practices: 1. by suggesting the need to emphasize earlier, for novice learning, certain elements of developmental expert thinking often thought to require years of accumulated learning (for example, putting students in positions to make judgments about primary source research or  multimedia authoring even if they lack the knowledge to “correctly” represent all aspects of a subject;) and 2. by prompting us to rethink assumptions about how we assess student progress along this developmental journey, compelling us to find better ways of offering students evaluative judgments on their representations of these intermediate processes.

Design findings
VKP case studies pointed to several important design considerations for teachers trying to realize the potential of visible intermediate processes on the development of adaptive expertise. In general, faculty must make room for uncertainty, openness to multiple paths, reflection, and productive iteration. Additionally, faculty who design for adaptive expertise in new media environments found that they needed to create new ways to capture the artifacts of student learning that reflect expert processes. These are inherently distinct from traditional summative assessments. Five design findings are elaborated below:

➢    Designing for adaptive expertise means recognizing what is “necessarily difficult” about a field.
When David Jaffee (City College of New York) says that he wants his students in his U.S. history class to “think visually,” he addresses a difficulty confronting the discipline. He designs student experiences with primary materials in digital archives with this goal in mind:

When a student searches through a greater variety of materials, pulls apparently unconnected texts into a relationship, and constructs a plausible story from those student-located and selected models, she models the practice of an ‘expert’ or professional historian who enters an archive with a series of questions and perhaps a tentative hypothesis in search of evidence. It was such moments that slowly revealed my own far greater interest in having the students ‘to think visually as historians,’ indeed to ‘do history,’ rather than merely add images to the potpourri of sources included in the course mix. (Jaffee)

Throughout the classroom inquiry projects, faculty went beyond disciplinary knowledge and habits of mind, wrestling with ways the “necessarily difficult” dimensions of a field could reshape their pedagogical designs. ¹¹

The focus on engaging difficulty emerged across the whole range of VKP classrooms. In teaching a basic skills reading class to EFL students, Sharona Levy (Borough of Manhattan Community College) developed a teaching strategy designed to make levels of difficulty visible to her students, including those who struggled to read complex texts. Making use of the “comment” feature in Word as a type of “think aloud” strategy, Levy asks students to mark the troublesome places in a historical text (such as a letter from George Washington to the Continental Congress) as falling in one of three categories of “difficulty:” words they don’t know; terms that seem confusing as used; and concepts that seem enigmatic or particularly complex. By engaging the text this way, her students--who would typically construe difficulty as their own undifferentiated failure--were able to separate out moments of difficulty that were merely a matter of vocabulary, for example, from those that would pose interpretive difficulty for any reader (Levy).¹² Instead of experiencing a sweeping sense of failure in their reading, Levy’s students were able to begin working with texts as more expert readers: compartmentalizing issues of decoding from more complex tasks of interpretation.

As teachers, we so often value dimensions of students learning that we do not actively design to produce. Many teachers value their students engaging what is most difficult about their fields; the challenge we saw emerging in VKP was how to intentionally pursue this goal: how to design for difficulty. Faculty engaged in classroom inquiry often discover this gap between what they value and what they explicitly include in assignments. Looking closely at student learning processes helped faculty reorient their design priorities to emphasize intermediate activities addressing the difficult dimensions of disciplinary thinking.

➢    Designing for adaptive expertise requires allotting time for intellectual play and uncertainty.
We found that designing for difficulty meant providing substantial time for exploration, experimentation with ideas, and uncertainty. In fact, a key outcome of linking difficulty with a robust definition of expertise (and expert processes) is to call into question traditional notions of rigor, to privilege time and space for activity that initially looks decidedly non-expert. In her study of student learning in an honors course in arts literacy, Paula Berggren (Baruch College) had one of those moments that shifts a teacher’s perspective. After weeks of asking formal questions about the arts and getting strained responses, she began a new unit with an online discussion question about “why people dance.” What ensued was a torrent of student responses, at first glance free-form and at times rambling. However, upon close analysis, Berggren realized that the postings displayed many of the intellectual principles of arts literacy that she was hoping to cultivate all along, albeit informally and incompletely shaped. The episode revealed to her the power of an open discussion space for students to engage a question (Why do people dance?) that stood at the boundary between their course’s academic topic (the arts), the related disciplinary discourse, and the students’ personal experiences. The value of such timely questions unfolding in a technology-enhanced context that affords room to play emerged over and over again in our classroom inquiries.

The fundamental insight to be emphasized here is the linkage between flexible space for intellectual play and the development of disciplinary expertise. Examining the value of online discussion spaces in a biology and literature course, respectively, Heidi Elmendorf and John Ottenhoff discovered that “one of the key signs of intellectual play within the online discussions that emerges during the semester is the students’ comfort with ambiguity and their ability to play with and build upon ideas that are not certain (deferring true understanding)…This flexibility of cognitive engagement is an important accomplishment--and a critical embodiment of expert behavior--for students who begin the semester by seeing texts as sources of compartmentalized knowledge, not as opportunities for expansion and questioning.”¹³

VKP faculty put students in positions of freedom. But they found it critical to carefully scaffold these open experiences. Navigating between being too restrictive and too vague, many VKP studies analyzed the effectiveness of particular scaffolds in cultivating authentic learning. Taimi Olsen (Tusculum College) studied the ways her students conduct research on the Web. Research, archival or otherwise, is an important site for engaging novice learners in expert-like activity, posing challenges of open and authentic work balanced by scaffolded guidance. In her study, Olsen is particularly interested in the relationship between serendipity (the accidents of research discovery intrinsic to the scholar’s experience) and reflective judgment (the small, intermediate moments when students make decisions about the appropriateness of a source or the generative quality of a research direction). Through her own cycles of inquiry, Olsen came to appreciate the importance of scaffolding to produce both experiences, creating specific parameters for the way students search for materials and how they record and reflect on that research. Through her inquiry, Olsen came to see what many other VKP faculty discovered: that the development of disciplinary skills largely takes place in these intermediate moments of reflective judgment. We cannot expect students to cultivate the capacity for such judgment unless we put them in structured scenarios with freedom of decision, building awareness of the consequences of better or worse judgment.

➢    Designing for adaptive expertise must include ways of capturing intermediate processes through student work.
If we value the intermediate learning processes that lead to summative student work, then we have to invent new ways both to foster activity in those intermediate spaces and to capture evidence of them. VKP faculty were struck by what could be learned from listening to students as they work in these intermediate spaces. Taimi Olsen found that if she gave weight to “research skills” as a real goal of her teaching, then she had to find meaningful ways of coaching those skills. She asks her students to keep research logs and to turn in copies of their sources with marginalia, generating important evidence of student thinking and development. In her inquiry, student marginalia constitute an important source of evidence of student reflective judgment. Revealing student rationale, these notes often gave a reasoned basis for choices that otherwise appeared at odds with Olsen’s own biases about sources. Careful reading of marginalia may not be for all faculty, but Olsen’s commitment to it suggests ways we must rethink the location of student learning.

Online discussion spaces, including emergent Weblog and wiki environments, also provide promising venues for intermediate processes. VKP studies explored how to move beyond conversational activity, creating occasions for students to harvest learning from the visibility of their own thinking. In Ed Gallagher’s initial inquiry project (Lehigh University), he decided that the ability to “enter the conversation” in a disciplinary context was a fundamental academic goal. Putting his course design where his principles were, Gallagher redesigned an American Literature course to center around an online discussion board, where 100% of the grade was based on discussion-intensive participation. A critical component for student learning was what Gallagher called their “meta-work:” each week, students had to reread and reflect on their best contributions and compare their own judgments with the professor’s. Olsen and Gallagher’s projects are two among many where faculty focused on new sites for gathering evidence of student progress taking advantage of the ways digital spaces made these processes visible. 

➢    Designing for adaptive expertise necessitates developing new ways to read and assess student work.
If designing for intermediate processes requires new places to look for evidence, it similarly compels us to find new ways of looking. Despite the shift in the last fifteen years around active learning and the use of online tools to engage such activity, we (as a higher education community) have made comparatively little progress on changing our summative assessments of student learning. Expanding our criteria for assessment is a further logical consequence of attending to intermediate processes and abilities we associate with adaptive expertise. This in no way implies a lessening of standards but an important acknowledgment that we must recalibrate our evaluative judgments to the places in the developmental process we are most hoping to shape.

For some faculty engaged in these inquiries, this recalibration was modest. For example, judging the kinds of intellectual moves that should be rewarded Paula Berggren found there are “different rules for reading online postings” than for formal papers. Ed Gallagher changed the assessment criteria in his discussion-centered course experiment, privileging academic writing that opened up into conversation and invited response above superficially more competent but closed expository discourse. Focused on strategies for using online collaboration as tool for building writing skills Juan José Gutiérrez, at CSU Monterey Bay, assigned nearly one-third of his course grade to students’ participation in a carefully structured peer review process.

But the implications may be more far-reaching. Thinking about the problem of assessment in the context of innovative new media practices led us to what we came to think of as “the cutting room floor problem.”¹⁴ Extended projects in digital environments--such as digital storytelling, documentary video production, multimedia authoring--often left faculty with the paradox that the richest evidence of student learning cannot be found in the final summative product (e.g. the five-minute multimedia narrative itself). Where is the learning in a process-driven authoring activity? In a twenty-page research paper, we want to believe that all the evidence of learning is in the final product. But what about a five-minute documentary video introducing the Tuskegee experiments to an audience who never heard of them? Where is the full evidence of that process? In many ways it is spread across a series of actions taken along the way and left on the cutting room floor, in the hundred decisions a student makes to include or exclude materials or effects, in all those reflective judgments made in the process of construction and (in some cases) the give and take of collaborative production. For many VKP faculty, this required rethinking “final projects” as compilations of a final assignment along with traces of the process, most commonly a reflection or series of reflections.

Such reflections can serve as a bridge between theory and practice. Rina Benmayor (CSU Monterey Bay) sees digital storytelling as a way to link theory and “the body” (personal experience and identity). Manifesting her thinking in her classroom activities, Benmayor developed a rich schema for evaluating student multimedia work on multiple levels--narrative, analytic, technical--and then had students reflect, explaining how their stories embody the theory they have been studying. Implemented repeatedly, over a series of semesters, her innovations allowed her to see how some students excel at certain dimensions and not at others; consequently, she was able to change, and in many ways, intensify her expectations, identifying the layers of their development and achievement.

The need for an expanded understanding of assessment will only increase with the expanded use of blogs, wikis, social bookmarking, virtual reality and simulation environments in education. Any pedagogy linking adaptive expertise to the kinds of processes made visible by new social tools especially depends on recognition of the increasingly expansive ways that previously invisible processes have status in artifacts and assessments. Participatory environments and constructive spaces offer us new ways to teach to adaptive learning and to access incremental stages of student development along the road to expert-like thinking. We can realize this potential only if we recognize how such spaces enable less than expert students to engage in “confronting and negotiating sites of understanding.”¹⁵ Assessing what happens when students with novice knowledge meet expert-level challenges is among the greatest imperatives faculty face in responding to the expanding field of visible evidence.

[jump to Part 3]


Notes
8. For example, experts notice patterns not noticed by novices; expert knowledge is organized in ways that reflects their domain; expert knowledge is “conditionalized” (not about isolated facts but situated in ways that new data make sense in larger contexts); experts are flexible and fluent with their knowledge. See John Bransford, et. al., How People Learn: Brain, Mind, Experience, and School (National Academy Press, 1999). Other work on expertise has looked, not at the differences between experts and novices, but at the differences between practicing experts and practicing non-experts. See Carl Bereiter and Marlene Scardamalia, Surpassing Ourselves: an Inquiry into the Nature of Expertise (Open Court Publishing Company, 1993). [return to text]
9. Giyoo Hatano and Yoko Oura, “Commentary: Reconceptualizing School Learning Using Insight From Expertise Research,” Educational Researcher 32, no. 8 (2003): 26-29, http://edr.sagepub.com/cgi/content/abstract/32/8/26; S. Brophy, L. Hodge, and J. Bransford, “Work in progress--adaptive expertise: beyond apply academic knowledge,” Frontiers in Education 3 (FIE 2004, 34th Annual, 2004): S1B/28- S1B/30, http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1408679. [return to text]
10. See for example, Lee Shulman, “Pedagogies of Uncertainty,” Liberal Education (Spring 2005), http://www.aacu.org/liberaleducation/le-sp05/le-sp05feature2.cfm, and William Sullivan and Matthew Rosin, A New Agenda for Higher Education: Shaping a Life of the Mind for Practice, (Jossey-Bass, 2008). [return to text]
11. James F. Slevin, Introducing English: Essays in the Intellectual Work of Composition (Pittsburgh: University of Pittsburgh Press, 2001). [return to text]
12. Levy’s work builds on the pioneering work on difficulty developed by Mariolina Salvatori, and then Salvatori and Patricia Donahue, who found that given the opportunity to make their sense of difficulty visible, students discover that much of what they find difficult in complex texts are indeed often “textual cruxes.” See for example Mariolina Salvatori, "Difficulty: The Great Educational Divide," Opening Lines: Approaches to the Scholarship of Teaching and Learning (Carnegie Publications, 2000) and Mariolina Salvatori and Patricia Donahue, The Elements (and Pleasures) of Difficulty (The Elements of Composition) (Upper Saddle River, New Jersey: Pearson Education, 2004). [return to text]
13. Heidi Elmendorf and John Ottenhoff, "The Importance of Conversation in Learning and the Value of Web-based Discussion Tools," Academic Commons (January 2009), http://www.academiccommons.org/commons/essay/importance-conversation-learning. [return to text]
14. See Heidi Elmendorf, “Learning through Teaching: A New Perspective on Enterning a Discipline,” Change (Nov/Dec 2006). [return to text]
15. Bernie Cook, "Producing Audiovisual Knowledge: Documentary Video Production and Student Learning in the American Studies Classroom," Academic Commons (January 2009, forthcoming). [return to text]