All of the individual quiz questions in the Waymaker courseware are tied to an identified skill, which in turn is linked to a learning objective. Learning objectives are linked to module outcomes, which are themselves linked to course outcomes. Thus, each individual quiz question is connected in some way to a production-based learning outcome in WRIT 100/101. One concern faculty may have about the courseware is that the quizzes are only valid assessments of module content knowledge and that there is no substantive relationship between module content knowledge and writing skill. However, if we accept the validity of our course outcomes, as measured with rubric-based assessment of student writing, we must also accept the validity of these quiz questions as a measure of essential rhetorical skills.
This question appears in the analysis module quiz:
Which of the following best describes how evidence should function in an analytic writing?
Here is the full skill map for this question:
|Skill:||Recognize keys to successful analysis writing|
|Objective||Recognize and evaluate keys to successful analysis writing|
|Module Outcome:||Evaluate keys to successful analysis|
|Course Outcome:||Exploration and Argumentation: Students will use writing and other modes to analyze texts, explore unfamiliar ideas, engage with thinking different from their own, develop sound arguments, and reflect.|
All of the skills and objectives in the courseware target the first two levels of Bloom’s taxonomy. If we accept the validity of Bloom’s taxonomy, foundational knowledge and comprehension are prerequisite for application, which is in turn prerequisite for analysis, evaluation, and other higher-order knowledge work. Students cannot analyze texts in their own writing until they can evaluate keys to successful analysis. In order to evaluate, they must first recognize those keys.
When we evaluate student writing, we base our assessment on an application of learning objectives from the top three tiers of Bloom’s Taxonomy: Evaluation, Synthesis, and Analysis. For example, when we decide if a student’s thesis statement is “supported by sound reasons” or if it “demonstrates awareness of the depth of the issue,” we’re looking at the student’s application of higher-order critical thinking skills. Are they able to construct a logical thesis that is supported by the right mix of evidence which targets a specific time and an identified audience?
Does assessing higher-order skills necessarily tell us about a student’s lower-order skills? If a student cannot construct a thesis statement with sound reasons, can we know conclusively that it’s because he or she doesn’t understand logical fallacies? In the strictest sense, we cannot know. The instrument of assessment– our rubric– is not a valid measure of the lower-order skills. It’s only a valid assessment of the criteria it explicitly measures.
Since students have gaps in knowledge or other significant preparation barriers coming in to college writing, it is important that we know where those gaps are. Assessing lower-order skills does not detract from our ability to teach and evaluate higher-order skills. A concern we hear about the courseware is that the quizzes just do not relate to the work the students are actually doing in class. This perception is not accurate: every quiz question in the modules aligns to specific skills, objectives, and learning outcomes. They target the three lower tiers of Bloom’s taxonomy. This is by design: when we designed the courseware, we wanted to focus on lower-order skills. We already know that student writing itself is the best way to measure high-order skills. Nobody would try to replace reading and responding to student writing with multiple choice quizzes. Rather, these modules serve to measure skills that we were not otherwise looking at explicitly. Previously, we assessed this foundational rhetorical knowledge through assumption and guesswork. All the quizzes do is provide actual data about how students understand basic rhetorical skills.
Take a look at the chart below. It breaks down one of the WRIT 100/101 rubric categories into specific outcomes and traces how the courseware skills align to those outcomes. Notice how the rubric targets the top of Bloom’s taxonomy while the courseware skills target the bottom.