Moving to Multiple Measures
By Vikash Reddy and Elisabeth Barnett
Standardized tests have long been an option for colleges deciding how to place students in developmental education, but their popularity began to grow during the late 1980s. This trend continued over the next few decades until, by 2010, 100 percent of community colleges reported using standardized tests to place students in math, and 94 percent reported doing so in English.
In recent years, researchers have begun to ask whether placement systems limited to tests such as ACCUPLACER, the recently discontinued Compass exam, and other similar single assessments are assigning students to the correct level of coursework. Recent studies covering a range of contexts have found that systems using such tests as a sole measure tend to place substantial numbers of students into developmental coursework who would likely have been successful in a college-level course. As a result, many colleges are turning toward using a combination of data, such as test scores, grade point average (GPA), and other high school transcript information, to more accurately place students. However, there are several steps to designing such a “multiple measures” system.
In a new working paper released by CAPR, we discuss the history of college placement testing at nonselective colleges and some of the limitations of current practices. We then set out a typology of approaches that colleges and universities can use when developing a multiple measures system for assessment and placement: data points that can be used in placing students, ways in which these measures can be combined, and more varied ways in which colleges might employ assessment results.
What placement measures could be used?
While traditional placement exam scores (mainly ACCUPLACER) are currently the dominant tool used in placement decisions, there are a number of choices that colleges may consider. Virginia and North Carolina are examples of states that have developed their own assessments and placement systems, but developing an effective instrument and conducting the requisite validity research can be costly. Colleges and universities are also turning to diagnostic tools such as ALEKS, which give more detailed information about student strengths and weaknesses. These assessments are often paired with modularized instruction that allows students to focus their time on improving areas of weakness while limiting time spent on concepts they already understand.
In addition, a number of assessments are available that measure noncognitive traits, such as SuccessNavigator, Engage, the Learning and Study Strategies Inventory, and the College Student Inventory. While such assessments may provide useful information, limited research has been done on their use for placement purposes. Colleges may also administer writing assessments, computer skills assessments, and other questionnaires to gauge students’ skills and confidence in their academic ability.
Beyond these alternative or supplemental assessments, colleges can also turn to high school transcripts or other indicators of prior achievement. Some of the studies mentioned earlier have pointed to the strong relationship between high school grade point average and performance in college-level coursework. Colleges may also give more granular consideration to transcript information. North Carolina, for example, considers students college-ready in math if they have taken four approved math courses and maintained an overall GPA of at least 2.6.
Colleges can also make use of standardized tests that are commonly administered to high school students, such as the SAT and ACT. Additionally, assessments linked to the Common Core State Standards, most notably those developed by PARCC and Smarter Balanced, can give colleges and universities valuable information about students’ college readiness.
Institutions looking to incorporate these types of measures—in particular GPA or other transcript data—into their placement processes face the substantial challenge of ensuring the data will be available in a timely fashion. Alternatively, colleges can make use of self-reported high school GPA.
How can measures be combined to reach a placement determination?
When more than one measure or data point is used for placement, a system must be created to use them in combination. Several options are possible. First, colleges may waive placement testing requirements for students who can demonstrate proficiency on an alternative measure or a combination of alternative measures, as in Ohio’s “remediation-free” standards.
Once a student is referred to the testing center, however, there are still several ways the test scores can be employed. Some systems, like the type of placement formula being tested by CAPR, incorporate multiple measures simultaneously, weighting each measure according to its historical relationship with success in a given college-level course. While such formulae can be constructed with or without placement test scores, our own initial analyses indicate that models that include test scores and high school GPA explain a greater share of variance in outcomes than models with either measure by itself. A college may also decide to evaluate further evidence of student preparedness if the student falls within a given range on a primary measure.
Systems and individual institutions are also experimenting with directed self-placement. Self-placement systems allow students to select their own level of coursework, in some cases after seeing the results of their placement tests. Florida moved to this type of placement system statewide in 2013. Initial evidence suggests mixed results for the policy, but additional research (e.g., at Florida State University) is underway to help us better understand the implications of self-placement systems.
What else could assessments be used for?
In some cases, especially when they include a noncognitive assessment among their measures, colleges may use initial assessment results to place students into support services as well as into appropriate math and English courses. Some colleges are also looking at how to best place students into alternative courses, such as corequisite courses or different math pathways.
* * *
The developmental education space is the subject of a number of reform efforts, typically targeting instruction and support services. But there is also growing experimentation around assessment and placement, and the importance of finding an accurate way to place students is hard to overstate.