A Long-Term Look at Multiple Measures Assessment: Impact Findings from a Randomized Controlled Trial

In 2015, researchers from the Center for the Analysis of Postsecondary Readiness (CAPR) initiated a randomized controlled trial (RCT) at seven community colleges in the State University of New York (SUNY) wherein three cohorts of incoming students were randomly assigned to one of two groups: one receiving multiple measures assessment (MMA) placement (program group students) and the other receiving the existing test-based placement (business-as-usual students). The aim was to learn whether MMA, in which colleges use high school GPA and other measures in addition to placement test scores to assign incoming students to either developmental or college-level courses in math and English, yields placement determinations that lead to better student outcomes than a system based on test scores alone. The current study tracks students for up to nine terms after an algorithm model of MMA to observe whether the positive impacts of MMA were sustained over time. Primary outcomes of interest include college-level enrollment and completion, as well as credit and degree attainment.

Findings reveal that students placed using MMA experienced higher rates of enrollment in college-level coursework, regardless of subject area. These impacts were greater for students who were given a college-level placement under MMA, with some of these students earning associate’s degrees at higher rates. Larger and, in some cases, longer lasting impacts were associated with lower algorithm cut scores in English, with program group students being more likely to enroll in and complete college-level English. We observe the opposite in math, wherein cut scores were set higher, preventing students from meeting those cut scores and subsequently requiring enrollment in developmental courses. In conducting subgroup analyses by race/ethnicity, Pell recipient status, and gender, we did not observe the closing of gaps between subgroups. While we found improved outcomes among individual subgroups, those improvements did not shrink the pre-existing academic differences between groups. With few exceptions, this suggests that MMA alone is not sufficient to remediate long standing disparities that occur in higher education.

The results previewed here further support the use of MMA as compared to traditional test-based systems, with a few specific lessons for policy and practice. First, findings from the current study suggest that increased access to college-level courses improves students’ chances of completing college-level math and English courses. In other words, MMA’s potential to improve student outcomes is explained by the redistribution of students from developmental courses to college-level courses. Secondly, although research has shown that many students will succeed in college-level courses when given the chance to enroll in them, we find that the impact of MMA on student outcomes decreases over time. Therefore, to bolster long-term student success, MMA should be implemented alongside additional student supports. Finally, MMA has the potential to address disparities by race/ethnicity, socioeconomic status, and gender if placement systems are designed with inclusive placement measures and implemented alongside targeted support programs post-placement to meet the needs of specific populations as they navigate college.

Participants

Elizabeth Kopko, Community College Research Center and CAPR