CAPR Holds Second Annual Meeting
The Center for the Analysis of Postsecondary Readiness (CAPR) held its second annual meeting on March 8, 2016. CAPR researchers met with advisory board members, key personnel from CAPR research sites, and other national experts on remediation to discuss evolving changes in developmental education, the current state of research knowledge about innovative practices to improve outcomes for underprepared students, and the progress of current CAPR projects.
CAPR, a partnership between the Community College Research Center (CCRC) and MDRC funded by the Institute of Education Sciences, is conducting three major studies: a descriptive study using survey and interview data on the landscape of developmental education at two- and four-year colleges across the nation; an evaluation at several New York community colleges of an alternative assessment model that uses multiple measures to decide if students should enroll in developmental courses; and an evaluation at several Texas colleges of the New Mathways Project, an alternative approach to traditional developmental math that emphasizes acceleration and college-level math pathways that are more aligned with social sciences, health, and liberal arts professions. Supplemental studies will look at the Early Start policy in California, which requires students to complete remediation during the summer prior to their freshman year, and the Emporium Model of computerized developmental math instruction in Tennessee.
Thomas Bailey, Principal Investigator, CAPR
The studies are being conducted as community colleges, university systems, and states around the country are experimenting with new methods of assessing and teaching students in response to research showing that traditional methods are often ineffective and sometimes harmful for students. “The whole landscape of developmental education is changing rapidly,” said Thomas Bailey, the director of CCRC and principal investigator of CAPR, as he opened the meeting at Teachers College in New York City.
Lashawn Richburg-Hayes, Co-Principal Investigator, CAPR
A year and a half into the five-year life of the center, researchers are administering surveys, conducting interviews, and implementing randomized controlled trials for the evaluations. In the descriptive study, researchers have started to look at early information that has been gathered from more than a hundred interviews with college officials about their assessment practices, instructional strategies, readiness standards, the use of technology, and other aspects of their programs for underprepared students. One early finding is that colleges often have different ideas about what it means to use multiple measures in assessing students. Indeed, some college representatives reported that multiple measures simply meant that they are expanding the exemption criteria for letting students avoid placement tests.
CAPR’s multiple-measures assessment study is investigating the effects of a placement system in which an algorithm is used to weigh several factors, including high school GPA, to indicate readiness for college-level work. The study will provide timely information to help colleges move beyond simple placement tests. At the meeting, participants generally felt that better predictors of success should be sought out and used for placement into college-level courses. Discussion of the study spurred questions about how to deal with older students, whose GPAs may lose their predictive power several years out, and about how colleges communicate with students about the high-stakes nature of placement tests. Past research has found that many colleges downplay the importance of the tests and that students often do not prepare for them.
Participants, including John Hetts, senior director of data science at the Educational Results Partnership, and Brad Bostian, professor at Central Piedmont Community College, also discussed the utility of the aggregate high school GPA, which has been found to be a much more reliable predictor of how a student will perform in college than other measures, even when it is several years old. This is likely because GPA represents many measures of performance across subjects averaged over time, and may reflect success skills that go beyond content knowledge.
Participants at the meeting also discussed CAPR’s study of the New Mathways Project, which was developed by the Charles A. Dana Center at the University of Texas at Austin and is being implemented in 13 states. New Mathways lays out different math pathways that tailor the type of math students learn to different majors and careers, and it accelerates remediation to get students into college-level math in less than a year. The CAPR study is examining the difference between New Mathways courses and traditional courses, whether students do better in New Mathways courses, and the effects on costs. Discussion about New Mathways focused on two overarching topics: how changes in pedagogy and curriculum may or may not relate to changes in student performance, and the best ways to train teachers in implementing new practices; and how to relate the lessons learned from the New Mathways evaluation to other multiple math pathways reforms that may have differing designs. Participants discussed the challenges of implementing new pedagogical methods such as “active learning” and “problem solving,” and the best ways to support teachers in learning these methods through professional development.
Participants considered how changes in structure, such as accelerating developmental education courses, versus changes in instruction and content, such as adding different math pathways, may or may not contribute to changes in student outcomes. CAPR researchers said that while there seemed to be relatively strong fidelity to the model at the study sites in terms of teachers using contextualization, active learning techniques, and the like, in general, instructional innovations spelled out in a reform model do not always make it into the classroom. Moreover, it’s a major challenge to measure changes in teaching.
Others at the meeting mentioned examples of training that effectively introduce instructors to new methods. Steve Hinds, the director of Active Learning in Adult Numeracy, noted that teachers at CUNY Start—a prematriculation program at the City University of New York designed to help students with substantial needs in basics skills succeed in college—spend a semester as participant observers with master teachers so they can learn ways to, for instance, have students do math in the class rather than solely listen to lectures.
Karon Klipple, the executive director of Community College Pathways at the Carnegie Foundation for the Advancement of Teaching, said she has seen two approaches to training at schools scaling alternative math pathways: one school including regional professional development structures that feature workshops and mentorships and another based almost exclusively on written materials.
“We’ve seen about a 20 percent difference in success rates between those two schools,” she said.
Participants also discussed some of the structural challenges that can arise with students who have not yet decided on a major or who may want to change pathways. The New Mathways Project has dealt with this challenge by creating one foundational developmental education course for all three math pathways and building supports into the college-level algebra course pathway for STEM students. Myra Snell, a math professor and a leader of the California Acceleration Project, said she has found that students may not know what they want to do, but they know what they don’t want to do, which is enough to guide them into a “meta-major” and an associated math pathway. Still, Uri Treisman, the executive director of the Dana Center, said, we need to know a lot more about how students make choices and how educators can help them.
Participants at the meeting also highlighted another general issue: the need to make college leaders understand the need for reform and to make them feel more comfortable with the kinds of changes being proposed in projects such as those under study at CAPR. Lashawn Richburg-Hayes, director of MDRC’s higher education policy area and co-principal investigator of CAPR, said that while the colleges that CAPR researchers are working with are receptive to new ideas, personnel at other colleges may not be knowledgeable about these kinds of innovations or even about the evidence on the effectiveness of traditional developmental programs, making educating the educators an important part of any developmental education reform. The projects in CAPR are well suited to contribute to the pressing discussions and needs of such educators.