Supplemental Study: Exploration of the Use of Learning Technologies in Developmental Math

July 2014–June 2020

Developmental math has a higher failure rate than any other course in higher education. Yet we know little about the effects of increased technology use in developmental courses. This study evaluates the adoption of a hybrid emporium model for developmental math courses taught at all public colleges in Tennessee.

The hybrid emporium model replaces traditional lectures with interactive instructional software and on-demand personalized assistance. Students spend much of their class time in a computer lab learning the course content online at their own pace, with faculty serving more as tutors who deliver individualized instruction, as opposed to lecturers. Much of the course content is divided into smaller blocks, or modules, and is taught using tutorials, practice exercises, and online quizzes and tests. When students fail to master a concept, they are allowed to complete the module again, with new problems replacing the old. This technology-aided instruction is made possible through the use of an online interface created by third-party providers such as Pearson or ALEKS.

Using nine years of student enrollment and transcript data (2006–07 to 2015–16) provided by the Tennessee Board of Regents, this study examines the effects of the model on student outcomes. It also examines how students and faculty experienced the model. (In the years since the adoption of the hybrid emporium model in Tennessee, the state has moved toward a credit-bearing, corequisite model for offering developmental courses. Since the fall of 2015, all developmental math courses [at both two- and four-year colleges] are taken in the same semester as a college-level math course.)

The Study Design

This study makes use of survey data collected from the institutions, interviews and focus groups, and transcript data from the institutions and the Tennessee Board of Regents. The analytic strategy for the quantitative analysis involves subtracting a pair of mean differences to obtain an estimate of the effect of being assigned to a developmental math course taught using the hybrid emporium model on student outcomes. This difference-in-differences method of analysis compares otherwise similar colleges that were “early adopters” and “late adopters” of the model. Surveys, site visits, and interviews at a subset of colleges are used to examine how students and faculty experienced the adoption of the model.

Student Perceptions of the Model

In focus groups at six colleges in Tennessee, students said they perceived math content and their faculty to be more accessible in the technology-centered model than in traditional lecture courses. The perceived stronger connection with faculty is notable because it was not an intended goal of the model. Students said the model:

• Gives them a greater sense of responsibility for their own learning
• Allows them to access content and support materials any time and gives them flexibility in when and where they complete assignments, though older technology can make that challenging
• Gives them access to various explanations of math concepts
• Allows abundant opportunities for practice
• Connects them with immediate feedback
• Opens up avenues to connect with faculty
• Makes them feel less intimidated by math and more in control of their math learning

Findings on Outcomes

While students in the study focus groups perceived math content and their faculty to be more accessible under the technology-centered model, the quantitative analysis suggests that the model does not improve math performance.

For community college students, being assigned to a hybrid emporium developmental math course led to lower pass rates in their first college-level math course, fewer cumulative credits earned over time, and a lower likelihood of earning an associate degree within six years, as compared to students assigned to traditional developmental math courses.

At four-year colleges, the adoption of the model resulted in a higher percentage of students passing their developmental math courses and thus spending fewer terms in developmental math. However, the pass rates of these students in their first college-level math courses were lower than those of students who were assigned to traditional developmental math courses. The magnitude of the effects varied by gender, age, and ACT math score.

This research was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305C140007 to Teachers College, Columbia University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.