University of Mount Saint Vincent NSF IUSE Hispanic-Serving Institutions Program Evaluation:

Evaluand is providing comprehensive, independent evaluation support to a National Science Foundation project funded by a grant to the University of Mount Saint Vincent (UMSV), under the Improving Undergraduate STEM Education: Hispanic-Serving Institutions (HSI) Program. UMSV was awarded a pilot project to implement and test a model of academic supports incorporating supplemental weekly, one-hour recitation sessions into general chemistry and organic chemistry sequences, implemented in conjunction with a two-week, literature-based guided-research summer project.

Intended outcomes for the program include increased goal setting, chemistry problem-solving skills, scientific literacy, and understandings of course connections to the real world; all aiming to improve academic performance, course completion, and persistence in among chemistry or biochemistry majors—especially inner-city students of color. Evaluand is applying a collaborative evaluation orientation to engage the project team in methodological planning, data collection, and analysis to support the research aims of the grant.

The evaluation design applies a theory-based approach, with questions to be answered aligned with a logic model developed by Evaluand in collaboration with the UMSV project team. Study of the curricular supports being piloted will use a mixed-method approach to produce both formative feedback (assessing progress and providing input for improvement) and summative results (documenting the model’s outcomes and evaluating success in achieving its aims). Given that it is a pilot study, outcomes for the project are limited to near-term, measurable results. This is appropriate for a project structured as an Exploratory Research and Development study, designed to (a) investigate changes to the recitation model as informed by past research, (b) better understand the factors involved in implementing new model components, (c) iterate design of activities and resources to support the model, and (d) gather preliminary data for a potential follow-on proposal.