> About Us > Faculty > ACE Effectiveness
ACE Effectiveness
From initial program research to daily feedback within instruction, the Academy for College Excellence (ACE) uses a wide range of data and evidence. Quantitative and qualitative data are regularly gathered, analyzed and applied. Along with internal studies, more than ten studies have been conducted by external evaluators. Those conducted before 2010 refer to the program as Digital Bridge Academy, the name changed to the Academy for College Excellence in 2009. ACE is not only a research-based practice, but shifts the paradigm to practice-based research, with questions and inquiry shaped by what is learned in practice.
Student Outcome Studies: The first and most compelling question about ACE is the effect on students: does participation in ACE increase students’ academic performance, persistence, and success? We will look at four studies below.
Faculty Studies: The core question is the effect on teachers in their professional lives, how teachers integrate ACE principles in their classrooms. One more question was important early in the evaluation cycle: some successful innovations depend on the talent and charisma of their developers. Could other teachers successfully teach ACE classes, particularly the Foundation Course. We will look at four studies below.
Student Outcome Studies
- Evaluation of the Academy for College Excellence: Year 1 Interim Report (2011)
- Educational Outcomes of Cabrillo College’s Digital Bridge Academy: Findings from a Multivariate Analysis (2009)
- Watsonville Digital Bridge Academy Report I: Student Outcomes Evaluation Cohorts 1 & 2 (2005)
- Watsonville Digital Bridge Academy Report 2: Persistence And Achievement (2007)
MPR Associates Inc. is conducting a comprehensive evaluation of ACE programs at six colleges.
The ACE semester begins with a two-week full-time Foundation Course that focuses on communication and work styles. This initial educational experience is designed to ‘light the fire of learning.” ACE believes that affective changes are the preparation and precursor for full-time academics. Working with ACE personnel and University of California Santa Cruz Psychology Professor (emeritus) Martin Chemers, MPR created the College Student Self-Assessment Survey (CSSAS) an instrument that measures students’ perspectives on themselves. This work is also reported in Change Magazine, in an article called “Supporting the Students of the Future” by Diego Navarro, the Director of the Academy for College Excellence. 
Change Over Time in ACE Students' Scores on CSSAS Scales, Fall 2010 and Spring 2011;
* = statistically significant
The Foundation Course focuses on building students’ capacity in each of the affective areas. Results show that this training is particularly effective in the dimensions of Self-Efficacy, Teamwork, Identity, Interaction, Mindfulness, Describing, Accepting, and Observing. The last four variables—mindfulness, describing, accepting and observing—are aspects of mindfulness as measured in the Kentucky Inventory of Mindfulness Skills used by researchers in medicine.
Based on CSSAS results over time, interventions early in the semester—between Time 1 (before starting ACE) and Time 2 (after the Foundation Course)—provide the base for change and remain constant or grow slightly over the course of the Bridge Semester, as measured at Time 3(end of semester). The change is statistically significant.
One more finding to date is that ACE students report ambitious long-term academic goals. Over half of the students hope to earn an Associate’s degree, ver one-third hope to earn a Bachelor’s degree, and nearly half hope to attain a Master’s or higher post-graduate degree.
MPR is currently analyzing longitudinal college transcript data on student performance and will report findings in the first half of 2012.
__________________________________________ Top of Page
Community College Research Center (CCRC) at Columbia University released a report in 2009 of a longitudinal study comparing ACE students at Cabrillo College with a comparison group of 11,578 non-ACE Cabrillo College students. The outcomes of ACE students were separated into two groups: students in cohorts 1-3 who took an accelerated English class that was one-level-below transfer English as part of the ACE semester, and students in cohorts 4-9. Cohorts 4-9 did not take an accelerated English class like cohort 1-3 because it was determined that their enrollment in a course more advanced than that indicated by the assessment/placement process was prohibited by the California Educational Code. The reading lab course did not count as an English course, therefore the ACE students lost one semester of English progression, yet as the study indicated, they still progressed in English just as well as non-ACE students. The results were the following:
The graph below shows results of the accelerated model for Cohorts 1-3.
The results in the table below show a statistically significant positive difference between ACE students—both accelerated and non-accelerated—and the comparison group. The exceptions to this pattern are there are no significant differences between non-accelerated ACE students and the comparison group in passing English courses one level below transfer and transfer level, and in transfer credits earned. In these areas, performance matches the comparison group.
| Comparison Group |
Accelerated ACE Students | Non-Accelerated ACE Students | |
| College Credits Earned | 28 | 49 | 37 |
| Transfer Credits Earned | 21 | 25 | 23 |
| Persisted 1 Semester | 80% | 95% | 88% |
| Persisted 2 Semesters | 63% | 82% | 74% |
| Enrolled Full-Time in Second Semester | 33% | 65% | 59% |
| Passed 1 Level Below Transfer English | 29% | 71% | 26% |
| Passed Transfer-Level English | 37% | 68% | 31% |
“In summary, this study found significant positive effects for participation in both the accelerated and non-accelerated versions of the DBA…
“To the extent that colleges are seeking strategies for increasing the rate at which academically underprepared students complete 'gatekeeper' courses such as college-level English and earn college credits, the accelerated version of the DBA program seems to hold particular promise.
“While the results of this analysis show that participation in the DBA program is correlated with better outcomes on most measures for students…DBA students are very likely more at-risk than are Cabrillo students with similar levels of academic preparation…
“To the extent that the DBA students in the sample were substantially more disadvantaged than other students, [see demographics] it may be that the estimates produced through this analysis understate the effect of participating in DBA.”
Note on demographics: Close to two-thirds of ACE students have at-risk characteristics such as being English Language Learners, being the first in their family to go to college, or placing two levels below transfer-level in English, mathematics, or reading. Approximately one-quarter of the students have high-risk factors such as not being a high school graduate or GED holder or being a parent of dependent children. Fifteen percent of the students have received public assistance.
__________________________________________ Top of Page
In these evaluations funded by the National Science Foundation, Dr. Norena Badway of the Higher Education Evaluation and Research Group conducted two sequential evaluations on the characteristics of Digital Bridge Academy students and their personal and academic growth related to their participation.
These studies used the metrics of academic progress: GPA, retention through the semester, progress toward an award, and persistence to the next term. The evaluators viewed these outcomes not only in terms of academic progress, but as markers of self-efficacy indicating confidence that the student can succeed in college. Their findings were that participation in DBA had substantial impact on those four measures.
For the initial two cohorts, which experienced the initial accelerated design, retention in the Bridge Semester was 83% and 79% respectively. Because of the tightly linked curriculum, students who completed the Bridge Semester earned 19.5 college credits.
The second report summarized the longitudinal effects for the five DBA cohorts from Fall 2003 to Fall 2005 and found similar results.
The retention rate for first semester DBA students varied from 74-100%, compared to retention rates of 60% for all Cabrillo basic skills students and 67% for students in credit-level coursework. The majority of DBA students completed 16 credits with greater success than did their non-DBA peers at Cabrillo.
These findings alone commend the DBA program: under-prepared, at-risk students make marked progress towards academic, career and personal goals. The DBA evidence suggests a need to re-examine beliefs about sheltering at-risk underprepared students.
Read the full reports:
__________________________________________ Top of Page
Faculty Studies
In addition to studies of student outcomes, external evaluators have studied effects of participation in ACE professional development by faculty and administrators. At any college implementing ACE, faculty who are going to teach an ACE class participate in a five-day Faculty Experiential Learning Institute (FELI). Other teachers and administrators at the college may also choose to participate for their own professional growth. The curriculum, structure, and delivery of the FELI are similar to the Foundation Course. This is not only so that faculty will empathize with the students or understand the content, but because they need to experience and embody the changes they hope to see in their students. We will review four studies below.
- MPR Associates Inc., as part of their comprehensive evaluation are conducting a study on the effects of the FELI, which will be released early in 2012.
- In Becoming an ACE Teacher (2011) Dr. Rose Asera qualitatively describes three aspects—and expectations—of teaching in ACE that are different from typical community college teaching:
- Getting to know students well, with explicit attention to affective dimensions of teaching and learning
- Using a scripted curriculum for the ACE signature courses
- Working collaboratively with faculty colleagues
- Passing the Torch: An Evaluation of the Digital Bridge Academy Replication (2006) Driven by concerns that many successful programs rely on the talents and skills of a charismatic leader, CJTC researchers examined the questions of whether the Foundation Course could be taught successfully by faculty other than the DBA Founder and Director. The study looked at the effectiveness of the faculty training and the faculty experiences with the DBA curriculum. They also examined whether the Foundation Course could be adapted successfully for other ethnicities, beyond the Latino population in Watsonville, and in urban or suburban locations.
- Feeding the Fire – Professional Development and Digital Bridge Academy Faculty Training (2007)
- School should be relevant to students and an effort should be made to relate course material to students’ lives
- Personal reflection and self-awareness is valuable
- Student learning communities are a powerful tool for helping students learn and stay in school
- The classroom should be safe and fun
- Authentic communication in which instructors are honest and self-reflective is useful for connecting with students and building their trust
- Expectations should be clear
In this study, CJTC researchers looked at further effects of faculty participation, and followed faculty from the 2006 study to see effects on teaching, including teaching in non-DBA classes. In interviews, teachers reported—and gave examples of–introducing a number of principles to their classrooms:
Faculty also reported improved interactions with students, colleagues, and in their personal lives.
Read the full reports:
Read the full report:
Becoming an ACE Teacher by Rose Asera, Ph.D. (2011)
__________________________________________ Top of Page
The Center for Justice, Tolerance and Community at University of California Santa Cruz conducted two studies that examined faculty participation in and effects of Digital Bridge Academy.
Based on observations, questionnaires, individual interviews and focus groups, the study concluded that faculty besides the program director could successfully teach the Foundation Course and that it is relevant for students of many ethnicities, genders and ages. The faculty who were trained to teach the Foundation Course found it to be a transformational experience both professionally and personally and all were interested in participating in an ongoing community of practice.
__________________________________________ Top of Page