English & ESL
CCSF Course Placement Assessment
The CCSF English and ESL Departments have reviewed many published, “off-the-shelf” placement tests for possible use in the ENGL and ESL course placement processes. But no second-party tests, as yet, possess the content to adequately measure the full range of English and ESL skills of entering CCSF students, and the full range of content/skill areas taught in the CCSF English and ESL curricula. For this reason, both departments have developed placement tests that that more effectively measure the specific knowledge and skills expected of entering CCC students at the various course placement levels. We’ve been fortunate to have the resources to develop and manage these CCSF placement tests – the faculty expertise, Office of Research assistance, and Matriculation Office coordination and funding.
Studies of test-validity and fairness
One of the Matriculation Services Program outcomes states, “Students will be able to place into the appropriate level of English, ESL, or Math courses to maximize their success.”
The ongoing evaluation of the validity and fairness of college course placement processes is mandated by the California Community College Chancellor’s Office.
In April and November 2012, the College completed CCCCO-mandated test validation studies for three locally-developed placement instruments to obtain renewal of approval for their use in CCSF course placement processes. The instruments included the credit English Placement Test, the credit ESL Writing Sample Test, and the Noncredit ESL Placement Test. The purpose of the studies was to examine whether CCSF’s placement tests generate valid, reliable placement recommendations, free of cultural or linguistic biases, and to recommend any needed revisions based on the findings.
The Assessment and Prerequisite Coordinator, Office of Matriculation and Assessment, coordinated the following studies, which required the input of students, faculty, individuals representing the diversity of test-takers, and consultation of the College Researcher:
- Content Validity -- Item by item analysis of test content, performed by faculty who teach the range of courses in the English and ESL (credit and noncredit) sequences.
- Consequential (Cut-Score) Validity – Classroom surveys of students and faculty at all levels of ENGL and ESL placement to monitor for appropriateness of placement recommendations.
- Reliability – Data-based evaluation of reliability of student test scores of the CCSF English Placement Test and CCSF Noncredit ESL Placement Test; student data-based evaluation of inter-prompt reliability of the CCSF ESL Writing Sample Test, and inter-rater reliability of the Writing Sample scoring process.
- Bias – Evaluation of test-items, reading passages, and writing prompts for possible cultural or linguistic bias, performed by representatives of various ethnic, gender, and age groups.
- Disproportionate Impact (of test results) – Student data-based evaluation of any disproportionate rates of placement into the various course levels by ethnicity, gender, and age.
Findings and Resulting Actions/Improvements
The results of studies of content validity, reliability, bias, and disproportionate impact for all three placement tests met the standards outlined in the CCCCO assessment standards.
Discussions among the English Department, Office of Matriculation & Assessment, and the College Researcher regarding the findings of the English Placement Test cut-score validity study (based on student and faculty satisfaction survey data) resulted in the lowering of cut-scores for all but one of the course placement levels. The English Department, Research and Planning Office, and Matriculation Office will conduct a study of Fall 2012 student data to examine the effects of the cut-score changes on student success in English courses. The CCSF English Placement Test has received CCCCO approval for continued use through July 2018.
Discussions between the ESL Department and Office of Matriculation & Assessment regarding findings of the Inter-prompt reliability study of the ESL Writing Sample Test have resulted in the discontinuance of one of the WS prompts, which did not generate the same range of writing as did the other five prompts. In Spring 2013, follow-up surveys of students placed into ESL 120 and 130 will be conducted to double-check on appropriateness of cut-scores, as their rates of satisfaction with placement were at the threshold of the CCCCO standard. The CCSF ESL Writing Sample Test has received CCCCO approval for continued use through January 2019.
Follow-up satisfaction surveys of noncredit ESL Level 5 and 6 faculty will be conducted in Spring 2013, as their rates of satisfaction fell below the CCCCO standard. The findings will be discussed by the ESL Department and the Matriculation Office, with possible revision of the cut-scores and /or scoring rule if indicated.
It’s important to mention that placement of students into California Community College English, ESL, and Math courses is a “multiple measures” process. That is, while the test score is a primary measure in the course placement process at the majority of CCCs, other indicators of a student’s likelihood to succeed must also be factored into the final course placement recommendation. In recent semesters, the basic skills departments and the Matriculation Office have placed greater scrutiny on relevance and consistent use of these additional indicators of student readiness for basic skills courses. At the end of Spring 2012, the Matriculation Office convened a collaborative workgroup of counselors and basic skills faculty to identify relevant “multiple measures” that have been found to be useful indicators of course readiness. The workgroup developed guidelines for multiple-measures assessment and updated procedures for documenting changes to student course placement.
As part of the Bridge to Success Partnership grant, San Francisco Unified School District (SFUSD), the CCSF Math Department, and the Office of Matriculation implemented a new math placement pilot project. Using this enhanced placement process, graduating seniors enrolling at the College in Fall 2012 had the opportunity to raise their test placement by meeting two of the following criteria: high-school Math GPA of 2.7 or higher; high school attendance rate of 90 percent or greater; a score on the CST Math test of “Basic” or higher. For Fall 2012 enrollment, out of 1,400 applicants to the College, SFUSD identified 648 graduating seniors who met the aforementioned criteria. As a result, 276 first-semester CCSF students who had initially placed below college level math on the CCSF placement test during their last year of high school received a “bump” in their CCSF math placement.
This Spring 2013, the English Department has been developing criteria to supplement the current placement testing process. In addition, the department has received research assistance from the Bridge to Success Partnership to identify SFUSD high-school data that may serve as indicators of student preparation in an enhanced multiple measures placement process, similar to the Math placement “bump up” process.
 CCCCO suspended this required test-validation process in March 2009 and reinstated it in April 2012. Methodology and standards are provided in the guideline document: “Standards, Policies and Procedures for the Evaluation of Assessment Instruments Used in the California community Colleges.”