The longitudinal, cohort studies link the multiple layers of assessment activities

The assessment system touches almost every aspect of the university--academic affairs, student affairs, alumni affairs, academic and career advising, accreditation of the university and programs, internal academic program review, special programs, and grant-funded research projects. By incorporating all the data from multiple assessment activities into longitudinal, cohort datasets, the Office of Assessment and Planning is able to provide an integrated map of student learning across campus. The longitudinal, cohort datasets are an asset to the university because when taskforces, committees, and grant-writing teams need critical information, there is no need to scramble to find relevant data to mine.

Every year, an assessment plan is developed to ensure that there are multiple measures of the competencies promoted by the Liberal Education Program and graduate school. The campus-level assessment plan reflects the university’s assessment philosophy. Data that are collected for longitudinal, cohort studies are used to empirically evaluate the model in which students’ demonstration of the competencies that employers desire in new hires is a function of their developmental trajectories, ability to work autonomously and to handle cognitive complexity, learning in the Liberal Education Program’s areas of knowledge, and an orientation to the future that informs goal setting and taking actions in the here-and-now to achieve desired futures. Most of all, it is all about relationships.

New surveys are driven by questions that the university community has about the students and how well the university functions to promote their learning and development. Surveys are developed by teams with representatives from different offices across campus, including faculty, non-instructional staff responsible for student support services, student affairs professionals, and the university leadership. The cores of the institutional surveys tend to be written by students in research methodology courses. Once drafts of surveys are completed, they are shared widely through the university community in order to elicit feedback and additional items. For example, the draft of the Transfer Student Survey was improved by the recommendations of the committee on transfer students, the enrollment management team, student affairs, the academic transfer student office, academic advisement, and university leadership. Presentations of results in a wide variety of meetings on campus are opportunities for faculty, staff, and administrators to review the assessment plan, launch taskforces to further investigate pressing concerns, and guide future assessment activities. For example, the idea for the Transfer Student Survey emerged at a meeting of the Transfer Student Taskforce comprising representatives from student support offices on campus, faculty, and administrators.

The surveys and rubrics are part of the overall assessment plan to determine whether students’ progress along the university learner outcomes meet or exceed expectations. The underlying approach of the assessment plan is to assess student learner outcomes through multiple measures. Consider, for example, critical thinking. The critical thinking papers of sophomores and juniors are scored by faculty on Reading Day using the rubric developed by the Critical Thinking affinity group. In tandem, the critical thinking papers of first-year students and seniors are scored by faculty in other states as part of the Multi-State Collaborative to Advance Quality Student Learning (MSC). So, too, the Collegiate Learning Assessment Plus (CLA⁺) is administered to measure the level of critical thinking among first-year students and seniors. In 2016, seniors who had taken CLA⁺ as first-year students completed CLA⁺ once again in order to determine the extent of growth over time.

In addition to multiple measures of student learning outcomes, the university’s comprehensive assessment plan incorporates common assessment activities designed to provide evidence of student learning across campus. For example, the core of the annual alumni survey is common for all programs. A program adds its own unique learning outcomes to the survey, which only alumni for that specific program see through the use of “skip patterns” built into the survey. The academic program review student survey, a hardcopy survey, also has a core that all students complete; An insert is added to the survey with a program’s unique learner outcomes. All students in the School of Education’s certification programs are assessed on common unit assessments. Students throughout the university are assessed on such common assessment instruments as the CLA⁺ and the MSC rubrics for quantitative literacy, critical thinking, and written communication. Reports are prepared for individual programs that present both the scores on the program-specific learning outcomes and the university’s common learning outcomes. Reporting both underscores the alignment between program-specific and university assessment plans. As the cores of the assessment instruments are the same regardless of program, the university merges the data from the core sections of the instruments in order to provide evidence of how well the university as a whole functions in promoting students’ learning and development.

The Office of Assessment and Planning conducts longitudinal, cohort studies in order to alleviate the conditions that lead to student withdrawal and strengthen the conditions that promote students' learning and development. The students are followed from New Student Orientation through graduation from the university, or subsequent enrollment in other colleges and universities. As each incoming class enters the university, a cohort dataset is established. A cohort dataset initially contains such demographic information from Banner as high school rank, high school GPA, SAT scores, gender, ethnicity, residential status, registration with Veterans Services and the Disability Resource Center, and English and Math placements. Each semester, new data are added, including earned credits, cumulative GPA, registration status, and scores on surveys and direct assessments. The cohort datasets incorporate results from all the assessment activities on campus.

Surveys and assessments incorporated into the longitudinal, cohort studies include the New Student Orientation Survey, Beginning College Survey of Student Engagement (BCSSE), the First-Year Experience Self-Assessment (a locally-developed instrument), the National Survey of Student Engagement (NSSE), the Southern Experience Survey (a locally-developed continuing student survey), and the Collegiate Learning Assessment Plus (CLA⁺), a performance-based assessment which is administered to freshmen and seniors. Also added to the datasets are students' scores on the papers submitted to the Multi-State Collaborative to Advance Quality Student Learning (MSC), a collaboration among higher education institutions, the Association of American Colleges and Universities (AAC&U), and the State Higher Education Executive Officers (SHEEO), which is the national association of the chief executives of statewide governing, policy, and coordinating boards of postsecondary education.

Competencies measured through assessments are measured multiple times. For example, students' competency in Critical Thinking is measured by the Collegiate Learning Assessment Plus (CLA⁺), the Multi-State Collaborative to Advance Quality Student Learning, and assessments developed by interdisciplinary "affinity" groups. The affinity groups are responsible for the process of understanding what and how students are learning; They developed common, shared rubrics to measure the competencies embedded within the Liberal Education Program ("general education"), including an assessment of students' level of Critical Thinking. So, too, students' attainment of Quantitative Reasoning is measured multiple times. For the first-year students and seniors, this competency is measured by the Multi-State Collaborative and the Collegiate Learning Assessment. The Math department decided to add four additional questions on final examinations in order to assess students' progress in meeting four outcomes related to Quantitative Reasoning. Quantitative Reasoning is also measured through the scoring of juniors' final papers by SCSU faculty other than the course professors using the AAC&U's Quantitative Literacy rubric. 

Another source of data for longitudinal studies is the university's online assessment system. The university's online assessment system, Tk20, is used by all the certification programs and similar on campus, including all the programs in the School of Education, Social Work, Marriage & Family Therapy, and Communication Disorders. The students' scores on assessments are disaggregated by degree level (initial/advanced certification, undergraduate/graduate, sixth year, doctoral), location (some Educational Leadership and Special Education sections are offered in local schools), modality (online, hybrid, traditional), gender, and ethnicity. Students in certification programs are assessed on an ongoing basis to ensure that they are on track to meeting professional standards and certification requirements. For example, an interdisciplinary Educator Preparation Program Board developed common unit assessments that are relevant to all education certification programs. Assessments address candidates' knowledge, skills, dispositions, and preparedness for teaching (or the equivalent). These assessments include the Professional Dispositions Assessment; Clinical Field Experience Evaluation; Student Teaching Evaluation (cooperating teachers), Student Teaching Observation (university supervisors), and Impact on P-12 Student Learning Assessment. The implementation of an online assessment system (Tk20), a centralized data collection system, enables monitoring of candidates' progress. Students view the scores that they received on assessments via their Tk20 accounts. Since Tk20 is administered by the Office of Assessment and Planning, all the assessment activities of the university are coordinated in one coherent system.