Analysis of 2005 Academic Profile Test of General Education
April 18, 2005
by Pat Bryson, Administrative Assistant, Office of Institutional Research
Summary:
Each spring Adams State College requires all sophomores (students with 30 to 59 cum hours) to take the Academic Profile Examination (AP Exam). This exam tests students’ knowledge of general education subject matter, including reading, writing, mathematics, and critical thinking.
Main findings:
Total mean scores – ASC vs. comparison institutions: Adams State College sophomores did slightly better on the examination compared to students from 96 comprehensive peer institutions. However, ASC means are not significantly different from peer means. The 2005 ASC mean score of 448.59 puts our students at the 71st percentile in relation to students who took the exam that are enrolled in institutions in our category. As shown below, Adams State College has improved the percentile ranking by over 10 points since 2001.
Comparisons for total scores: (Range 400-500)
Total Scores |
|||
Year |
ASC Mean |
Peer Mean |
ASC % |
2005 |
448.59 |
443.6 |
71% |
2004 |
447.52 |
443.6 |
67% |
2003 |
447.42 |
442.4 |
62 % |
2002 |
448.44 |
444.1 |
59 % |
2001 |
447.28 |
444.2 |
57 % |
General Education Effectiveness:
The total score does not increase with an increase of general education courses completed until the student had completed all of the General Education courses required for graduation.
Self reported amount of |
0 |
About 25% |
About 50% |
About 75% |
About 100% |
Number of Respondents |
1 |
8 |
46 |
137 |
24 |
Mean Scores |
445 |
446.88 |
446.13 |
446.88 |
461.38 |
Sub-scores: The following table shows the mean scores by area for ASC students and for sophomores from the comparison institutions in 2005. In all subject areas, the mean scores of ASC students were slightly above the means of the comparison group students. Adams State stands out above the peers’ percentile rankings in Critical Thinking and Natural Sciences. There is not a significant change in scores in any area for the four years of data.
Comparisons for Sub-Scores (Range: 100- 130)
Mean Subscores |
||||||
Area |
2001 ASC |
*2003 ASC |
2004 ASC |
2005 ASC |
2005 Peer |
2005 ASC% |
Humanities |
115.37 |
115.84 |
115.58 |
115.89 |
114.5 |
59% |
Social Sciences |
114.42 |
114.04 |
114.09 |
114.29 |
113.2 |
64% |
Natural Sciences |
116.83 |
116.73 |
117.08 |
117.47 |
114.78 |
91% |
Reading |
119.37 |
119.27 |
119.33 |
119.57 |
118.26 |
56% |
Writing |
115.06 |
114.92 |
115.10 |
115.62 |
114.4 |
63% |
Mathematics |
112.16 |
114.84 |
114.65 |
114.54 |
113.55 |
57% |
Critical Thinking |
114.33 |
112.64 |
112.61 |
113.00 |
110.69 |
91% |
*2002 test was short form and no sub-scores are available
Student Proficiencies:
The criterion-referenced portion of the AP examination measures student proficiency levels in reading, writing, mathematics and critical thinking. Students who score less than level one, the lowest level, have yet to achieve college-level proficiency. Proficiency levels are hierarchical: students who achieve level 3 have also achieved levels 1 and 2.
From 2004 to 2005 the percent NOT PROFICIENT at Level 1 has gone down in reading. The critical thinking component shows some improvement in PROFICIENT at Level 1. Similarly, math, reading, and writing all show decreased proficiency at this level.
Percent of Students Proficient at Level 1 |
|||||||
Proficiency Area |
2004 % Not Proficient |
2005 % Not Proficient |
2004 % Marginal |
2005 % Marginal |
2004 % Proficient |
2005 % Proficient |
|
Reading ASC |
10 |
8 |
15 |
21 |
75 |
71 |
|
Comparison Group |
14 |
14 |
20 |
20 |
65 |
66 |
|
Writing ASC |
1 |
5 |
13 |
12 |
85 |
83 |
|
Comparison Group |
8 |
9 |
23 |
23 |
69 |
68 |
|
Mathematics ASC |
9 |
12 |
19 |
20 |
72 |
68 |
|
Comparison Group |
17 |
16 |
28 |
28 |
55 |
56 |
|
Critical Thinking ASC |
79 |
80 |
16 |
13 |
5 |
7 |
|
Comparison Group |
84 |
83 |
12 |
13 |
4 |
4 |
|
Of the 55 not proficient in math, 16 were enrolled in remedial math classes in the semester that the test was administered.
Although the overall scores indicate performance at or above previous levels, the following data would suggest that some students who do require remediation have deficiencies in more than area.
Number of students who are not proficient in Reading, Writing or Math.
| One area | Two areas | All three areas | |
| 2005 | 44 | 23 | 15 |
| 2004 | 92 | 20 | 10 |
Response Rate: The 2005 response rate was 67.3%. Over the first 5 years that the test has been administered, the response rate improved each year. The decline in the 2005 response rate may be due to removing the requirement that those students who did not take the test enter mandatory remediation classes. In February, 2005, the Office of Institutional Research and Assessment sent 344 letters inviting all sophomores to take the 2005 AP Exam. A total of 223 students took the test. One hundred and thirty-six students received a letter and did not take the test and 15 students did not receive the letter and took the test anyway. Two were excused, 8 had already taken the test, and three of the letters were returned.
Year Tested |
N tested |
Response Rate |
2005 |
223 |
67.3% |
2004 |
268 |
87.3% |
2003 |
323 |
87% |
2002 |
268 |
81.2% |
2001 |
282 |
79%. |
2000 |
143 |
40% |
Status:
Two hundred eleven of the students tested listed themselves as full time. Our cross check shows that 210 were registered for 12 hours or more during that term indicating that they have a good sense of their status. 124 have pre-registered for fall term. Eight reportedly speak better in another language than English. Most takers were between the ages of 20 and 29, (61%) with 29% under 20 years old. According to Banner records, 6 takers were freshmen, 205 sophomores, 8 juniors, and 4 senior in that term.
Students who work 16-30 hours, performed better than students who are not employed.
Hours Worked |
0 |
1-15 |
16-30 |
>30 |
ASC Percent of students |
24% |
44% |
19% |
13% |
ASC mean scores |
444.26 |
450.39 |
451.29 |
446.33 |
In all areas, males and females were fairly equal. Mean total scores for males were 449.45 and for females 448.17.
Student Major:
Students who major in Art and English scored the highest and those who major in HGP and foreign Languages scored the lowest. Predictably, the students in the Associates programs scored lower due to ASC policy of assigning the Associates major to the conditionally enrolled students.
Mean Total Score by Major
Major |
N |
Total Score |
Art |
3 |
468.66 |
English |
4 |
466.25 |
Biology |
26 |
457.07 |
Math |
14 |
452.78 |
Music |
7 |
452.14 |
Nursing |
2 |
452.00 |
Chemistry |
5 |
452.00 |
Undecided |
17 |
449.52 |
Business |
38 |
448.86 |
Teachers Ed |
26 |
446.30 |
Sociology |
8 |
445.25 |
Psych |
12 |
443.16 |
EPLS |
19 |
443.00 |
SPT/Theater |
2 |
441.00 |
HGP |
9 |
440.77 |
For Lang |
1 |
430.00 |
Associates |
6 |
421.16 |
Other notes:
One hundred eleven students qualified for the small scholarship with a total of $7352 awarded.
If this exam is to show program changes and efficacy, one would expect to see significant increases in scores over time. Recommendation for the assessment committee and the institution is to use this analysis to determine strengths and weakness of programs as well as evaluation of individual student performance.