Appendices
Appendix A
Alignment of the University’s Civic and Community Engagement Student Learning Outcomes with the AAC&U Civic Engagement VALUE Rubric
The University of Virginia’s undergraduate civic and community engagement student learning outcomes align closely with the AAC&U Civic Engagement VALUE Rubric criteria used for institution-wide assessment, as shown in Table A.1.
University of Virginia Civic and Community Engagement Student Learning Outcomes |
AAC&U Civic Engagement VALUE Rubric Criterion(s) |
|---|---|
|
1. Display openness and curiosity about others. |
Diversity of Communities and Cultures |
|
2. Display empathy and humility when working with different people and communities. |
Diversity of Communities and Cultures |
|
3. Seek, analyze, and evaluate different points of view about civic issues. |
Analysis of Knowledge; Civic Communication |
|
4. Connect and apply knowledge from academic study to civic engagement experience. |
Analysis of Knowledge; Civic Action and Reflection |
|
5. Work constructively with community partners to address important civic issues. |
Civic Action and Reflection; Civic Contexts or Structures |
|
6. Articulate and clarify their commitment to citizenship or social responsibility. |
Civic Identity and Commitment |
Appendix B
Inter-Rater Reliability
Each artifact was independently scored on the AAC&U Civic Engagement VALUE Rubric by two AAC&U raters. Table B.1 summarizes inter-rater reliability across the six rubric criteria, including Weighted Gwet’s Agreement Coefficient, percent exact agreement, and percent agreement within one category. Weighted Gwet’s Agreement Coefficient estimates agreement between raters while accounting for agreement that could occur by chance; values closer to 1 indicate stronger agreement. Across the six criteria, agreement coefficients ranged from 0.65 to 0.72, exact agreement ranged from 31% to 41%, and agreement within one category ranged from 79% to 86%
Criterion |
Weighted Gwet's AC |
Percent Exact Agreement |
Percent Agreement within 1 Category |
|---|---|---|---|
|
Diversity of Communities and Cultures |
0.65 |
31% |
79% |
|
Analysis of Knowledge |
0.65 |
33% |
79% |
|
Civic Identity and Commitment |
.72 |
41% |
86% |
|
Civic Communication |
.70 |
33% |
82% |
|
Civic Action and Reflection |
.69 |
37% |
80% |
|
Civic Contexts or Structures |
.70 |
34% |
86% |
Appendix C
Course-Based Performance Assessment Results by Credit Hours Earned
Additional details are presented on course-based performance assessment results disaggregated by students’ accumulated credits earned. Table C.1 presents results for each rubric criterion, showing the percentage of scored artifacts at each performance level and the total number of artifacts scored within each credit-hour group.
Diversity of Communities and Cultures |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Fewer than 60 Credits |
12% |
34% |
49% |
5% |
41 |
|
60-89 Credits |
11% |
46% |
35% |
8% |
85 |
|
90 or More Credits |
5% |
48% |
39% |
8% |
75 |
Analysis of Knowledge |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Fewer than 60 Credits |
15% |
34% |
41% |
10% |
41 |
|
60-89 Credits |
9% |
51% |
35% |
5% |
86 |
|
90 or More Credits |
5% |
40% |
47% |
8% |
78 |
Civic Identity and Commitment |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Fewer than 60 Credits |
13% |
42% |
39% |
5% |
38 |
|
60-89 Credits |
5% |
44% |
46% |
5% |
80 |
|
90 or More Credits |
5% |
20% |
59% |
15% |
74 |
Civic Communication |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Fewer than 60 Credits |
15% |
29% |
51% |
5% |
41 |
|
60-89 Credits |
3% |
45% |
42% |
10% |
93 |
|
90 or More Credits |
4% |
28% |
49% |
20% |
82 |
Civic Action and Reflection |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Fewer than 60 Credits |
6% |
33% |
47% |
14% |
36 |
|
60-89 Credits |
4% |
29% |
53% |
13% |
75 |
|
90 or More Credits |
0% |
22% |
65% |
13% |
63 |
Civic Contexts or Structures |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Fewer than 60 Credits |
3% |
36% |
49% |
13% |
39 |
|
60-89 Credits |
8% |
37% |
43% |
13% |
79 |
|
90 or More Credits |
3% |
29% |
56% |
12% |
68 |
Appendix D
Course-Based Performance Assessment Results by Pell Grant Eligibility
Additional details are presented on course-based performance assessment results disaggregated by Pell Grant eligibility status. Table D.1 shows that a larger share of Pell-eligible students’ work scored at milestone (2) or higher for civic action and reflection and civic contexts or structures.
Diversity of Communities and Cultures |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Eligible |
10% |
52% |
35% |
3% |
31 |
|
Not Eligible |
9% |
43% |
40% |
8% |
170 |
Analysis of Knowledge |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Eligible |
10% |
53% |
30% |
7% |
30 |
|
Not Eligible |
9% |
42% |
43% |
7% |
175 |
Civic Identity and Commitment |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Eligible |
7% |
31% |
55% |
7% |
29 |
|
Not Eligible |
7% |
35% |
49% |
9% |
163 |
Civic Communication |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Eligible |
3% |
38% |
50% |
9% |
32 |
|
Not Eligible |
6% |
35% |
46% |
13% |
184 |
Civic Action and Reflection |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Eligible |
4% |
29% |
62% |
4% |
24 |
|
Not Eligible |
3% |
27% |
55% |
15% |
150 |
Civic Contexts or Structures |
Capstone (4) |
Milestone (3) |
Milestone (2) |
Benchmark (1) |
Total |
|---|---|---|---|---|---|
|
Eligible |
4% |
41% |
56% |
0% |
27 |
|
Not Eligible |
5% |
33% |
48% |
14% |
159 |
Appendix E
Civic and Community Engagement Overall Results: Evidence vs. No Evidence
Overall Civic Engagement results from the VALUE Scoring Collaborative are distinguished by student artifacts that demonstrate learning, as opposed to those that do not. “Evidence” includes artifacts scoring at any benchmark, milestone, or capstone level (scores of 4–1) on each Civic Engagement rubric dimension, while “No Evidence” reflects artifacts receiving a score of 0, indicating an absence of demonstrated learning for that dimension. Results are reported as both counts (n) and percentages (%) for each rubric dimension in Table E.1.
Criterion |
Artifacts with Evidence, n (%) |
Artifacts with No Evidence, n (%) |
|---|---|---|
|
Diversity of Communities and Cultures |
201 (84%) |
39 (16%) |
|
Analysis of Knowledge |
205 (85%) |
35 (15%) |
|
Civic Identity and Commitment |
192 (80%) |
48 (20%) |
|
Civic Communication |
216 (90%) |
24 (10%) |
|
Civic Action and Reflection |
174 (72%) |
66 (28%) |
|
Civic Contexts or Structures |
186 (78%) |
54 (22%) |
Appendix F
Civic and Community Engagement Assessment Committee
Committee members provided subject-matter expertise and institutional context to guide assessment sampling decisions, refine methods, interpret findings, and identify meaningful uses of results for program improvement. The committee included:
Ellen Blackmon, Senior Academic Program Officer, Academic Outreach
Dreama Johnson, Director, Public Service Pathways
Corey Waters, Assessment and Survey Specialist, Institutional Research and Analytics
Kelsey Kirland, University Assessment Director, Office of Academic Compliance