Assessment How-to

Develop Program Outcomes

1. What are program learning outcomes?
2. Why develop and publish program learning outcomes?
3. Characteristics of program learning outcomes
4. Developing program learning outcomes
5. Examples of program learning outcomes
6. Good practices
7. Worksheets/Checklists/Resources

1. What are program learning outcomes?

Program learning outcomes are clear, concise statements that describe how students can demonstrate their mastery of program goals (Allen, M., 2008). These statements identify the knowledge, skills, or attitudes that students will be able to demonstrate, represent, or produce upon successful coompletion of the program.

2. Why develop and publish program learning outcomes?

Learning outcomes:

Help students learn more effectively
Make clear what students should expect from their educational experience

Encourage students to be intentional learners who direct and monitor their own learning

Help faculty design courses, curriculum, and programs
Make graduates’ skills and knowledge clear to employers, accrediting agencies, etc.

Questions which learning outcomes address include the following:

What knowledge, skills, abilities, and values should the ideal student graduating from our program demonstrate?
How will they be able to demonstrate these capabilities?
How well does our program prepare students for careers, graduate school, professional study, and/or lifelong learning?
What assessment can we use to demonstrate growth in students’ knowledge, skills, abilities, and values as they progress through our program?

Characteristics of program learning outcomes:

Describe what students learn, rather than what faculty will do or “cover”
Framed in terms of the program and not individual courses
Observable or measurable, and important
Alignment:

Program outcomes align with school/college goals and institutional goals
Course outcomes align with program outcomes
Map to co-curriculum and educational practices

Rely on verbs that specify definite, observable behaviors
Focus on the central abilities of the discipline. Incorporate or adapt professional organizations' outcome statements when they exist.
Stated such that evidence related to the outcome can be gathered by more than one data-collection method
Collaboratively authored and collectively accepted
3-6 outcomes are ideal

Good outcomes use verbs that describe definite, observable actions
Bloom’s taxonomy is a well-known description of levels of educational objectives. It may be useful to consider this taxonomy when defining your outcomes.

Bloom’s Taxonomy

Level	Cognitive Behaviors
1. Knowledge	To know specific facts, terms, concepts, principles, or theories
2. Comprehension	To understand, interpret, compare and contrast, explain
3. Application	To apply knowledge to new situations, to solve problems
4. Analysis	To identify the organizational structure of something; to identify parts, relationships, and organizing principles
5. Synthesis	To create something, to integrate ideas into a solution, to propose an action plan, to formulate a new classification scheme
6. Evaluation	To judge the quality of something based on its adequacy, value, logic, or use

Verb Power
Concrete verbs such as “define,” “identify,” or “create” are more helpful for assessment than verbs such as “know,” “understand,” or passive verbs such as “be exposed to.” Some examples of verbs frequently used in outcomes are included in the table below.

Knowledge	Comprehension	Application	Analysis	Synthesis	Evaluation
cite define describe identify indicate label list match memorize name outline recall recognize record relate repeat reproduce select state underline	arrange classify convert describe defend diagram discuss distinguish estimate explain extend generalize give examples infer locate outline paraphrase predict report restate review suggest summarize translate	apply change compute construct demonstrate discover dramatize employ illustrate interpret investigate manipulate modify operate organize practice predict prepare produce schedule shop sketch solve translate use	analyze appraise break down calculate categorize compare contrast criticize debate determine diagram differentiate discriminate distinguish examine experiment identify illustrate infer inspect inventory outline question relate select solve test	arrange assemble categorize collect combine compile compose construct create design devise explain formulate generate manage modify organize perform plan prepare produce propose rearrange reconstruct relate reorganize revise	appraise assess choose compare conclude contrast criticize decide discriminate estimate evaluate explain grade judge justify interpret measure rate relate revise score select summarize support value

[From: Gronlund, N. E. (1991). How to write and use instructional objectives (4th ed.). New York: Macmillan Publishing Co.]

4. Developing program learning outcomes

Before developing program outcomes, it might be helpful to consider these questions, which focus on outcomes in slightly different ways:

For each of the stated program goals, what are the specific knowledge, skills, or attitudes that would tell you this goal is being achieved?
What would a skeptic need (evidence, behavior, etc.) in order to see that your students are achieving the major goals you have set out for them?
In your experience, what evidence tells you when students have met these goals – how do you know when they’re “getting” it?

Learning outcome statements may be broken down into 3 main components:

A verb that identifies the performance to be demonstrated
A learning statement that specifies what learning will be demonstrated in the performance
A broad statement of the criterion or standard for acceptable performance

For example:

Verb
(performance)

Learning Statement
(the learning)

Criterion
(the conditions of the performance demonstration)

produces

documents

using a word processing software

analyzes

global and environmental factors

in terms of their effects on people

Tips: Effective program outcomes are widely accepted and supported by faculty members. Developing appropriate and useful outcomes is an iterative process; it’s not unusual to revisit and refine outcome statements. In most cases, it is only when you try to develop ways of assessing program outcomes that the need for refining them more precisely becomes apparent.

5. Examples of Program Learning Outcomes

Natural Sciences

Students can apply scientific methodology to research proposals.
Students can evaluate the validity and limitations of theories and scientific claims in experimental results.
Students can assess the relevance and application of science in everyday life.

Psychology

Graduates can write research papers in APA (American Psychological Association) style.
Graduates can analyze experimental results and draw reasonable conclusions from them.
Graduates can recognize and articulate the foundational assumptions, central ideas, and dominant criticisms of the psychoanalytic, behaviorist, humanistic, and cognitive approaches to psychology.

History

Students can list major events in American history.
Students can describe major events and trends in American history.
Students can apply their knowledge of American history to examine contemporary American issues.

6. Good practices

Publicize program outcomes in the catalog, on web, on syllabi, annual report, brochures, etc.
Use program outcomes to guide course and curriculum planning so students experience a cohesive curriculum
Use program outcomes to shape assessment efforts and faculty/staff conversations surrounding student learning
Collaboratively develop program outcomes; discuss and collectively accept program outcomes

7. Worksheet/Checklist/Resources

Use this worksheet and checklist to record and evaluate your outcomes (Word document).

Handouts from Student Learning Outcomes workshops: see Workshops/Events.

Sources consulted:
Allen, M. (2008). Assessment workshop at UH Mānoa on May 13-24, 2008. [Available at the Assessment Office]
How to Write Program Objectives/Outcomes. [PDF] University of Connecticut assessment web site
Program Assessment Handbook: Guidelines for Planning and Implementing Quality Enhancing Efforts of Program and Student Learning Outcomes. [PDF] University of Central Florida. (June 2008 edition).
Program-Based Review and Assessment: Tools and Techniques for Program Improvement. [PDF] Office of Academic Planning and Assessment. University of Massachusetts Amherst. (2001).
Tools & Techniques for Program Improvement: Handbook for Program Review & Assessment of Student Learning. [PDF] Office of Institutional Assessment, Research, and Testing. Western Washington University. (2006).

updated 1/28/2013

Verb (performance)	Learning Statement (the learning)	Criterion (the conditions of the performance demonstration)
produces	documents	using a word processing software
analyzes	global and environmental factors	in terms of their effects on people