Getting Assessment Completed and Reported

• Getting assessment completed and reported
• Completing the annual report
• Some caveats and advice: Problems with collecting and analyzing information
• Some caveats and advice: Problems with documenting assessment

Getting Assessment Completed and Reported

So I'm assigned to work on assessment for my department. What's needed?

Every program at UND is expected to have an assessment plan. This means every major and every certificate program at both the graduate and the undergraduate levels should have a plan. If a department offers multiple degrees, it would make sense that each degree program has some difference in intended learning outcomes, however similar the programs may be. (Why offer two different master's degree programs if students are expected to achieve identical outcomes no matter which degree is chosen?) On the other hand, it also makes sense that there will be a lot of overlap in learning outcomes because the curriculum for one major likely has a great deal in common with the curriculum for the other. So an engineering department, for example, should have a plan for the Master of Science degree and a plan for the Master of Engineering degree. But there may be only a single goal that distinguishes one program from the other, with the MS providing a greater focus on research skills and the ME a focus on practice. That small difference will be reflected in the two similar, yet not identical, assessment plans.

Every program at UND is expected to have faculty collecting data about learning every year. That doesn't mean every faculty member, every learning outcome, and every method in every year. But make sure that some assessment activity occurs annually, as described in your program's assessment plan.

Every program should offer opportunities for faculty to talk about and share assessment results and findings (i.e., "what do you think this bit of information means?") every year. There is great merit in having this discussion during a retreat – where there's time for meaningful discussion. One department which does this well has a standard format which provides a good model:

1. Collect information all year.
2. At an annual retreat, begin by reviewing all assessment information collected during the year, possibly including looking together at actual student work products which were the basis of some of the assessment results and findings that you'll be discussing. Talk about what the collected documents show and what the information means.
3. Review notes from last year's retreat. What information did you see there? What actions, if any, did you agree would be taken this year? How have those things worked out? If plans weren't followed, why was that?
4. Spend the last part of the retreat planning for next year. Based on the assessment findings and on your discussion of actions taken over the past year (and drawing in whatever additional information might be relevant), what should be on your department's "to-do" list for the next year? You might want to talk about individual courses ("can some faculty include a bit of introduction to presentations in 200-level classes so the expectations in the capstone are not so unfamiliar?"), curriculum ("should course offerings, requirements, electives, content be tweaked?"), and/or the overall assessment process ("are we finding that assessment activities provide useful information, or should we be looking for better ways to answer our questions about learning?").

Every program should have someone documenting the assessment activities completed each year. One convenient place to create a (permanent) paper trail of documentation is in the annual reports. If you have your own program accreditor, you may want to document in another way that meets your accreditor's expectations. If, even without a program accreditor, you have developed an effective and well-honed practice of documenting internally, you can do that. In any case, someone should file an assessment report annually for every program. That report should include

• reviewing your posted assessment plan,
• indicating assessment methods used during the last year,
• providing a sample of assessment results and conclusions from those results, and
• describing any loop-closing activities that occurred during the past year, either in response to your new assessment findings or as a result of previous assessment work.

top

Completing the annual report

Instructions for completing the annual report are available at the annual report website. You can also find plans for assessing the various programs offered at UND on the Assessment Committee website. 

Some caveats and advice: Problems with collecting and analyzing information

Too many factoids, no analysis: It may seem counterintuitive to say this on an assessment website, but burying yourself in data may not be a good thing for assessing student learning. You want to collect enough information to gain a systematic (researched) understanding of student learning, but you don't want so many pieces of information that it's impossible to manage the paper flow or find time to analyze what's been collected. The aim isn't to have the most data. It's to have information that reveals patterns and trends in learning that will help faculty in your program make good decisions about any changes that might be contemplated or needed.

No one wants to talk about the data: If you aren't generating information that's interesting, the truth probably is that no one does want to talk about it. It feels like a waste of time. It may be time to regroup and focus on learning outcomes that everyone agrees are critical, methods for collecting information likely to yield intrinsically interesting findings, and questions about learning that program faculty are genuinely curious about. It can be hard to see how to do this, so one approach is to get an outside perspective. You can find people willing to offer advice from their own experiences by checking the list of assessment consultants on UND's assessment website. These are your own campus colleagues who've been involved in successful (and useful) assessment projects. Or get in touch with Dr. Joan Hawthorne, Director of Assessment and Regional Accreditation. 

Faculty in my department see assessment as busywork: This problem becomes self-perpetuating. If colleagues assume assessment is busywork, they may do whatever is easiest rather than what is most likely to be useful. The best way to change that attitude is with a bit of success. Try creating an assessment project that will yield genuinely interesting findings. Engage your colleagues in discussion of your results. And then build on that first small success.

top

Some caveats and advice: Problems with documenting assessment

Difficulties with organization: The more information you collect, the greater your organizational challenge. And there is no single strategy for success.

The best advice is what you already know. It makes sense to keep all materials in two places, one for all hard copies and one for all electronic documents. Make sure that minutes are kept at every meeting where assessment is discussed, because a lot of the results, findings, and loop-closing information will be elicited from faculty discussions. If it's not written down, you will not remember when it's time to report. Add those minutes to the assessment file as soon as they're written up. Assign one individual in the department (ideally someone with good organizational skills) to be in charge of maintaining the records, even though you will want to be sharing responsibility for other aspects of the work. At the beginning of the semester, set up a schedule for assessment reminders, and choose someone to be in charge of them. Find a colleague in another department where they've been doing assessment well for a long time and ask how they do it (and our list of assessment consultants is a good place to start). Beg, borrow, or steal the best methods you find.

Every group wants things reported in a different way: You will have no choice about documenting in ways that are acceptable to your program accreditor, if you have one. UND is not prescriptive about how you document and you'll note that our annual report has a place which allows you to paste in an existing assessment report rather than rewrite using our template. But please ensure that you've provided the same kinds of information as what's requested in our template – or add in the additional information as a supplement to your existing report. Annual Report Assessment Questions

If you have a very long assessment report (some departments have assessment reports for accreditors that run 50 or even 100 pages long, with all the appendices included), it isn't necessary to paste the full document into the annual report. You can probably cut some of the detail (e.g., lengthy charts and appendices) and include a portion of what you've already written rather than the entire document. Yes, members of UND's Assessment Committee will wade through 100 pages of text, if you upload it, but you'll probably receive more useful feedback if you submit information that is directly linked to the questions in UND's assessment template rather than dumping in a very long report with lots of information that committee members do not need to review. (If you were assigned to the committee, you'd probably appreciate the more reasonable document length too!)

Our institutional accreditor (the Higher Learning Commission), like UND, does not specify reporting format. They do expect to see reports that show what you've done, what you learned, and how you used that information. If they need additional detail, they will ask for it.

top