Organizational Effectiveness, Planning and Assessment (OE)

The Office of Organizational Effectiveness, Planning and Assessment (OE) is responsible for planning, assessment and continuous quality improvement throughout the School. OE carries out its mission by facilitating:

Monitoring and reporting of progress on the School’s strategic plan
Data collection in the Factbook to guide benchmarking, data reporting, and decision-making
The course evaluation process to ensure continued quality improvement of teaching and learning
Data collection and annual reporting of faculty activity and productivity;
The administration, coordination, monitoring, and use of surveys and survey data conducted within the School
Assessment of the development of student learning
Assessment of degree program (PharmD, MS, PhD) effectiveness and impact

The Office of Organizational Effectiveness, Planning and Assessment (OE) manages all surveys for the School. Faculty, staff, and students conduct surveys throughout the year in an effort to collect meaningful information to guide quality improvement efforts across various aspects of the School and our programs. In addition, we contribute as a School to national surveys conducted by the American Association of Colleges of Pharmacy (AACP) and use this information as well in our quality improvement efforts.

Survey Policies and Procedures

This document presents an overview of the School’s survey policies and procedures for faculty, staff, students, and alumni. General policies and procedures outlined include requests from external institutions/ individuals; faculty, staff, and students interested in conducting a survey; survey approval, set-up and administration; IRB approval; and, survey data review, interpretation, and planning.

OE Survey Form (Powered by Qualtrics)

The OE Survey Form is used to inventory and characterize survey landscape of the School. If you wish to conduct a survey of faculty, staff, students, or alumni, please complete and submit a Survey Form to the Office of Organizational Effectiveness, Planning and Assessment (OE). Be prepared to share information about the purpose of your survey; who will respond to the survey; and, how the results data will be used.

If you or your office/division/organization has administered a survey, in accordance with School Survey Policy and Procedures, please use this form to document major survey findings and proposed actions for using survey results in planning and implementing program or organizational improvement or change. To assist OE in preparing our annual report of all School surveys, we will request that you complete and submit this form to us on an annual basis.

AACP Institutional Research Annual Surveys & Services

The AACP institutional research department gathers, analyzes and summarizes data related to the colleges and schools of pharmacy accredited by the Accreditation Council for Pharmacy Education. The UNC Eshelman School of Pharmacy uses the AACP surveys to assess trends and make decisions with regard to the School’s pharmaceutical education.

Student evaluation of course and educator effectiveness is one component of an overall course and educator assessment process. The evaluations serve to:

provide students an opportunity to share feedback on the effectiveness of the course and the educators who teach in a course;
provide educators with feedback regarding student-perceived effectiveness of the course as well as their teaching methodologies and styles;
provide the School, specifically the Professional Program Curriculum and Assessment Committee and the Graduate Education Committee, with data to consider in course and curricular quality improvement efforts; and,
provide the School’s leadership, including Division Chairs and the Executive Committee, with information regarding student-reported effectiveness and impact of faculty teaching efforts.

The Office of Organizational Effectiveness, Planning and Assessment (OE) assumes responsibility, on behalf of the School, for facilitating the course evaluation process for all required and elective course offerings in the School at both the professional and graduate levels.

Software: Explorance Blue

The School uses an online course evaluation system, Explorance Blue, to gather feedback on courses and educators. This system is contracted and used in collaboration with the University of North Carolina at Chapel Hill. Course directors, educators, and students can access course evaluations and course evaluation reports by visiting https://blueeval.unc.edu/blue/.

Information and Resources for:

Student access to the Blue Course Evaluation System:
- Directly with onyen single-sign-on (SSO) at the UNC Blue Home: https://blueeval.unc.edu/blue/
- Links provided in email invitation and reminders, including the active tasks list of open evaluations
- ConnectCarolina: Course Evaluation System link is under SelfService on the left. Blue Central Home Page will display all available evaluations.
- Sakai: Go to the “Home” Sakai page and click on the “Course Evals” option on the left-hand menu. Only available on the “Home” Sakai page, and not on individual course sites.
Students receive an email invitation on opening day with a direct link to their list of live evaluations, as well as periodic reminders until they have clicked SUBMIT to complete all evaluations.
The feedback you provide on course evaluations is valued by course directors and educators, and used to improve their courses and teaching.
The School reports aggregated metrics on response rates and overall educator and course effectiveness by degree program in the Faculty Impact and Academics sections of the Factbook.
Your responses are confidential and no identifying information is ever directly connected to the feedback you submit.
Explorance-created Student Guide (ESOP-relevant sections: access your evaluations, complete your evaluation)
Student auditors in PhD courses will have an opportunity to provide feedback on their course and educators via an OE-administered Qualtrics survey

Educator access to the Blue Course Evaluation System:
- Directly with onyen single-sign-on (SSO) at the UNC Blue Home: https://blueeval.unc.edu/blue/, select user group “Instructor“
- ConnectCarolina: Navigate to Admin WorkCenter Links and click on Course Evaluation System link
- Links provided in email invitation and reminders, including the active tasks list while evaluations are live
Monitor real-time response rates for live evaluations in Blue from the “Home” page and click on the “Subject Management” button in upper left corner
View reports in Blue (select user group “Instructor“) and from the “Home” page navigate down to the lower half to the “Report” section
- Report types include overall course report, individual instructor report, individual facilitator report
- Link for Guide for Educators – 3 Different Evaluation Reports
Course directors provide OE with data each semester so evaluations can be built in the Blue Course Evaluation System, including:
- Instructors to be evaluated
- Facilitators for PHCY 617 and PHCY 732 (fall terms), and PHCY 631 and PHCY 718 (spring terms)
- Any custom questions, adjustments to evaluation open or close dates, and other evaluation needs
- Information needed for Qualtrics-based evaluations of courses and educators (e.g. student auditors in PhD courses)
- Typically requested by OE in May/June for fall terms, Nov/Dec for spring terms, Mar/Apr for summer terms
Explorance-created Instructor Guide (relevant sections for ESOP Educators: access your evaluations and reports, view the Blue central page to open tasks and analyze results, view the Subject Management page to monitor your response rates and analyze results)

Administrator report access to the Blue Course Evaluation System as a dynamic report viewer (DRV):
- Directly with onyen single sign on (SSO) at https://blueeval.unc.edu/blue/, select user group “SOP_DRV_Users“
- ConnectCarolina: Navigate to Admin WorkCenter Links and click on Course Evaluation System link
Division chairs (DC) can view individual instructor reports and individual facilitator reports for educators associated with their respective division (CBMC, DPMP, DPET, DPOP, or PACE).
- Link for Division Chair DRV Instructions and Tips – Guide for Administrators
Program deans (PD) can view overall course reports categorized in the PharmD, PhD, or MS degree programs.
- Link for Program Dean DRV Instructions and Tips – Guide for Administrators
Link for Guide for Educators – 3 Different Evaluation Reports
Link for guide Blue Central Home Page Features for Administrators
Explorance-created Academic Leader Guide (relevant sections for ESOP Administrators: access your evaluations and reports, view the Blue central page to analyze results)

Evaluation Schedule (open & close dates):

PY2 Block 1: Mon 2/19 – Fri 3/1
Grad Block 1: Fri 2/23 – Fri 3/8
PY1 Whole: Mon 4/1 – Fri 4/19
PY3 Whole: Mon 4/1 – Fri 4/19
PY4 Seminar: Mon 4/8 – Fri 4/19
PY2 Block 2: Mon 4/15 – Fri 4/26
Grad Whole and Grad Block 2: Wed 4/17 – Fri 5/10
Undergraduate Minor: follows the evaluation schedule of the ESOP “parent” course

Graduate includes PhD, MS Residential, MS Online, and MPS Online courses

PY1 PBC: Mon 8/14 – Fri 8/25
PY3 Block 1: Tues 9/12 – Tues 9/26
Grad Block 1: Mon 10/2 – Fri 10/13
PY4 Seminar: Mon 11/6 – Fri 11/17
PY3 Block 2: Mon 11/13 – Thur 11/30
PY1 Whole: Mon 11/13 – Wed 12/6
PY2 Whole: Mon 11/27 – Fri 12/8
Grad Whole and Grad Block 2: Wed 11/29 – Fri 12/15

PHCY 512: Mon 6/19 – Fri 6/30
PHCY 850: Wed 7/5 – Wed 7/19

Policy and Procedures for Student Evaluation of Course and Educator Effectiveness

This document presents an overview of the School’s course evaluation process, and the purpose of gathering student feedback on all required and elective courses and educators at both the professional and graduate levels.

Instrument for Student Evaluation of Course and Educator Effectiveness

Instrument is updated as needed with input and endorsement of the School’s curriculum and assessment committees.

Summary Reports of Student Evaluation of Course and Educator Effectiveness

Organizations

Conferences and Events

Journals

Web-based Articles

Tools

Rubrics

Assessment Commons rubric resources (see “Assessment Rubrics” under “General Resources”)
Authentic Assessment Toolbox
National Institute for Learning Outcomes Assessment, Resource Library (search by “Rubrics”)
VALUE: Valid Assessment of Learning in Undergraduate Education

What is our process of assessment?

The mission of the Office of Organizational Effectiveness, Planning and Assessment (Office of OE) is to foster and support a collaborative and coordinated approach to assessment and sound decision-making, using data (quantitative and qualitative) that have been thoughtfully collected, critically interpreted, and discussed widely.

When effectively adopted, this process will facilitate the ability of the School to make decisions, better assess needs, set priorities, understand the impact of initiatives, and fulfill its responsibilities efficiently.

As illustrated in the diagram below, the closed loop process allows for the collection, analysis, and reporting of key data to those best positioned to enact meaningful change.

Closed Loop Assessment Process

How do we ensure a closed loop process of academic program assessment?

To effectively provide a high-quality student learning experience that prepares our graduates to be leaders and change agents in the profession of pharmacy, the School must engage in a systematic, closed loop assessment process that promotes continuous quality improvement.

The closed loop process ensures data are collected, managed, analyzed, and used to inform decision making and establish best practices to improve student learning and curricular effectiveness.

The Office of OE aids in facilitating this process by collecting, analyzing, and summarizing data findings for stakeholders, and ensuring a closed loop process is in place in which resultant action items are implemented, evaluated, documented, and communicated.

Key stakeholders review the generated summary report and develop recommendations and action items, as needed. Authoritative offices and assigned individuals oversee the implementation of these actions items to ensure the successful implementation of a closed loop process. Stakeholders are held accountable for agreed upon action items at subsequent Dean’s Council review meetings.

The School’s Factbook and Report Catalog summarize key assessment data including links to authoritative data (most data refreshed monthly) in Tableau to support data transparency and the ability for stakeholders to perform ad-hoc reporting. All data are also available to support the development of stakeholder reporting views to drive additional insights.