Organizational Effectiveness, Planning and Assessment (OE)
The Office of Organizational Effectiveness, Planning and Assessment (OE) is responsible for planning, assessment and continuous quality improvement throughout the School. OE carries out its mission by facilitating:
- Monitoring and reporting of progress on the School’s strategic plan
- Data collection in the Factbook to guide benchmarking, data reporting, and decision-making
- The course evaluation process to ensure continued quality improvement of teaching and learning
- Data collection and annual reporting of faculty activity and productivity;
- The administration, coordination, monitoring, and use of surveys and survey data conducted within the School
- Assessment of the development of student learning
- Assessment of degree program (PharmD, MS, PhD) effectiveness and impact
The Office of Organizational Effectiveness, Planning and Assessment (OE) manages all surveys for the School. Faculty, staff, and students conduct surveys throughout the year in an effort to collect meaningful information to guide quality improvement efforts across various aspects of the School and our programs. In addition, we contribute as a School to national surveys conducted by the American Association of Colleges of Pharmacy (AACP) and use this information as well in our quality improvement efforts.
Survey Policies and Procedures
This document presents an overview of the School’s survey policies and procedures for faculty, staff, students, and alumni. General policies and procedures outlined include requests from external institutions/ individuals; faculty, staff, and students interested in conducting a survey; survey approval, set-up and administration; IRB approval; and, survey data review, interpretation, and planning.
OE Survey Form (Powered by Qualtrics)
The OE Survey Form is used to inventory and characterize survey landscape of the School. If you wish to conduct a survey of faculty, staff, students, or alumni, please complete and submit a Survey Form to the Office of Organizational Effectiveness, Planning and Assessment (OE). Be prepared to share information about the purpose of your survey; who will respond to the survey; and, how the results data will be used.
If you or your office/division/organization has administered a survey, in accordance with School Survey Policy and Procedures, please use this form to document major survey findings and proposed actions for using survey results in planning and implementing program or organizational improvement or change. To assist OE in preparing our annual report of all School surveys, we will request that you complete and submit this form to us on an annual basis.
AACP Institutional Research Annual Surveys & Services
The AACP institutional research department gathers, analyzes and summarizes data related to the colleges and schools of pharmacy accredited by the Accreditation Council for Pharmacy Education. The UNC Eshelman School of Pharmacy uses the AACP surveys to assess trends and make decisions with regard to the School’s pharmaceutical education.
Student evaluation of course and educator effectiveness is one component of an overall course and educator assessment process. The evaluations serve to:
- provide students an opportunity to share feedback on the effectiveness of the course and the educators who teach in a course;
- provide educators with feedback regarding student-perceived effectiveness of the course as well as their teaching methodologies and styles;
- provide the School, specifically the Professional Program Curriculum and Assessment Committee and the Graduate Education Committee, with data to consider in course and curricular quality improvement efforts; and,
- provide the School’s leadership, including Division Chairs and the Executive Committee, with information regarding student-reported effectiveness and impact of faculty teaching efforts.
The Office of Organizational Effectiveness, Planning and Assessment (OE) assumes responsibility, on behalf of the School, for facilitating the course evaluation process for all required and elective course offerings in the School at both the professional and graduate levels.
Software: Explorance Blue
The School uses an online course evaluation system, Explorance Blue, to gather feedback on courses and educators. This system is contracted and used in collaboration with the University of North Carolina at Chapel Hill. Course directors, educators, and students can access course evaluations and course evaluation reports by visiting https://blueeval.unc.edu/blue/.
Information and Resources for:
- Student access to the Blue Course Evaluation System:
- Directly with onyen single-sign-on (SSO) at the UNC Blue Home: https://blueeval.unc.edu/blue/
- Canvas: Go to “Account” and click on “Blue CourseEval”
- Links provided in email invitation and reminders, including the active tasks list of open evaluations
- ConnectCarolina: Course Evaluation System link is under SelfService on the left. Blue Central Home Page will display all available evaluations.
- Students receive an email invitation on opening day with a direct link to their list of live evaluations, as well as periodic reminders until they have clicked SUBMIT to complete all evaluations.
- The feedback you provide on course evaluations is valued by course directors and educators, and used to improve their courses and teaching.
- The School reports aggregated metrics on response rates and overall educator and course effectiveness by degree program in the Report Catalog of the Factbook.
- Your responses are confidential and no identifying information is ever directly connected to the feedback you submit.
- Explorance-created Student Guide (ESOP-relevant sections: access your evaluations, complete your evaluation)
- Student auditors in PhD courses will have an opportunity to provide feedback on their course and educators via an OE-administered Qualtrics survey
- Educator access to the Blue Course Evaluation System:
- Directly with onyen single-sign-on (SSO) at the UNC Blue Home: https://blueeval.unc.edu/blue/, select user group “Instructor“
- ConnectCarolina: Navigate to Admin WorkCenter Links and click on Course Evaluation System link
- Links provided in email invitation and reminders, including the active tasks list while evaluations are live
- Monitor real-time response rates for live evaluations in Blue from the “Home” page and click on the “Subject Management” button in upper left corner
- View reports in Blue (select user group “Instructor“) and from the “Home” page navigate down to the lower half to the “Report” section
- Report types include overall course report, individual instructor report, individual facilitator report
- Link for Guide for Educators – 3 Different Evaluation Reports
- The School reports aggregated metrics on response rates and overall educator and course effectiveness by degree program in the Report Catalog of the Factbook.
- Course directors provide OE with data each semester so evaluations can be built in the Blue Course Evaluation System, including:
- Instructors to be evaluated
- Facilitators for PHCY 617 and PHCY 732 (fall terms), and PHCY 631 and PHCY 718 (spring terms)
- Any custom questions, adjustments to evaluation open or close dates, and other evaluation needs
- Information needed for Qualtrics-based evaluations of courses and educators (e.g. student auditors in PhD courses)
- Typically requested by OE in May/June for fall terms, Nov/Dec for spring terms, Mar/Apr for summer terms
- Explorance-created Instructor Guide (relevant sections for ESOP Educators: access your evaluations and reports, view the Blue central page to open tasks and analyze results, view the Subject Management page to monitor your response rates and analyze results)
- Administrator report access to the Blue Course Evaluation System as a dynamic report viewer (DRV):
- Directly with onyen single sign on (SSO) at https://blueeval.unc.edu/blue/, select user group “SOP_DRV_Users“
- ConnectCarolina: Navigate to Admin WorkCenter Links and click on Course Evaluation System link
- Division chairs (DC) can view individual instructor reports and individual facilitator reports for educators associated with their respective division (CBMC, DPMP, DPET, DPOP, or PACE).
- Program deans (PD) can view overall course reports categorized in the PharmD, PhD, or MS degree programs.
- Link for Guide for Educators – 3 Different Evaluation Reports
- Link for guide Blue Central Home Page Features for Administrators
- Explorance-created Academic Leader Guide (relevant sections for ESOP Administrators: access your evaluations and reports, view the Blue central page to analyze results)
- The School reports aggregated metrics on response rates and overall educator and course effectiveness by degree program in the Report Catalog of the Factbook.
Evaluation Schedule (open & close dates):
- PY1 PBC: Tue 8/20 – Fri 8/30
- PY3 Block 1: Mon 9/16 – Fri 9/27
- Grad Block 1: Mon 9/30 – Fri 10/11
- PY1 PHCY 509: Mon 10/28 – Mon 11/11
- PY4 Seminar: Mon 11/11 – Fri 11/22
- PY3 Block 2: Mon 11/11 – Fri 11/22
- PY1 Whole: Mon 11/18 – Tue 12/10
- PY2 Whole: Mon 11/18 – Tue 12/10
- Grad Whole and Grad Block 2: Mon 11/18 – Fri 12/13
- Undergraduate Minor: follows the evaluation schedule of the ESOP “parent” course
Graduate includes PhD, MS Residential, MS Online, and MPS Online courses
- PY2 Block 1: Mon 2/19 – Fri 3/1
- Grad Block 1: Fri 2/23 – Fri 3/8
- PY1 Whole: Mon 4/1 – Fri 4/19
- PY3 Whole: Mon 4/1 – Fri 4/19
- PY4 Seminar: Mon 4/8 – Fri 4/19
- PY2 Block 2: Mon 4/22 – Fri 5/3
- Grad Whole and Grad Block 2: Mon 4/22 – Fri 5/10
- Undergraduate Minor: follows the evaluation schedule of the ESOP “parent” course
Graduate includes PhD, MS Residential, MS Online, and MPS Online courses
- PHCY 512: Mon 6/19 – Fri 6/30
- PHCY 850: Wed 7/5 – Wed 7/19
Policy and Procedures for Student Evaluation of Course and Educator Effectiveness
This document presents an overview of the School’s course evaluation process, and the purpose of gathering student feedback on all required and elective courses and educators at both the professional and graduate levels.
Instrument for Student Evaluation of Course and Educator Effectiveness
Instrument is updated as needed with input and endorsement of the School’s curriculum and assessment committees.
Summary Reports of Student Evaluation of Course and Educator Effectiveness
- 2023-2024 Annual Student Evaluation of Course and Educator Effectiveness Summary Report
- 2022-2023 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2021-2022 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2020-2021 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2019-2020 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2018-2019 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2017-2018 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2016-2017 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2015-2016 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2014-2015 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2013-2014 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2012-2013 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
- 2011-2012 Annual Student Evaluation of Course and Instructor Effectiveness Summary Report
Organizations
- Accreditation Council for Pharmacy Education
- American Institutes for Research
- American Educational Research Association
- American Evaluation Association
- Association for the Assessment of Learning in Higher Education
- Association of American Colleges and Universities
- National Council on Measurement in Education
- National Institute for Learning Outcomes Assessment
- Southern Association of Colleges and Schools Commission on Colleges
- Southern Association for Institutional Research
- Virginia Assessment Group
Conferences and Events
- American Evaluation Association Annual Conference
- Association for Institutional Research (AIR) Conference
- Assessment Institute in Indianapolis – Indiana University Purdue University Indianapolis (IUPUI)
- Institute for the Development of Excellence in Assessment Leadership (IDEAL)
- Southern Association for Institutional Research (SAIR) Conference
- Virginia Assessment Group Annual Conference
- List of higher education assessment conferences
Journals
- Assessment in Education: Principles, Policy & Practice
- Assessment Update: Progress, Trends, and Practices in Higher Education
- Assessment and Evaluation in Higher Education
- Educational Assessment, Evaluation and Accountability
- Journal of Higher Education
- New Directions for Institutional Research
- New Directions for Higher Education
- Research in Higher Education
Web-based Articles
- Inside Higher Ed, Assessment and Accountability
- National Institute for Learning Outcomes Assessment, Resource Library
- Practical Assessment, Research, and Evaluation
- Research & Practice in Assessment
Tools
- American Association of Colleges of Pharmacy (AACP), Assessment and Accreditation Management System
- Association of American Colleges and Universities (AACU) Assessment Resources
- Assessment Commons
- The Measuring Quality Inventory
Rubrics
- Assessment Commons rubric resources (see “Assessment Rubrics” under “General Resources”)
- Authentic Assessment Toolbox
- National Institute for Learning Outcomes Assessment, Resource Library (search by “Rubrics”)
- VALUE: Valid Assessment of Learning in Undergraduate Education
What is our process of assessment?
The mission of the Office of Organizational Effectiveness, Planning and Assessment (Office of OE) is to foster and support a collaborative and coordinated approach to assessment and sound decision-making, using data (quantitative and qualitative) that have been thoughtfully collected, critically interpreted, and discussed widely.
When effectively adopted, this process will facilitate the ability of the School to make decisions, better assess needs, set priorities, understand the impact of initiatives, and fulfill its responsibilities efficiently.
As illustrated in the diagram below, the closed loop process allows for the collection, analysis, and reporting of key data to those best positioned to enact meaningful change.
Closed Loop Assessment Process
How do we ensure a closed loop process of academic program assessment?
To effectively provide a high-quality student learning experience that prepares our graduates to be leaders and change agents in the profession of pharmacy, the School must engage in a systematic, closed loop assessment process that promotes continuous quality improvement.
The closed loop process ensures data are collected, managed, analyzed, and used to inform decision making and establish best practices to improve student learning and curricular effectiveness.
The Office of OE aids in facilitating this process by collecting, analyzing, and summarizing data findings for stakeholders, and ensuring a closed loop process is in place in which resultant action items are implemented, evaluated, documented, and communicated.
Key stakeholders review the generated summary report and develop recommendations and action items, as needed. Authoritative offices and assigned individuals oversee the implementation of these actions items to ensure the successful implementation of a closed loop process. Stakeholders are held accountable for agreed upon action items at subsequent Dean’s Council review meetings.
The School’s Factbook and Report Catalog summarize key assessment data including links to authoritative data (most data refreshed monthly) in Tableau to support data transparency and the ability for stakeholders to perform ad-hoc reporting. All data are also available to support the development of stakeholder reporting views to drive additional insights.
Our Team
Chris Gosk
Director
cgosk@unc.edu
(919) 445-3035
Christina Pomykal
Program Manager
christina_pomykal@unc.edu
(919) 966-2021
Vishal Tiwari
Sr. Data Analyst
vishal.tiwari@unc.edu
(919) 445-4950
Jackie Zeeman, Pharm.D.
Clinical Associate Professor
jackie_zeeman@unc.edu
(919) 966-0884