Topic outline
Course Information
Course description:
The course deals with principles, techniques and trends in evaluation practices used in health professional schools with focus on student achievement and effectivity of health professions educational programs.
Credits: 3 units (core course)
Schedule: Tuesday: 3 pm - 5 pm
Faculty in charge:
Nemuel S. Fajutagana, MD, MHPEd
Additional Course Information
TOPICS, REFERENCES and Final Paper Components
First Semester | SY 22-23
(Use this matrix as a simple guide in preparing your final paper components.)
Use this eBook for replay of recorded Zoom sessions for FS SY 22-23.
Unit 1: Overview of Evaluation
Schedule:
First Zoom Session
Date: Week 1 - 4
Date: September 6, 2022
Meeting ID: 960 4186 4313
Passcode: 16326447
Session Outcomes:
At the end of this session, participants will be able to:
- Discuss the role of evaluation in health professions education
- Use evaluation terms appropriately
- Explain major concepts in educational evaluation
- Explain Logic Model in the context of evaluation.
- Discuss principles in planning educational evaluation
- Discuss the different educational evaluation models and approaches.
In preparation for this session, please read available reading materials.
Complete pdf version of the book. Read Chapter 1 Using Logic Models
Jody L. Fitzpatrick, James R. Sanders, Blaine R. Worthen - Program Evaluation_ Alternative Approaches and Practical Guidelines-Pearson (2010).
This a newer edition than what we have in the Library.
Interactive Presentation
Topic 1.1: Introduction to Evaluation Process
Date: Week 3-5
Session Outcomes:
At the end of this session, participants will be able to:
- Discuss the role of evaluation in health professions education
- Use evaluation terms appropriately
- Explain major concepts in educational evaluation
- Explain Logic Model in the context of evaluation.
- Discuss principles in planning educational evaluation
- Discuss the different educational evaluation models and approaches.
- Formulate Evaluation Questions
SYNCHRONOUS SESSION: SEPTEMBER 20
To be able actively participate in this F2F (synchronous) session, read Chapters 4 to 8 of Worthen and Sanders.
In preparing your group's assigned report on the different approaches, I want you to apply those approaches in making evaluation decisions. For this week, you evaluative decisions will on the purpose, evaluation questions, and objectives for the evaluation of your identified evaluation 'object.'
Specifically, you are to do the following:
- Identify proposed object for educational evaluation (e.g. training program, educational innovation, educational tools, faculty development program, etc.)
- For the identified evaluation 'object' please formulate the following:
- A short description of the evaluation object (Name, current status, program goals and objectives):
- Do the matrix below: For each evaluation approach/model" formulate the appropriate:
- purpose of evaluation
- evaluation questions
- Be ready to present your matrix when requested during our F2F (synchronous) session.
Matrix: Approach, Rationale, Purpose, and Evaluation Questions
Evaluation Approach Rationale for use of this approach Purpose of Evaluation Evaluation Questions Expertise and Consumer Oriented Approaches Program Oriented Evaluation Approaches Decision Oriented Evaluation Approaches Participant Oriented Approaches 🎦 ┤WATCH├ 🎦
Video Instruction for Individual Assignment
In preparation for this session, please read available reading materials.
- Scanned from Worthen and Sanders.
Note:
In the 4th Edition, the approaches have been categorized into the following:
Approaches Representative Approaches Reporter Consumer and Expertise Oriented - Consumer Oriented
- Expertise Oriented
- Lally Pearl Malatang
- Edgardo Mendoza
- MA Ypil
Program Oriented - Objective Oriented (Tyler, Provus)
- Logic Model
- Theory Based
- Goal Free
-
KJ Bajandi
-
CL Guinto-Ilarde
-
Rovi Samedra
Decision Oriented - CIPP
- UCLA Evaluation Model
- Utilization Focused Evaluatio
-
R Esagunde
-
ML Palomar
-
SM Reyes
Participant Oriented - Practial Participatory Evaluation
- Developmental Evaluation
- Transformative Participatory Evaluation
- Stakeholder-Based Evaluation
- Empowerment Evaluation
- Deliberative Democratic Evaluation
-
J Ann Catindig
-
Maita Lopez
-
W Ypil
✍ ┤DO├ ✍
Topic 1.2: The Evaluation Question
Session Outcomes:
At the end of this session, participants will be able to:
- Discuss the role of evaluation in health professions education
- Use evaluation terms appropriately
- Explain major concepts in educational evaluation
- Explain Logic Model in the context of evaluation.
- Discuss principles in planning educational evaluation
- Discuss the different educational evaluation models and approaches.
- Formulate Evaluation Questions
🎦 ┤WATCH├ 🎦
🎥Video Lecture on Formulating Evaluation Questions 🎥
In this video, I will explain why it is important to have the right questions before one starts doing the actual evaluation process. I will also discuss how you can expand and then prioritize your evaluation questions.
😷
In preparation for this session, please read available reading materials.
Please read Chapter 13 of Evaluation Approaches by Worthen and Sanders, 4th Edition (pdf file is available in Course Information). Also read some of the shared Medical Teacher articles below. You may find some of the articles relevant to your own evaluation object of interest.
The contents of this page are just the highlights of Chapter 13. I strongly advise that you read the whole chapter for more details.
Recommended Reading
This material will be able to provide you with detailed instructions on how to prepare evaluation questions.
While the material is very much development project based, the discussion on how to formulate questions is also relevant to educational evaluation.
Learn from the articles provided.
😷
Unit 2: The Evaluation Proposal
In this unit, we will focus our attention to the development of your Health Education Program Evaluation proposal. So far, you have, I hope:
- Identified you evaluation 'object.'
- Decided on the purpose(s) of evaluation.
- Formulated your evaluation question (and decided on the possible approach or approaches to use).
✍Start drafting the first two sections of you Evaluation Proposal. Click 'evaluation report components' below.✍
📃List of components of your final program evaluation proposal📃
Sample projects for your inspiration
Note: Not for general circulation. These papers are not to be shared to non HP 231 students.
- Original paper published in Journal of Medical Education. JME is under the auspices of the Taiwanese Association of Medical Education.
Use this portal to submit Section 1 and Section 2 of your evaluation proposal. See evaluation report components for coverage.
Topic 2.1: The Evaluation Matrix
Completing an evaluation matrix is an important initial step in the development of your evaluation proposal. To learn more about the Evaluation Matrix, view the provided interactive presentation that I prepared. You can also see a sample matrix in Worthen and Sander's book.
✍Your individual assignment is to fill up the first four (4) columns of your 'Evaluation Matrix.' Please be ready to present your EM during our F2F session on October 13.✍
📖To know more about the different data collection procedures, please read Chapter 16: Collecting Evaluation Information of Program Evaluation by Fitzpatrick, Sanders, and Worthen and Part 2: Practical Data Collection Procedures of HANDBOOK OF PRACTICAL PROGRAM EVALUATION by Wholey et.al, pages 205-411.📖
💻👨Face to Face Session 👧💻
Agenda: Presentation of the first four (4) columns of your matrix.
Date: TBA
Please run the interactive below 👇for guidance in your preparation.
💻 ┤INTERACT├ 💻
- I prepared an example of how elements of Evaluation Matrix are generated. I used an actual evaluation project to illustrate.
📝Video Annotations 📝
Watch and Learn from each other's output.
Topic 2.2 Study Design and Evaluation Information Collection Plan
To further clarify your evaluation questions, you can use as guide the study design that you intend to use for your evaluation. You will also need your study design to finalize your evaluation information collection plan.
💻👨Face to Face Session 👧💻
Agenda: Presentation of the first five (5) columns of your matrix.
Session Focus: Column 4 and 5 of the Evaluation Matrix
Readings:
Chapter 15 and 16 of Evaluation Approaches by Worthen, et. al.
Part II and Part III of Handbook of Practical Program Evaluation by Wholey, et. al.Date: TBA
🎥Zoom Meeting and Annotations 🎥
🎦 ┤WATCH├ 🎦
Improving Further Your Questions. The Role of Study Design
To further clarify your evaluation questions, you can use as guide the study design that you intend to use for your evaluation. Note: The evaluation approaches helped you identify your evaluation focus in the conceptualization phase of your evaluation. To be able to further clarify your evaluation questions and finalize your information collection plan, you need to identify the study design you will use in your evaluation.
Please read Chapter 15 and 16 of Evaluation Approaches by Worthen, et.al. For quick reference, see TABLE 15.1 Characteristics of Commonly Used Designs in 388.
Collecting Evaluative Information: From Questions to Tools
After you have clarified and finalized your questions and study design, you will now be ready to finalize your data collection plan. This would mean completing the first five columns of your evaluation matrix.
💻 ┤INTERACT├ 💻
Constructing data collection. instruments
Online Lectures: The Kirkpatrick's Evaluation Model
💻 ┤INTERACT├ 💻
Discussion of L1 and L2
Interactive Version and the latest revision. Some examples relate to your presentations last week.
🎦 ┤WATCH├ 🎦
Video Lecture Version
- This topic
Synchronous Session: November 24
💻👨Face to Face Session 👧💻
Agenda: Presentation of Data Collection Plan and Sample Data Collection Tools
Session Focus: Column 4 and 5 of the Evaluation Matrix
Meeting ID:
Passcode:
Date: TBA
📖 ┤READ├ 📖
Readings: Chapter 15 and 16 of Evaluation Approaches by Worthen, et. al. Part II and Part III of Handbook of Practical Program Evaluation by Wholey, et. al.
📝┤DO├ 📝
Data Collection Tools
- Checklist
- Rating scale
- FGD / Interview Guide
- Questionnaire
- Record Review Guide
Topic 2.3: Data Analysis Component of the Matrix
In this topic you will learn how to fill-up the remaining parts of the Evaluation Matrix. Several interactive presentations have been prepared for you and you will also find our two main textbooks useful as you work on these components.
💻 ┤INTERACT├ 💻
Interactive Presentations
A simple presentation on how to describe and present variables or research data.
Are you having problems deciding on what statistical test to use for your research or evaluation projects?
Well, I have a simple solution for you. This simple interactive object can help you decide on the most appropriate statistical test based on a) the statistical question you want answered, and b) the type of data or variable that you will process. I also included simple glossary of terms just in case you need more basic information. Happy interacting.📖 ┤READ├ 📖
Downloadable and Mobile-Friendly Versions of your interactive files.
(Just in case you need to access them offline).
PDF Version