UP Manila Virtual Learning Environment

NATIONAL TEACHER TRAINING CENTER FOR THE HEALTH PROFESSIONS

UNIVERSITY OF THE PHILIPPINES MANILA

Program Outcomes:

BCHPEd:

Teachers with basic skills in teaching based on their roles:

Student performance Assessor, Clinical teacher, Curriculum & instructional planner, or administrator

DHPEd:

Utilization of research (application of evidence from research to teaching practice)

MHPEd:

Educational researcher and program evaluator

Utilize research and conduct research

PhD:

HP 232 TEST CONSTRUCTION AND ANALYSIS IN HEALTH SCIENCES

Course Description:Principles and methods with skill development in the construction and scoring of paper and pencil, oral and psychomotor tests, including comprehensive examinations, and the analysis of data from such tests; decision making based on such assessment.

Credit: 2 units

Faculty-in-charge:    Prof. Maria Lourdes Dorothy S. Salvacion, DDM, MHPEd

                                    mssalvacion1@up.edu.ph                                                                                          Consultation hours:  Tuesdays 2:30 - 5:00 pm; Wed. and Fri. 1:00 -5:00 pm

About this course:

Content

This course aims to enhance the skill of the participants in developing valid and reliable tests for various purposes, consistent with the learning outcomes.  Students will experience the intricacies of test development, implementation and analysis.

Course Outcomes:

The NTTCHP student who has completed this course shall be able to demonstrate ability to:

1.Effectively communicate to an interprofessional audience, concepts on assessment, testing, and measurement, in both written and oral form

2.Identify the significance and rationale of their testing methods to the needs of their Filipino students

3.Collaborate with other health professionals in the production of assessment tools, instruments, and other course outputs

4.Develop context-based assessment tools for their chosen unit of instruction, in line with instructional outcomes

5.Utilize effective teaching skills in presenting various assessment, testing, and measurement concepts

6.Develop different types of tools to assess learning and effectiveness of instruction

7.Critique given assessment tools based on a sound evaluation criteria

8.Utilize and analyze existing evidence in the production of a research project related to assessment, testing, and measurement

9.Lead classroom groups effectively

Objectives

At the end of this course, students will be able to:

1. Explain the role of tests in teaching in the health sciences.
2. Distinguish among testing, grading, assessment, measurement, and evaluation.
3. Discuss important test concepts: validity, reliability, norm referenced and criterion referenced testing.
4. Explain the qualities of an effective test.
5. Prepare a test blue print.
6. Discuss the uses, advantages and limitations of commonly used types of written tests.
7. Construct valid and reliable test items
8. Determine minimum passing level for a test.
9. Assign grade to a raw scores
10. Perform test item analysis.
11. Utilize relevant current scholarly work in the production of written and oral outputs
12. Communicate effectively using various forms of media

Structure: This course is divided into four units:

Unit 1: Assessment- Context, Issues, and Trends

In this unit, students will be introduced to the different concepts in assessment, testing, and measurement.  The role of testing in health professions education will be discussed as well as the different issues affecting testing.  The qualities of a good test will be explored and the steps in test development.

Key concepts:  achievement assessment, testing, measurement

Unit 2: Planning for Testing and Assessment

The second unit will introduce students to the planning phase in test development. The importance of choosing the appropriate assessment tools given the learning outcomes which cover the different learning domains is emphasized.  Students will also be asked to prepare a table of specifications which serves as the blueprint for any written test.

Key concepts:  learning domains and learning outcomes, purposes of various assessment tools, table of specifications

Unit 3: Constructing test items

This unit will help students differentiate between the different test types and formats and how items are constructed, assembled, and administered.

Key concepts:  objective test items, selection/ supply items, simple forms, multiple choice, complex achievement, interpretive exercise, essay, performance-based assessment, portfolio, OSCE

Unit 4: Interpreting Test Results

The last unit will look into grading, reporting, and interpretation of test results. Test and item statistics will be discussed and hands-on activities on the calculation of the minimum pass level, and item and test statistic.

Key concepts:  Minimum pass level, item statistic, item analysis, grading, test statistic

How to work during the course

Sessions will be composed of individual presentations (the topics of which, will be pre-assigned at the beginning of the course), workshops, discussions, and individual and group activities. 

Course evaluation

Grading for this course will be based on the following:

            Individual Presentations                      -                  15

            Research Project                                -                  15

Class Participation                              -                  20

Portfolio                                               -                  25

            Final Project                                        -           ___ 25____

                                                                                         100%

·Individual Presentations are evaluated using the following criteria:

Content                       -           20%

Visual Aids                  -           20%

Presentation Skills      -           30%

Creativity                     -           10%

Ability to answer questions __20%______

                                                100%

This criteria will be used by the presentor, the teacher, and the students in evaluating the presentation.

            Self-evaluation            -           20%

            Peer evaluation           -           30%

            Teacher evaluation     -           50%

                                                            100%

·A Research Project will be conducted throughout the semester to be done in groups. 

·Class Participation is evaluated by the teacher each session:

                        Attendance                                                                  -           40%

                        Willingness to share ideas                                           -           20%

                        Participation in individual and group activities            -           20%

                        Contribution to the quality of discussion                     -           20%

                                                                                                                 _________

                                                                                                                        100%  

·The Final Project will be an item analysis of a chosen written examination. 

·A Portfolio will be submitted at the end of the semester.  It will be a collection of test items in different formats to be developed throughout the course.

Course Schedule: Thursday 11:00 am - 1:00 pm, NTTC Classroom

Unit 1: Assessment- Context, Issues, and Trends

Unit 2: Planning for Testing and Assessment

Unit 3: Constructing test items

Unit 4: Interpreting Test Results

References

1. Constructing Written Test Questions For the Basic and Clinical Sciences. National Board of Medical Examiners. A 10 chapter manual providing in-depth guidance on constructing objective test items. Topic include: Issues Regarding Format and Structure of Test Questions, Importance of Psychometric Considerations, Multiple-Choice-Item Formats, Technical Item Flaws, Writing One-Best-Answer Questions, Testing Application of Basic Science Knowledge, Testing Application of Clinical Science Knowledge, Extended-Matching Items (R-type and Pick N type); Interpretation of Item Analysis Results, and Establishing a Pass/Fail Standard. Includes a useful appendix of sample item-writing templates, items, lead-ins, and option lists for the basic and clinical sciences. Full 181-page guide (PDF file)
2. Designing Tests. University of Tennessee at Chattanooga - Teaching Resource Center. Briefly outlines the appropriate use, advantages, disadvantages of multiple choice, true/false, matching, short answer, and essay test questions, including tips for constructing good types of each. Provides similar suggestions for oral exams, portfolio assessment and performance measurement.
3. Handbook for Teaching Assistants: Elements of Effective TA-ing: Evaluating What You've Done (Chapter 2). University of California - Santa Barbara. Provides a good, concise overview of the key elements involved in testing and grading, including test construction do's and don't, grading objective and essay tests, and evaluating your teaching. Written for TAs but also handy for new/novice faculty.
4. How Can We Construct Good Multiple Choice Items?. Cheung, D & Bucat, R. Using research evidence from the literature, identifies eight key guidelines for writing good multiple choice items, with examples drawn from chemistry.
5. How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Burton, SJ, Sudweeks, RR, Merrill, PF and Wood, B. An excellent booklet designed to help teachers (1) determine when to use/not use multiple choice items; (2) apply commonly-accepted criteria to evaluate multiple-choice items and identify specific item flaws; (3) improve poorly-written multiple-choice items; and (4) construct well-written multiple-choice items to measure a given objective.
6. How to Prepare Better Tests: Guidelines for University Faculty. Brigham Young University Testing Center. A concise, 17-page booklet that covers the key aspects of testing ia a Q&A format. Topics include: Developing a Test; Preparing, Assembling, and Administering a Test; Evaluating a Test; Scoring a Test and Interpreting and Using Test Results. Also includes a Checklist for Effective Tests and bibliography.
7. IDEA Papers - Improving Multiple-Choice Tests (No. 16). Kansas State University. Defines the common structure and various types of multiple choice items and outlines their strengths and limitations. Specifies preconditions for using multiple choice items, when to use them and how to construct good items. Recommends how to organize the layout of a multiple choice test and describes the use of multiple choice interpretive exercises (a series of multiple choice items posed in response to a common stimulus). Provides a list of references for further reading.
8. IDEA Papers - Improving Essay Tests (No. 17). Kansas State University. Defines the two major types of essay questions (extended and restricted response), outlines the strengths and limitations of essay tests and recommends when to use essay items and how to construct and score them. Provides a list of references for further reading.
9. Improving Your Test Questions. University of Illinois Urbana-Champaign - Center for Teaching Excellence. An excellent tutorial on writing good test items. Cover the following topics: Choosing between Objective and Subjective Test Items; Suggestions for Using and Writing Test Items; Multiple Choice; True-False; Matching; Completion; Essay; Problem Solving; Performance; and Two Methods for Assessing Test Item Quality. Includes checklists for evaluating test items and references for further reading.
10. Information for Designers of Instructional Systems: Test and Measurement Handbook (Air Force Handbook 36-2235). United States Air Force. A 240 page in-depth handbook providing information and guidelines for applying current instructional test and measurement learning theory to the design and development of criterion-referenced tests and the measurement of student performance, and information and guidelines for conducting validation of instructional resources and evaluation of instructional systems.
11. More Multiple-Choice Item Writing Do's and Don'ts (ERIC Digest). ERIC Clearinghouse on Assessment and Evaluation. Provides a fairly comprehensive list of recommendations for writing multiple choice items based on both psychometric research and logical deduction.
12. Selecting Test Items. Virginia Tech. Provides a table that compares and contrasts the pros and cons of various test item types and specifies what learning outcomes they best measure. Recommends that you select item types based on the outcomes you are trying to assess.
13. Test Blueprint. Virginia Tech. Provides an example of a test blueprint that lists your goals and objectives in the left-hand column, and the outcome behaviors you are using across the top row. Using this format, each cell identifies the assessment items used to measure those outcomes.
14. Test Construction Manual. Cornell University - Center for Learning and Teaching. Details in outline format the key procedures and steps involved in constructing good teacher-made tests. Discusses test planning (identify learning outcomes; preparing a table of specifications; and choosing appropriate test item types); test preparation (constructing good test items; reviewing and editing items according to guidelines; arranging items and groupings; preparing directions for the test and individual items; deciding on method of scoring); and test analysis and revision (performing test/item analysis to determine difficulty, discrimination and reliability; deciding to retain, edit or discard items; revising the test as a whole if necessary). PDF file.
15. Testing and Assessment Issues. Florida State University. Chapter 12 in Instruction at FSU: A Guide to Teaching & Learning Practices. Describes the types of tests that are typically used in university settings and general tips about testing, including how to plan a test and how to write test items. Also discusses ways of dealing with cheating.
16. Writing Multiple-Choice Test Items (ERIC Digest). ERIC Clearinghouse on Assessment and Evaluation. Provides basic suggestions based on conventional wisdom for planning and constructing multiple-choice tests, including stem and option writing.
17. Written assessment. Schuwirth, LW & van der Vleuten, CP. Full text of British Medical Journal article on this topic (BMJ 2003; 326: 643-645), from the ABC of learning and teaching in medicine series (clinical reviews).
18. Assessment of student achievement / C. Keith Waugh, Norman E. Gronlund. -- 10th ed. -- Boston: Pearson, c2013. 267p.
19. Assessment of student achievement /Norman E. Gronlund. -- 7th ed. -- Needham Heights: Allyn and Bacon, c2003. 229p.
20. Measurement and Assessment in Teaching 10^th Edition/ M. David Miller, Robert L. Linn, Norman E. Gronlund, c2009.

Mldss 1/1/2017