______________________________

Editors

Sherralyn S. Cox, Ph.D., Walter E. Pofahl, II, M.D., and

Walter J. Pories, M.D.

Section Editors

Part One Eric A. Toschlog, M.D.

Part Two Carl E. Haisch, M.D., Walter E. Pofahl, II, M.D.

Part Three Scott G. Sagraves, M.D.

Part Four Jeffrey W. Hazey, M.D., Rosa E. Cuenca, M.D.

Part Five John C. Fitzpatrick, M.D., J. Scott Roth, M.D.

Part Six Jeffrey C. Pence, M.D.

Part Seven Walter E. Pofahl, II, M.D.

Ó 2002 The Association of Program Directors in Surgery

Arlington, Virginia

This curriculum is dedicated to the memory of Dennis W. Jahnigen (1947-1998) who served as leader, confidante, and mentor throughout much of the period of project development. It was Dr. Jahnigen who, as a leading educator and geriatrician, provided the impetus to link the scheduled third revision of this document with the addition of new learning objectives focusing upon those aspects of the surgical discipline that are critical for surgical residents as they work to meet the needs of the older surgical patient. We are humbled by Dennis’ spirit of openness and flexibility in creating a multidisciplinary project that would link the non-primary care specialties, including general surgery, in a working network of informed, skilled, and sensitive practitioners and scholars.
        We gratefully acknowledge the funding support that we have received from The John A. Hartford Foundation/American Geriatrics Society project, “Increasing Geriatrics Expertise in Surgical and Related Medical Specialties.” This support has enabled us to work to integrate the information on the geriatric aspects of surgical care throughout our curriculum, as was the wish and plan of Dr. Jahnigen.

The Editors

The fourth edition of the Surgical Resident Curriculum emphasizes the principle that positive educational outcomes are best attained if the goals, objectives, and expected outcomes are clearly defined for resident learners. The current curriculum revision includes: (1) competencies for surgical advances since publication of the third edition in 1999 and (2) refined and expanded objectives emphasizing the knowledge and skills needed for residents to gain expertise to meet the health care needs of their elderly patients.

Advances in science and technology continue in the dynamic field of surgery, including those advances in minimally invasive and robotic surgery, and in our understanding of molecular biology. Our aging population presents an ever-growing challenge. It is not uncommon to see 85-year-old patients undergo surgical procedures. Nor is it unusual for residents to communicate with the families of patients and patients in their ninth decade. We are grateful to the American Geriatrics Society and the John A. Hartford Foundation of New York City for providing the Association of Program Directors in Surgery (APDS) with funding for education to further the study and teaching of surgical geriatrics and for support of resident research and special interest group activities at surgical meetings.

We regard a curriculum as a road map for an educational journey. Just as in a trip across the United States, there may be many ways to get to the destination and there are a number of places to linger. Similarly, this curriculum document for the residency in surgery delineates competencies to be achieved while it facilitates choices. Surgical educators will tailor-make a curriculum for their own residents. This document can serve as a point of reference to faculty and education committees as they determine program priorities, to residents as they plan a course of action in their study and board preparation, and to program directors as they organize their documentation of education for the Accreditation Council for Graduate Medical Education (ACGME). As in previous years, to assist program directors, the Surgical Resident Curriculum is being made available electronically from APDS through these editors.

How could one use this curriculum? Here are several examples:

Select competencies from the curriculum as the basis for revising one’s existing residency content to meet the ACGME Outcome Project requirements

Direct the planning of rotations for residents by level

Guide a resident’s study program as he or she progresses through the various rotations

Provide a scaffold for the scheduling of formal lectures in basic science and the general surgical specialties

Help plan individual lectures and presentations (e.g., What should I cover in my presentation of hyperparathyroidism?)

Provide an outline of goals and objectives for the rotations through the surgical subspecialties

Organize reviews for ABSITE and the Qualifying and Certifying Examinations of the American Board of Surgery

Offer a "check-list" for general surgeons in practice to measure their competency

We are sure that you can think of other approaches as well. We offer you this guide with our best wishes for a challenging and rewarding residency. Good luck to you.

Sherralyn S. Cox, Ph.D.

Walter E. Pofahl, II, M.D.

Walter J. Pories, M.D.

It has now been more than fifteen years since the Association of Program Directors in Surgery (APDS) first supported the concept of a project to develop and disseminate a residency curriculum document representing surgical educators’ efforts to structure the extensive and complex knowledge, psychomotor skills, and attitudes that are Surgery. Department chairs, program directors, other surgical educators, and residents from a variety of institutions through the APDS provided conceptual guidance for the curriculum. Implementation has been facilitated by Mr. Tom Fise and Ms Liz Starnes at the Association.

The task of preparing the first edition of the Surgical Resident Curriculum fell to Jay C. Smout, Ph.D. His efforts in large part provided Surgery with its first national curriculum in 1992, following years of work on conceptual organization by Hazel M. Aslakson, Ed.D. and Walter J. Pories, M.D. The second edition, published in 1995 and headed by Sherralyn S. Cox, Ph.D. and Dr. Pories, saw content and organizational changes. M. Beth Foil, M.D. and Timothy N. Patselas, M.D rounded out the editorial team. The third edition, edited by Dr. Cox and Dr. Pories, continued refinement in 1999. Now, for the fourth edition in 2002, Walter E. Pofahl, II, M.D., and a group of surgeons as section editors have joined the team of Cox and Pories to provide new perspective to the project.

The American Geriatrics Society and John A. Hartford Foundation chose to fund the development of surgical geriatric materials through the existing team at East Carolina University because of the existence of the Surgical Resident Curriculum as a national curriculum vehicle. To date, nearly every surgical training program and many other entities such as libraries, bookstores, and practicing surgeons have placed orders with APDS to receive the document. More than 295 programs and individuals have requested and been provided electronic versions in addition to the Web version at the APDS site: http://www.apds.org/

Dissemination of the Surgical Resident Curriculum will undoubtedly serve as impetus for continuing discussions about what surgeons do and how they are educated to do it. We hereby acknowledge those of you who daily work to explicate the science and art that is Surgery.

Section 1.1

SURGICAL RESIDENT CURRICULUM GOALS

Summary Curriculum Goals:

The goal of the surgical curriculum is to assist program directors, faculty, and residents in their educational pursuits. Each program is required to have a clear set of goals that can be viewed as milestones or expectations, by level, for resident learners.

Specific Curriculum Goals:

· Create an organizational structure of academic, clinical, and technical criteria to facilitate the education of residents in general surgery.

· Provide an educational plan as an available guide for a diverse body of surgical programs, including: new programs, established programs, university- and community- based programs, public and private programs, urban and rural programs, five-year and six-year programs, and military-based programs.

· Maintain educational criteria that are congruent with the aims of the American Board of Surgery and the Residency Review Committee for General Surgery.

· Suggest teaching methodologies for expanding the number of ways to transmit knowledge, skills, and attitudes from faculty to residents.

· Establish the basis for evaluation activities tied to expectations of resident learning.

· Facilitate the self-directed study of residents via recommended readings and learning activities.

· Suggest learning experiences based on measurable objectives for the education of surgeons.

· Integrate principles of basic sciences with clinical experiences.

· Promote a broader understanding of the role of surgery and its interaction with other medical disciplines such as Internal Medicine, Psychiatry, and Pediatrics.

· Guide the mechanism for residents' progressive responsibility from initial patient care to complete patient management for all patient age groupings, from neonatal to the oldest-old.

· Provide surgical residents with a reference for functioning as teachers and consultants.

· Guide surgical residents to use research technology and skill in conducting studies that assist in solving surgical problems.

· Assist residents in achieving professional leadership and management skills.

· Promote the understanding of the economic, legal, and social challenges of contemporary and future surgery.

· Foster continuing education to promote lifelong individual initiative and creative scholarship.

Members of the Association of Program Directors in Surgery (APDS) Curriculum Committee and the Task Force prepared these curriculum guidelines for Surgical Geriatric Curriculum Development. The guidelines can be considered the starting point for a multifaceted program to assist surgical training programs organize and prioritize geriatric competencies for their general surgery residents. Since 1995, APDS has provided supporting structure for increasing geriatric knowledge and skills for general surgery residents, offering symposia, panel and workshop education sessions, and supporting reviews in the surgical geriatric literature for educators and practitioners. This work has been supported by generous funding from the John A. Hartford Foundation of New York City and the American Geriatrics Society (AGS), primarily through the project Increasing Geriatrics Expertise in Non-Primary Care Specialties. In the past year the AGS/Hartford Foundation has extended funding competitively to general surgery through the Geriatrics Education for Specialty Residents Program (GESR), a component of the broader project, Increasing Geriatrics Expertise in Surgical and Related Medical Specialties. In general surgery the following programs were selected to participate in the GESR initiative: University of California-Los Angeles, Yale University, University of Rochester, and East Carolina University.

The Surgical Resident Curriculum has integrated units, so that resident objectives related to care of the elderly patient are presented with, or immediately adjacent to, related surgical content. When one consults the table of contents, one can quickly determine if geriatric objectives are within a revised unit (e.g., Unit 2.2/2.2G, where "G" indicates "Geriatrics") or if the geriatric objectives are provided as a separate unit. This format allows for recognition of each contributor’s role.

The following guidelines provide the structural basis for increasing resident expertise in caring for the special needs of elderly patients.

CURRICULUM GOAL: Following study and implementation of a Surgical Geriatric Curriculum, the surgical resident will be prepared to manage or co-manage the health care needs of prospective surgical geriatric patients.

I. PRINCIPLES OF NORMAL AGING

The general principles will include the study of:

1. Demography of aging

2. Biology of aging relative to age-related physiologic changes

3. Preventive geriatrics: health maintenance

Elements of the Psychology of aging will include applying principles of:

1. Neuropsychiatric aging: brain-behavior relationships (dementia, acute delirium/changes in mental states)

2. Hypothalamic function and regulation of body temperature

 

Age-related physiologic changes will encompass:

1. Aging relative to tissues, organ systems, immune functions, and nutritional needs

2. Endocrine and metabolic alterations (e.g., carbohydrate and insulin metabolism)

3. Changes in laboratory values (e.g., expected changes in normal blood chemistries)

PATHOPHYSIOLOGY IN THE ELDERLY PATIENT

Knowledge of disease processes will include the study of:

1. Mortality: leading causes of death for those 65 and older

2. Morbidity: leading causes of disability

3. Factors affecting altered disease presentation

4. Comorbidity: chronic diseases superimposed on acute disease

5. Geriatric syndromes (dementia, failure to thrive, fractures, malnutrition, sleep problems)

Physiologic and Psychosocial implications will build upon a working knowledge of:

1. Changes in drug metabolism and excretion

2. Adjustment of doses and age-specific side effects

3. Use of psychotropic agents and pain medications

4. Identification of possible adverse drug-drug interactions

5. Significance of financial problems imposed by polypharmacy

III. PREOPERATIVE ASSESSMENT OF THE ELDERLY PATIENT

Factors to consider will include:

1. Developing attitudes toward and communicating with the elderly; age bias

2. Establishing lines of communication with health care team: personal physician/geriatrician, social worker

Full geriatric assessment of patient baseline data will include consideration of:

1. Functional capabilities: activities of daily living, mental and physiologic health

2. Psychosocial variables: ethnic factors, cultural mores, social supports, and community relations

3. Differences in health care preferences according to perspectives of patient, referring physician, and surgeon

4. Considering risks to desired surgical outcomes: comorbidity, frailty, and social supports

 

Interventions will require consideration of the following factors:

1. Weighing aggressive approach with patient’s right to autonomy: legal right to self-determination and perceptions of quality of life

2. Rights regarding competence and advance directives: informed consent, surrogate decision making, long-term care, extent of care, living wills, and decisions about death

3. Cost:benefit ratio determination

IV. OPERATIVE MANAGEMENT OF THE ELDERLY PATIENT

Monitoring of patient surgical needs will include:

1. Planning and supporting the selection and management of local, regional, and general anesthetics

2. Managing conscious sedation

3. Maintaining body temperature and metabolic homeostasis during surgery

4. Following Halsted’s Principles during surgical intervention

V. PERIOPERATIVE CARE OF THE ELDERLY PATIENT

Perioperative decisions will require:

1. Management of complications such as sepsis, cardiac problems, diabetes, pulmonary and renal failure.

2. Determining need for prophylaxis for common complications like DVT and PE, aspiration pneumonia

3. Sustaining patient with homeostasis, fluid management, ventilator support, wound and antibiotic management

4. Determining management for deconditioning, use of Foley catheters and NG tubes, use of invasive monitoring

5. Management of directive care issues such as life sustaining mechanisms: supportive care, extent of care issues

VI. LONG-TERM RECOVERY/REHABILITATION OF THE ELDERLY PATIENT

Data and resource utilization will include application of rehabilitation principles:

1. Optimizing patient health and maintaining function

2. Communicating with the patient and family regarding quality of life issues

3. Directing long-term recovery and rehabilitation for home, community, and/or institutional settings

4. Applying non-institutional support systems and institutional services for patient and family

 

VII. FINANCING, UTILIZATION, AND REIMBURSEMENT ISSUES

The consideration of factors related to health services will include an analysis of:

1. Elderly patient rights to benefits: age-based and needs-based services and entitlements

2. Delivery of health services available to the patient and his/her family

3. Cost:benefit ratio determination; economic impact of operative procedure

4. Implications of long-term care: the recovery period, quality of life

VIII. PATIENT OUTCOMES

Analysis and utilization of surgical data will include:

1. Selecting, maintaining, and analyzing a patient outcome database

2. Comparing patient outcomes with local medical community and national standards

3. Initiating improvements in patient care based on patient outcome data

SELECTED BIBLIOGRAPHY:

Adkins RB, Jr., Scott HW, Jr. (eds). Surgical Care for the Elderly (2^nd ed). Philadelphia: Lippincott-Raven Publishers, 1998;1-531.

Beers MH, Berkow R (eds). The Merck Manual of Geriatrics (3^rd ed). Whitehouse Station, NJ: Merck Research Laboratories, 2000;1-1507.

Binstock RH, Post SG (eds). Too Old for Health Care? Controversies in Medicine, Law, Economics, and Ethics. Baltimore: The Johns Hopkins University Press, 1991;1-209.

Callahan D, Meulen RHJ, Topinkova E (eds). A World Growing Old: The Coming Health Care Challenges. Washington DC: Georgetown University Press, 1995.

Cobbs EL, Duthie EH, Jr., Murphy JB (eds). Geriatrics Review Syllabus: A Core Curriculum in Geriatric Medicine (4th ed). Dubuque IA: Kendall/Hunt Publishing Company, 1999; Books 1-3.

Corman JM. Extension-of-life working group: questions for societies with "third age" populations. Acad Med 1997;72(10):856-862.

Cox SS. Surgical geriatrics: how should we teach it? Curr Surg 1999;56(3):133-135.

Cox SS. A call for educators in surgical geriatrics. Focus on Surg Ed 2000;18(1):34-36.

Forciea MA, Lavizzo-Mourey RJ. Geriatric Secrets. Philadelphia: Hanley & Belfus, Inc. and St. Louis: Mosby, 1996;1-275.

Graduate Medical Education Committee, American College of Surgeons. Prerequisite objectives for graduate surgical education: a study of the Graduate Medical Education Committee, American College of Surgeons. J Am Coll Surg 1998;186:50-62.

Ham RJ, Sloan PD (eds). Primary Care Geriatrics: A Case-Based Approach (3^rd ed). St. Louis: Mosby, 1997;1-615.

Interdisciplinary Leadership Group of the American Geriatrics Society. A statement of principles: toward improved care of older patients in surgical and medical specialties. JAGS 2000;48:699-701.

Jahnigen DW, Schrier RW. Geriatric Medicine (2^nd ed). Cambridge MA: Blackwell-Science, 1996;1-867.

Kramer AM. Health care for elderly persons—myths and realities. N Engl J Med 1995;332:1027-1029.

McLeskey CH (ed). Geriatric Anesthesiology. Baltimore: Williams & Wilkins, 1997;1-703.

Meakins JL, McClaran JC (eds). Surgical Care of the Elderly. Chicago: Year Book Medical Publishers, Inc., 1988;1-537.

Reuben DB, Herr K, Pacala JT, et al. Geriatrics at Your Fingertips. BelleMead, NJ: Excerpta Medica, Inc., 2001;1-215.

Rosenthal RA, Zenilman ME, Katlic MR (eds). Principles and Practice of Geriatric Surgery. New York City, NY: Springer, 2001;1-1098.

Sanders AB (ed). Emergency Care of the Elder Person. St. Louis: Beverly Cracom Publications, 1996;1-305.

Soloman DH, Burton JR, Lundebjerg NE. The new frontier: increasing geriatrics expertise in surgical and medical specialties. JAGS 2000;48:702-704.

Zenilman ME. Surgery in the elderly. Current Problems in Surgery 1998;35(2):1-179.

This curriculum, the fourth produced under the auspices of the APDS, is designed with function and utility in mind for surgical residents, surgical faculty including program directors, and any others who would benefit from clear sets of goals, along with knowledge and performance objectives, delineated as expected accomplishments by learners. There is no national consensus about what is the essential knowledge or skills or attitudes for surgeons. Not yet. But as surgical educators proceed with the development of clearer ideas and requirements for their learners’ curricula, and as we educators move closer to accurately measuring the accomplishments of our learners, we zero in on the "meat and potatoes" of what it takes to produce competent graduates of surgical training programs, effectively and efficiently.

We know that previous editions of this curriculum have been utilized, through the facilitation of John T Boberg, PhD (Jack), by the various Accreditation Council for Graduate Medical Education (ACGME) committees working toward what now has become the Outcome Project. This APDS curriculum attempts to map milestones for resident learners which, when successfully accomplished, are indexes of competence. The Surgical Resident Curriculum represents an effort to document enabling objectives representing the educational activities and procedures that combine to clarify the expected outcomes of a general surgery residency.

The Surgical Resident Curriculum constructs professional competencies as the bases for objective development and inclusion in each curricular unit. The core competencies were structured first as summary curriculum goals descriptive of the desired outcomes of surgical education. When competencies are viewed in conjunction with the objective criteria in each unit, one has a combination of indicators of what is essential for resident learning, and one can employ these essentials in implementing an instructional program.

The educational areas in this surgical resident curriculum, for which competencies and instructional criteria exist, are these:

Integration of theory and practice

Application of surgical skills

Increasing expertise in care for elderly patients

Use of critical thinking

Exercise of ethical judgment

Use of appropriate communication

Recognition of teaching responsibilities

Development of management abilities

Teaching and learning for a lifetime

 

The general competency areas for residents, in which residency programs are required by the ACGME, as explained more fully on the Web at (http://www.acgme.org/Outcome/comp2.asp) to define specific knowledge, skills, and attitudes are these organizing principles:

PATIENT CARE that is compassionate, appropriate, and effective for the treatment of health problems and promotion of health

MEDICAL KNOWLEDGE about established and evolving biomedical, clinical, and cognate (e.g. epidemiological and social-behavioral) sciences and application of this knowledge to patient care

PRACTICE-BASED LEARNING AND IMPROVEMENT that involves investigation and evaluation of their own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care

INTERPERSONAL AND COMMUNICATION SKILLS that result in effective information exchange and teaming with patients, their patients’ families, and other health professionals

PROFESSIONALISM as manifested through commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population

SYSTEMS-BASED PRACTICE as manifested by actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value

 

It seems prudent to consider that the educational areas and organizing principles listed above identify the content divisions that are critical for the comprehensive educational and professional preparation of a surgeon. We regard these listings as the competency-based structure of our curriculum. A competency-based education program, anchored by this structure, creates an educational back-up system of knowledge, skills, and attitudes that are helpful in assuring the public that a program graduate is competent to practice.

The objectives in each unit of the Surgical Resident Curriculum describe the learning activities that are to occur during the course of curriculum implementation. When these objectives have been met, the expected outcome is that core competencies, describing the abilities made possible by a professional education, can be performed acceptably. The competencies specify what the resident should know, be able to do, or have an attitude about at the completion of a defined point during or immediately upon completion of the surgical training. The following statement should preface each of the core competencies listed below:

· Make sound ethical and legal judgments appropriate for a qualified surgeon.

· Respect the cultural and religious needs of patients and their families, and provide surgical care in accordance with those needs.

· Manage surgical disorders based on a thorough knowledge of basic and clinical science.

· Utilize appropriate skill in those surgical techniques required of a qualified surgeon.

· Use critical thinking when making decisions affecting the life of a patient and the patient's family.

· Collaborate effectively with colleagues and other health professionals.

· Teach and share knowledge with colleagues, residents, students, and other health care providers.

· Teach patients and their families about the patient's health needs.

· Be committed to scholarly pursuits through the conduct and evaluation of research.

· Be prepared to manage complex programs and organizations.

· Provide cost-effective care to surgical patients and families within the community.

· Value lifelong learning as a necessary prerequisite to maintaining surgical knowledge and skill.

 

The purpose of the curriculum is to define educational activities in a structure to serve as a program resource. With this structure in hand, especially in electronic editable form, programs will determine their own best use of the curriculum document for integrating resident educational experiences.

Selected strategies for utilizing the Surgical Resident Curriculum follow.

· The six areas are patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.

· Combine objectives from several curriculum units that lead to experiences that define one or more of the required principles (e.g., curricular units on ethics and communication can assist one in defining behaviors displaying interpersonal and communication skills and/or professionalism).

Provide copies of the Surgical Resident Curriculum for faculty and residents.

· This can be accomplished by highlighting the curriculum at a faculty meeting and then routing it to those who express interest.

· One or more copies can be given to chief residents for dispensing to other residents in their library or lounge, during study periods, or through a checkout system.

· All education stakeholders can be notified of document availability through the departmental office of surgical education.

· Personal copies can be provided for each resident.

Establish an educational dialogue between program director, faculty, and residents.

· Dialogue can occur in curriculum conferences, education and/or curriculum committee agendas, faculty meetings, or resident seminars. There are many significant forums for exchange, not the least of which is the one-on-one discussion that occurs during teaching rounds.

· Focus the dialogue with questions important to all stakeholders, such as: What does our program consider to be essential? What are our residents expected to do and when? How will we determine if our residents have performed acceptably?

· Each faculty member should have access to the objectives appropriate to his/her clinic and hospital practice and teaching situations.

· Faculty analysis can determine which aspects of the curriculum can be self-learned and which need faculty explanation, structured development, and testing for competence or mastery.

· Emphasize that the criteria clarifying most objectives are included as examples of cognitive activities to consider or procedures to perform. These criterial listings are not exhaustive; they are not intended to be complete inventories, but they are indicative of what is essential. Specific faculty guidance must indicate program expectations.

 

Make a copy of the curriculum objectives to be available to conference planners/presenters.

· Maintain a running account of when and what coverage occurs. Selected faculty can be responsible for overseeing topical presentations on a cyclical basis (e.g., it may be prudent to hold grand rounds on surgical core topics at least every two years so that all residents have a recurring review of the topic's importance as it relates to their progress).

· Selected faculty can serve as facilitators for the optimal use of the curriculum by residents.

· Although many of the learning objectives will be accomplished prior to completion of the indicated residency period, all of the competencies should be met prior to completion of the surgical training program. The training program itself will establish the time frame within which residents are expected to be competent in each area presented in the curriculum.

· The curriculum is designed to require progressively increasing levels of resident responsibility. In most instances, the surgical specialty units do not show separate junior and senior objectives because rotations occur at different chronological periods according to the residency program. However, all competency-based objectives have been developed with the intention of progressing from the more basic (generally the beginning of each unit) to the more complex (generally the latter portions of each unit).

· Established levels of knowledge, performance, and attitudes can be set as program goals or requirements, with oral quizzes, written tests, performance demonstrations, and/or case presentations providing evaluative data on resident qualifications. Resident reflection and self-assessment is another important aspect of evaluation.

Formally incorporate the curriculum into the department's performance protocols.

· Expected performance at predetermined points during the residency form the basis for formative and summative evaluation of resident performance.

· Performance not meeting minimal standards established in the curriculum requires administrative and resident recognition, documentation, and action.

· Determine a schedule of remediation and supplementary learning for residents not meeting minimum standards.

Tailor-make the Surgical Resident Curriculum, or portions thereof, so that it is reflective of your own philosophies, conditions, and goals.

· Create a deliberative body to consider the curriculum coverage and presentation.

· Utilize the optional curriculum on diskette or via e-mail in redesigning your own curriculum from the existing structure.

· Determine which objectives are not essential, not realistic, and/or not otherwise appropriate for your program. Omit them.

· Determine which objectives are to be retained. Evaluate the clarity of each objective. Would your residents know what is expected of them? Examine the verbs used in stating the expected behaviors. Are these verbs appropriate for your own program's educational goals? Change any objectives to reflect your faculty/resident deliberative body's determination of what residents should be competent to do.

· Consider the educational resources, service obligations, and rotation patterns that characterize your program.

· Determine how much of the curriculum can be implemented during the first established time period. This includes earmarking the specific goals and objectives to be incorporated into the program's educational coverage over time.

· Assess whether or not learning resources are adequate.

· Formulate teaching strategies and methods to best accomplish your goals.

· Determine if formal guidance/instruction is necessary to maximize faculty skill and cooperation in stabilizing the program's curricular structure, content, and assessment procedures.

· Consider establishing a program of mentoring, peer review, and/or other kinds of networking to assist faculty in their roles as educators, assessors, and evaluators.

· Examine curricular coverage of desired and essential topics.

· Oversee the economic and efficient use of resources.

· Solicit faculty opinion and critique.

· Determine faculty development needs.

· Analyze clinical outcomes.

· Consider resident academic performance and change over time.

· Solicit the assessment of peers and experts from outside your department/ institution/ discipline.

· Determine a plan to implement change.

· Your experiences, suggestions, and critique will be valued and utilized by the curriculum editors in the ongoing development of improved resident curriculum resources.

Dr. SS Cox, Dr. WE Pofahl, II, Dr. WJ Pories Phone: 252-816-5353

Department of Surgery Fax: 252-816-3156

East Carolina University coxs@mail.ecu.edu

301-A PCMH-TA pofahlw@mail.ecu.edu

Greenville NC 27858-4354 pories@mail.ecu.edu

Accreditation Council for Graduate Medical Education. Outcome Project Website is

http://www.acgme.org/outcome/comp2.asp

Corbett EC. Defining educational objectives at the University of Virginia. Acad Med 2000;75(2):151-152.

Curry L, Wergin JF (eds). Educating Professionals: Responding to New Expectations for Competence and Accountability. San Francisco: Jossey-Bass Publishers, 1993.

DaRosa DA, Derossis A. Applying instructional principles to the design of curriculum. In: Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000;57-68.

Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000;343pp.

Heppell J, Beauchamp G, Chollet A. Ten-year experience with a basic technical skills and perioperative management workshop for first-year residents. Can J Surg 1995;38(1):27-32.

Lane DS, Ross V. The importance of defining physicians’ competencies: lessons from preventive medicine. Acad Med 1994;69(12):972-974.

Leibrandt TJ, Kukora JS, Dent TL. Integrating educational objectives and the evaluation process in a general surgery residency program. Acad Med 2001;76(7):748-752.

McGaghie WC. Evaluating competency for professional practice. In: Curry L, Wergin JF (eds), Educating Professionals: Responding to New Expectations for Competence and Accountability. San Francisco: Jossey-Bass Publishers, 1993.

Nash DB, Markson LE, Howell S, Hildreth EA. Evaluating the competency of physicians and practitioners: from peer review to performance assessment. Acad Med 1993;68(2):519-522.

Reisdorff EJ, Hayes OW, Carlson DJ, Walker GL. Assessing the new general competencies for resident education: a model from an emergency medicine program. Acad Med 2001;76(7):753-757.

Most of us agree that our roles as surgical educators are significant. And, since what is significant most often requires time, effort, and a considerable amount of thinking, our roles as educators cannot be taken lightly. Indeed, Aristotle described teaching as the highest form of understanding. What is it that motivates surgeons, basic scientists, and other professional educators to want to teach future surgeons? What is it that makes surgical teachers effective in imparting the current wisdom and in stimulating the formulation of future wisdom? Are we limited by the belief that we should teach in just the way that we were taught? Are we resistant to change? How can we as educators ensure that learning truly accompanies all the teaching that occurs; for without learning, has there really been any teaching?

These questions are more than rhetorical. They represent the professional queries surgical educators have about their roles as teachers and socializing agents. The questions represent the professional accountability for resident learners that surgical faculty members share when they endeavor to cast the surgeons of the next generation. Accountability describes the surgeon's role for his/her patients, and it summarizes the professional commitment to one's staff, peers, and profession. In the same fashion, surgical educators are accountable for the learning of their residents in multiple learning environments, such as at bedside, in ambulatory care settings, and in the operating room. But just as faculty are accountable to residents and their profession, so also are residents responsible for maximizing their own learning.

Medical and surgical educators have long emphasized the principles of adult learning which so appropriately fit the characteristics and circumstances of surgical residents. Residents learn best when the environment is supportive; when there is mutual trust, respect, and encouragement. They relate to their past experiences in solving current problems, learning best when they actively participate in problem solving. Indeed, their ability to learn independently marks them as suitable candidates to become surgeons. But learning works best when there is a joint effort with the faculty. Residents respond best to immediate feedback regarding their professional performance. An important aspect of gaining skill is the opportunity to practice a new behavior. Positive faculty reinforcement is a powerful tool for resident learning, so is incisive faculty critique of resident performance. Trainees who lack sufficient experience in practice can benefit by more explicit structure from the faculty member as a performance guide toward the resident’s achievement of expected outcomes.

When programs adjust the Surgical Resident Curriculum to suit their own philosophies and learning environments, they endorse the curriculum theory that emphasizes structure to meet the needs of resident learners in fulfilling program goals and objectives and in meeting national mandates for competency. When residents utilize a specific curriculum, they have guidance in prioritizing their personal program of self-study. What does the curriculum communicate to residents? There is a comprehensive presentation of professional competencies that program faculty agree represents background knowledge residents will hold at specified intervals in their training. Faculty must emphasize which aspects of the curriculum should have been introduced during medical school, which are to be learned early in the residency, which are required before graduating to senior level responsibilities, and which may not be accomplished until the final months of training. Currently, a resident’s competency is determined in the context of his or her own training program.

The Surgical Resident Curriculum presents a comprehensive package of clinical skills which faculty expect residents to perform competently at specified intervals. The value of this written commitment to specific behaviors is that residents can monitor their own learning under the tutelage of faculty or more senior residents. With structure (e.g., documented faculty expectations) the learning program is transformed from an often frustratingly global directive of "just know everything" to a more educationally sound message of "let's work together to assure your competence in these areas."

There are times when the surgical resident is more than a learner. He or she also serves as a teacher. In the teaching role, the resident practices by understanding, by interpreting, and by mimicking. Good faculty role models lead residents to transmit successful learning formulas to medical students, more junior residents, and to patients and their families. The curriculum, presented in the order of progressively higher levels of knowledge, skill, and responsibility can assist the resident to organize, prioritize, and impart information.

An additional curricular consideration is that while residents are often teachers, faculty are often students. The professional education literature documents several significant aspects of faculty teaching and learning that are applicable to surgery:

· The stimulating educational environment of a collegial surgical residency motivates faculty as lifelong learners. But faculty learning also comes from interacting with their residents who provide new insights and new questions.

· One pressing faculty need may be in the area of personal development for curriculum utilization. Anyone who believes that successful teaching comes naturally probably has a limited experience.

· Quality teaching requires considerable scholarship, and that scholarship should be recognized and rewarded just as are other faculty accomplishments. The Surgeons as Educators course of the American College of Surgeons is a prime example of scholarship and teaching.

· In the past few years there has been a growing number of national conferences, seminars, and fellowships focusing on improving the abilities of residents and faculty as instructors. More frequently, medical schools, hospitals, and other institutional settings are recognizing the need to design and implement faculty development programs to create curricula, enhance teaching skills, and promote the assessment of educational outcomes.

· Faculty members can improve their own insight into what it is they want to accomplish when their research includes education questions. Participation in a faculty special-interest group formed to discuss teaching and learning can be helpful. Group goals can include broadening insights by sharing ideas and reflection with colleagues, students, and residents; fielding common problems; reviewing the literature; and generally stimulating new thoughts and questions about teaching and learning.

· In a time of accountability, institutions are adopting policies to comply with state legislation mandated for the purpose of documenting teaching effectiveness--"proving" that the education they offer truly is value-added or that it adds value to resident performance. Documentation of teaching efforts is now included in faculty portfolios accompanying promotion and tenure applications as well as in the post-tenure review packet. The documentation of one's own teaching philosophy, techniques, evaluations, and progress is recognized as an important part of teaching accountability.

A final consideration in this section is selection of specific teaching strategies and methodologies. While creative teaching has endless possibilities, several Surgical Resident Curriculum authors and evaluators provided their own recommendations for facilitating and improving resident learning. A selection of teaching practices follows:

Weekly, focused didactic sessions between surgical faculty and residents are critical during the junior years, especially regarding expectations in the basic sciences. At the center of these sessions are specific reading assignments, discussed in classic give-and-take "pimping"- format designed to debrief the resident in selected areas.

 

Teaching rounds should occur at least weekly and should include detailed discussions of specific anatomy and physiology. Residents should do more and watch less. Team learning, where residents work together on problem solving and more interactive presentations like those activities required in evidence-based medicine, have proven to be helpful.

 

Discussions of clinical scenarios, certainly incorporated into weekly Morbidity and Mortality conferences, are an efficient way to stimulate and encourage resident involvement. Such activity prevents a passive resident role.

 

Conferences provide the setting for needed didactic lectures. But the teaching faculty can also emphasize resident participation through Socratic questioning techniques or actual resident presentation of the conference. A structured reading program, including Selected Readings, Yearbook of Surgery, and SESAP Reviews, in addition to the periodicals focused upon in journal clubs and case review sessions, is essential.

 

Anatomic and physiologic considerations are taught with patients, models, text reviews, and discussion.

 

Skills laboratories, including use of cadaver study and animal models, training devices, simulators, computers, and robots are highly recommended by those sophisticated enough to have incorporated the technologies into their programs. Autopsy or cadaveric dissection and mock operations provide needed practice.

 

Structured clinical instructional module (SCIM) and Objective structured clinical examination (OSCE) use allows practice and direct feedback in addition to more accurate performance evaluation on skills achievement.

 

Residents should do more and watch less. Participating in team learning can be valuable.

 

The integration of technology into clinical teaching assists residents in understanding the "big picture." But excessive emphasis on technique, skills, and cognitive factors tends to create residents who are like encyclopedias rather than professionals who can apply and transfer what they learn to other settings and contingencies.

 

Tutorials, especially computer-aided instruction using CD-ROM technology, show promise as educational adjuncts.

ACS. Prerequisites for graduate surgical education: a guide for medical students and PGY-1 residents. Chicago: American College of Surgeons, 1998.

Apter A, Metzger R, Glassroth J. Residents’ perceptions of their role as teachers. J Med Educ 1988;63:900-905.

Bardes CL, Hayes JG. Are the teachers teaching? Measuring the educational activities of clinical faculty. Acad Med 1995;70(2):111-114.

Bartlett RH, Zelenock GB, Strodel WE, et al. Medical Education: A Surgical Perspective. Chelsea, MI: Lewis Publishers, Inc., 1986.

Bland CJ, Wersal L, VanLoy W, Jacott W. Evaluating faculty performance: a systematically designed and assessed approach. Acad Med 2002;77(1):15-30.

Blount BW, Jolissaint G. Perceptions of teaching behaviors by primary care and non-primary care residents. Acad Med 1996;71(11):1247-1249.

Blue AV, Griffith CH, III, Wilson J, et al. Surgical teaching quality makes a difference. Amer J Surg 1999;177(1):86-89.

Brookfield SD. Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass Publishers, 1995.

Charlin B, Tardif J, Boshuizen HPA. Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning instruction and research. Acad Med 2000;75(2):182-190.

Copeland HL, Hewson MG. Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Acad Med 2000;75(2):161-166.

Curry L. Cognitive and learning styles in medical education. Acad Med 1999;74(4):409-413.

Curry L, Wergin JF (eds). Educating Professionals: Responding to New Expectations for Competence and Accountability. San Francisco: Jossey-Bass Publishers, 1993.

Da Rosa DA, Folse JR, Sachdeva AK, et al. Description and results of a needs assessment in preparation for the ‘Surgeons as Educators’ course. Amer J Surg 1995;169(4):410-413.

Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000;343pp.

Dunnington GL. Surgeons as Educators: A Syllabus. Chicago: American College of Surgeons, 1998.

Dunnington GL. The art of mentoring. Amer J Surg 1996;171(6):604-607.

Dunnington GL. Adapting teaching to the learning environment. In: Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000;69-83.

Dunnington GL, Da Rosa DA. Changing surgical education strategies in an environment of changing health care delivery systems. World J Surg 1994;18(5):734-737.

Edmonson KM, Smith DF. Concept mapping to facilitate veterinary students’ understanding of fluid and electrolyte disorders. Teach and Learn in Med 1998;10(1):21-23.

Elliott DL, Skeff KM, Stratos GA. How do you get to the improvement of teaching? A longitudinal faculty development program for medical educators. Teach Learn Med 1999;11:52-57.

Farquharson A. Teaching in Practice: How Professionals Can Work Effectively with Clients, Patients, and Colleagues. San Francisco: Jossey-Bass Publishers, 1995.

Garet MS, Porter AC, Desimone, et al. What makes professional development effective? Results from a national sample of teachers. AERA J 2001;38(4):915-945.

Haluck RS, Marshall RL, Krummel TM. Are surgical training programs ready for virtual reality? A survey of program directors in general surgery. JACS 2001;193(6):660-665.

Hekelman FP, Roberts BJ. Excellence in clinical teaching: the core of the mission. Acad Med 1996;71:738-742.

Hesketh EA, Bagnall G, Buckley EG, et al. A framework for developing excellence as a clinical educator. Med Educ 2001;35:555-564.

Kendrick SB, Simmons JMP, Richards BF, Roberge LP. Residents’ perceptions of their teachers: facilitative behaviour and the learning value of rotations. Med Educ 1993;27:55-61.

Martenson D. Learning: current knowledge and the future. Med Teach 2001;23:192-197.

McGaghie WC, Frey JJ (eds). Handbook for the Academic Physician. New York: Springer Verlag, 1986.

McLeod PJ, Berdugo G, Meagher TW. Utility of educational objectives: a study of learner and program director perceptions of their value in clinical courses. Teach and Learn in Med 1998;10(3):152-157.

Menges RJ, Weimer ME (eds). Teaching on Solid Ground: Using Scholarship to Improve Practice. San Francisco: Jossey-Bass, 1996;1-405.

Merriam SB, Caffarella RS. Learning in Adulthood: A Comprehensive Guide (2^nd ed). San Francisco: Jossey-Bass Publishers, 1998;1-400.

Patterson CJ, Eaton WH, Williams HT, et al. A continuing medical education strategy for care of the elderly by the surgical specialties. Can J Surg 1995;38(5):427-431.

Probst JC, Baxley EG, Schell BJ, et al. Organizational environment and perceptions of teaching quality in seven South Carolina family medicine residency programs. Acad Med 1998;73:887-893.

Schwenk TL, Whitman N. Residents As Teachers: A Guide To Educational Practice (2^nd ed). Salt Lake City, UT: University of Utah School of Medicine, 1993.

Regehr G, Rajaratanam K. Models of learning: implications for teaching students and residents. In: Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000;51-55.

Veldenz HC, Dovgan PS, Schinco MS, Tepas JJ, III. M and M conference: enhancing delivery of surgical residency curricula. Current Surgery 2001;58(6):580-582.

Walt AJ. The uniqueness of American surgical education and its preservation. Bull Amer Coll Surg 1994;79(12):8-20.

Weeks WB, Robinson JL, Brooks WB, Batalden PB. Using early clinical experiences to integrate quality-improvement learning into medical education. Acad Med 2000;75:81-84.

Westberg J, Jason H. Collaborative Clinical Education: The Foundation of Effective Health Care. New York: Springer Publishing Company, 1993.

Whitman N, Schwenk TL. The Physician as Teacher (2^nd ed). Salt Lake City: Whitman Associates, 1997.

Wilkerson L, Irby DM. Strategies for improving teaching practices: a comprehensive approach to faculty development. Acad Med 1998;73:387-396.

This section will highlight several aspects of assessment and evaluation that can be utilized in conjunction with the goals, competencies, and objectives included in the Surgical Resident Curriculum. Topics included are: assessment, evaluation, and instrumentation. A helpful introduction and reference for the discussion in this section can be found at the following Accreditation Council for Graduate Medical Education (ACGME) website location: (http://www.acgme.org/outcome/project/glossary2.asp).

Over the past several years, surgical educators have spoken more frequently about the importance of outcomes assessment in their residency programs. These discussions and the resulting actions are, at least in part, a result of the accountability movement that finds most health care professionals being asked by governing and accrediting bodies to provide measurable evidence of acceptable patient outcomes to the public, consumers, legislators, and their peers. At the same time, the assessment of quality in education and acceptability of educational outcomes has become a standard part of surgical education. Competency-based education creates an educational back-up system of knowledge, skills, and attitudes that are helpful in assuring the public that a program graduate is competent to practice, that is that he or she is fit to engage in the professional activities that define being a surgeon.

The ACGME Outcome Project to enhance residency education through outcomes assessment has motivated surgical educators to consider resident outcomes at the top of their lists of program planning and assessment activities. This is a good thing to do, and it is a necessary thing to do because the ACGME will incorporate selected general elements, also known as general competencies, into the requirements of all residencies, beginning in July, 2002. Specific knowledge, skills, and attitudes in six prescribed areas are to be developed and taught and then experienced and practiced by resident learners. The six organizational areas delineated by ACGME are patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.

Program faculty must observe and assess, or measure, resident-demonstrated achievements in order to determine the educational outcomes of their respective residency program. Residents’ achievement of competency-based learning objectives is evidence of their meeting the requirements for program completion.

Historically, the field of assessment has used multiple measures and observers to report cognitive, performance, and personal growth in learners, longitudinally. Assessment refers to the instruments and procedures used to gauge such things as an individual's competence for professional practice. There are at least three considerations to make when employing an assessment process.

1. The first requires obtaining information about the attainment of the academic endeavor under consideration. Documentation of learner experience is an important part of learner assessment. Data sources can include in-training examinations, content analysis of resident conference presentations, direct observation of manipulation of laboratory models, performance evaluations such as OSCE’s, summaries of responses to exit surveys on beliefs, and composite scores on computerized professional practice simulations.

2. The next consideration in the assessment process requires analyzing the data, often through comparison with existing data sources.

3. Finally, the process entails summarizing findings and interpretations in order to prepare a report of data for use in the appropriate evaluation. An evaluation can then be used to justify continuation of an academic endeavor, improve an existing endeavor, or call for a plan to institute change.

 

 

The utilization of this kind of assessment process can help to determine if the desired competencies have been achieved, thus documenting educational effectiveness or calling for and directing change. Outcomes assessment asks whether the academic endeavor does what it says it does, suggesting a comparison of goals and objectives with observable, measurable outcomes.

A plan for assessment includes specifying criteria known as outcomes measures. These measures are determined by competency-based objectives. Objectives specify the kind of content that will be sampled in order to judge resident behavior. The Surgical Resident Curriculum is formatted so that users can develop written examinations, structure oral questions, observe work progression, and analyze presentation coverage using the knowledge and performance objectives presented in each curricular unit as a basis for determining acceptable outcome variables. When one has adequately sampled resident performance using various assessment measures, one should have the data needed to determine whether expected outcomes have been achieved.

Outcomes assessment should involve multiple judges or raters working in a coordinated effort. The judges can come from faculty, peers, other practitioners, accrediting bodies, allied health professionals, students, patients, and community representatives. Pre- and post- test tools and the improvement observed in knowledge, skills, and/or attitudes can indicate progressive learning and support a program's efforts to show curricular effectiveness by relating resident competence to curricular offerings. Change scores may be a good index of value added (i.e., the documentation of positive change longitudinally, thus reinforcing the benefit the educational endeavor has had for the learner).

A major constraint of a broad assessment using multiple raters can be the lack of agreement, or interrater reliability, especially when there is not a single acceptable standard of performance. Outcomes assessment can have increased reliability when rater training and standardized collection instruments and methods are employed. Assessment is also improved through the use of such techniques as lifelike problem scenarios, OSCE’s, role-playing exercises, or computer simulations because these techniques can be better controlled and acceptably standardized. And they help to solve the assessment shortcomings of examining content rather than process; of examining knowledge and technique acquisition rather than ability to perform.

The purpose of examining outcomes is to enable the measurement of institutional goals achievement, to make recommendations for program improvement, and to assure to the public and the profession the quality of graduates according to the program's established standards. The standards are expressed as competencies and are defined by the program's educational objectives.

The ACGME has recently emphasized the necessity for residency programs to demonstrate an effective plan for assessing resident performance using dependable measures, incorporating timely performance feedback to residents, and utilizing evaluation results to improve resident performance and the quality of the residency program.

Educational evaluation is involved with determining the worth, value, or acceptability of such processes or products as: a technique, a resident's performance, a course, or a curriculum. Evaluation relates to assessment in that evaluation employs standards using selected assessment methods. Consider this example: the method of assessment is the certifying examination (as a tool or instrument), while the evaluation uses a standard set at 75 as the lowest acceptable score (for passing the exam). Evaluation is a way to provide feedback when evaluative efforts are grounded in competency-based objectives which have been accepted as the standard of performance.

Evaluation of competence should occur at prescribed times during the residency. Formative evaluation of competence allows more immediate feedback and the opportunity for remediation, while summative evaluation occurs at the end of training, or a training period, and allows the resident less opportunity to prove his/her worth. Surgeons have participated for years in the American Board of Surgery qualifying and certifying examinations and re-certification process, indicating that sequential professional competency evaluation is a fact of life. However, there is no gold standard for evaluation of a surgeon's competence. Therefore, disagreements can certainly occur regarding the acceptability of resident performance. This is even more reason for there being a defined set of expectations for residents--a curriculum defining program requirements.

Frequently, evaluation of knowledge occurs through use of objective measures such as multiple choice or true-false examinations. Evaluation of skills often uses direct observation, including an examination of the process as well as the final product. Evaluation of attitudes can use listening and observation by a peer or preceptor, or it can be accomplished through some mechanism of self-assessment.

Evaluation of a program can and should be a multifaceted process, entailing such procedures as: curriculum review; self-study; rating of the educational environment; summaries of resident performance on examinations, clinical rotations, research volume and quality, and conference presentations; faculty aggregate data regarding publications, receipt of grant monies, visiting professorships, and teaching awards; and examinations of patient/resident/physician outcomes. Multiple perspectives should be incorporated into summary evaluation reports. The accreditation process is a multifaceted evaluation of a residency program.

Evaluation of teaching includes reviews of methods as well as instructor characteristics. Methods and characteristics that facilitate learning and show improvement over time can be isolated and used as the focus of faculty development and dossier documentation. Medical students, residents, trained educators, peers, and one’s self contribute the multiple sources of evaluative data necessary to complete the portrayal of an individual's success as a teacher. No single approach is adequate for evaluating teaching. The selected approaches to evaluation should be flexible and multifaceted. Teaching skills are usually enhanced by instruction, including feedback and self-evaluation. The evaluation of teaching can also consider the outcomes of formal education such as test and board scores.

A critical yet often overlooked aspect of educational evaluation is meta evaluation. The meta evaluation process entails consideration of a program's multiple evaluations and evaluation reports—an evaluation of one’s evaluations. The process analyzes such questions as: How useful is our evaluation system? Is our system feasible? Are our evaluation methodologies valid and reliable? Are our evaluation methodologies ethical? How can we rectify our evaluation system shortcomings?

The reader is referred to the ACGME Website for a compendium of instrument categories and content suggestions that will be useful in preparing tools to facilitate the assessment process.

Just as much attention is needed to develop appropriate data-gathering assessment instruments as is needed to identify data that are valid for evaluating a program or an individual. The utilization of selected instruments most often occurs at the end of an educational process, as a final examination or a final rating of subspecialty rotation achievement, providing summative program or individual information. Summative evaluation relates directly to goals and objectives; it is a terminal form of judgment. However, formative evaluation is part of the development process. It provides feedback to enhance learning and to improve performance prior to the endpoint of the educational endeavor. Feedback is perhaps the most critical part of an evaluation system because it can give information to residents about their performance rating, help faculty compare learning objectives with what was learned, indicate the need to change rotation goals, and provide constructive criticism and praise for more junior residents and faculty. Educational evaluation, through sensitive instrumentation, provides an ongoing source of feedback for change.

Another piece of what is often an assessment puzzle requires reaching program faculty consensus on the kinds of procedures, measures, and variables to be utilized. Unit authors of the Surgical Resident Curriculum were asked to contribute their suggestions for outcomes assessment and program evaluation in surgical education. Selected responses follow:

· Present assessment mechanisms to residents in advance of their course of study.

· Utilize attendings and more senior residents familiar with the resident's work when assessing residents.

· Record assessment of resident performance in individual faculty reports on the progress of each resident; then conveniently place the assessment on a performance form completed at the end of each service rotation. Two or more faculty should submit this same form to the program director.

· Monitor specific resident performance with grade sheets.

Provide face-to-face meetings between the program director and resident at least two times each year for performance summary, troubleshooting, and encouragement.

Assessment is the responsibility of each faculty member or attending. The level of capability of each surgical resident with whom the attending works must be adequately documented.

· Assess the morning report (e.g., the resident's ability to present patients and treatment plans).

· Utilize a checklist of the procedures accomplished by the resident, providing a running account of the resident's operative experience and allowing for recognition of deficiencies in time for catch-up to occur. Print out operative log summaries for precise assessment of minimal numbers.

· Assess physician orders, progress notes in charts, and laboratory report summaries.

· Assess daily progress notes and consultations to find a recommended course of action compatible with established standards of care.

· Prepare a content review of the resident's oral presentations.

· Utilize interactive lectures with faculty using case studies (and including radiographs and laboratory reports) as educational tools; then administer pre- and post- tests.

· Determine psychomotor competency as it is reflected in laboratory skills exercises, operative technique, successful performance of basic bedside procedures, and quality of assistance during complex operative procedures.

· Prepare mock oral practice examinations similar to the certifying examination for the ABS.

· Create monthly basic science examinations with oral examinations; compare results to previous years scoring on the ABSITE.

· Utilize ABSITE/SESAP performance comparisons each year. These can serve a dual purpose of monitoring and evaluating on an ongoing basis.

· Incorporate suggestions from expert, outside reviewers and from the surgical literature.

 

· Prepare separate pre- and post- tests developed and administered to residents as evaluative and educational tools. These can be compared, in total, to previous residents' performance regarding short-term learning increases and then compared to ABS board examination for noting long-term increases.

· Use a resident-directed survey to identify and correct training weaknesses. Chief residents can prepare these.

· Use alumni surveys to assess the adequacy of training relative to current alumni practice.

· Assess more than resident test scores. Look also at the assessment of methods used for teaching.

· Emphasize self-awareness and improvement, not just punishing the weak and rewarding the effective.

Prepare a program effectiveness survey, utilizing the competencies listed in Section 1.2.

 

The Surgical Resident Curriculum can be useful in defining progressive knowledge and performance for resident academic and skills achievement at defined levels. Generally, the more specific the educational objectives, the better it is for program definition and assessment. Objectives are constructed so that they can describe measurable behaviors. The curriculum provides goals, unit objectives, and professional competencies for knowledge, performance, and attitudes. With these activities defined, program and self-evaluation and targeted feedback can occur more systematically.

Utilizing the curriculum resources, those who would construct an assessment of educational outcomes and a program evaluation could well ask these basic questions:

· What is expected of all residents? Do the goals and objectives reflect what our program plans for residents to achieve?

· Are there specific knowledge and skill achievements expected of all graduates? This could include passing boards, establishing practice, or serving mankind.

· Are there available measures that will indicate growth when assessing attitudes and values? Assessment tools can include documentation of observations, self-report inventories and questionnaires, and locally prepared interview protocols.

· Does our program have alumni profiles to demonstrate previous resident/graduate achievements? These can provide comparative data and establish the basis for longitudinal program research.

· Does our faculty require a series of structured development opportunities in order to maximize their effectiveness in implementing our curricular assessment and evaluation plans? Development experts may become a necessary addition to faculty preparation procedures, including determination of inter- and intra- rater reliability.

Airasian PW. Classroom Assessment (3^rd ed). New York: McGraw-Hill, 1997.

Albanese M. Rating educational quality: factors in the erosion of professional standards. Acad Med 1999;74(6):652-658.

Batalden PB, Nelson EC, Roberts JS. Linking outcomes measurement to continual improvement: the serial "V" way of thinking about improving clinical care. Jt Comm J Qual Improv 1994;20:167-180.

Cox SS, Swanson MS. Identification of teaching excellence in operating room and clinic settings. Am J Surg 2002; 183(3):251-255.

Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000;343pp.

Faulkner H, Regehr G, Martin J, et al. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med 1996;71(12):1363-1365.

Giardino AP, Giardino ER, Mac Laren CF, Burg FD. Managing change: a case study of implementing change in a clinical evaluation system. Teach and Learn in Med 1994;6(3):149-153.

Gordon MJ. Cutting the Gordian Knot: a two-part approach to the evaluation and professional development of residents. Acad Med 1997;72(10):876-880.

Henerson ME, Morris LL, Fitz-Gibbon CT. How to Measure Attitudes. Newbury Park, CA: Sage Publications, 1987; 185pp.

Leibrandt TJ, Kukora JS, Dent TL. Integrating educational objectives and the evaluation process in a general surgery residency program. Acad Med 2001;76(7):748-752.

Madaus GF, Scriven MS, Stufflebeam DL (eds). Evaluation Models: Viewpoints on Educational and Human Services Evaluation. Boston: Kluwer-Nijhoff Publishing, 1985.

Marks I. Overcoming obstacles to routine outcome measurement: the nuts and bolts of implementing clinical audit. B J Psych 1998;173:281-186.

McGaghie WC, Frey JJ (eds). Handbook for the Academic Physician. New York: Springer Verlag, 1986.

Miflin BM, Campbell CB, Price DA. A conceptual framework to guide the development of self-directed, lifelong learning in problem-based medical curricula. Med Educ 2000;34:299-306.

Newble D, Dawson B, Dauphinee D, et al. Guidelines for assessing clinical competence. Teach and Learn in Med 1994;6(3):213-220.

Palomba CA, Banta TW. Assessment Essentials: Planning, Implementing, Improving. San Francisco: Jossey-Bass Publishers, 1999;1-300.

Reisdorff EJ, Hayes OW, Carlson DJ, Walker GL. Assessing new general competencies for resident education: a model from an emergency medicine program. Acad Med 2001;76:753-757.

Reznick RK. Teaching and testing technical skills. Am J Surg 1993;165:358-361.

Reznick RK, Rajaratanam K. Performance-based assessment. In: Distlehorst LH, Dunnington GL, Folse JR (eds). Teaching and Learning in Medical and Surgical Education: Lessons Learned for the 21^st Century. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2000:237-243.

Shadish WR, Cook TD, Leviton LC. Foundations of Program Evaluation: Theories of Practice. Thousand Oaks CA: Sage Publications, Inc., 1991.

Shepard LA. The role of assessment in a learning culture. Ed Researcher 2000;29:4-14.

Short JP. The importance of strong evaluation standards and procedures in training residents. Acad Med 1993;68:522-525.

Sloan DA, Donnelly MB, Zweng TN, et al. The structured clinical instruction module: a novel strategy for improving the instruction of clinical skills. J Surg Research 1995;58(6):605-610.

Swanson HL, Lussier CM. A selective synthesis of the experimental literature on dynamic assessment. Review of Ed Research 2001;71(2):321-349.

Tekian A, McGuire CH, McGaghie WC and Associates. Innovative Simulations for Assessing Professional Competence—from Paper and Pencil to Virtual Reality. Chicago: University of Illinois at Chicago, 1999;254pp.

Wade TP, Kaminski DL. Comparative evaluation of educational methods in surgical resident education. Arch Surg 1995;130:83-87.

Web reference http://www.acgme.org/outcome/project/glossary2.asp

Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct validity of a structured technical skills assessment form. Am J Surg 1994;167:423-427.

Yudowsky R. Can resident evaluations demonstrate increases in residents’ skills over time? Acad Med 1999;74(10 Suppl):S108-110.

[Sample form from East Carolina University, School of Medicine]

Response Scale: OUTSTANDING = met all or virtually all of the criteria

SATISFACTORY = met most of the criteria

MARGINAL = met some of the criteria

UNSATISFACTORY = met few or none of the criteria