A WEB-BASED CORE CURRICULUM TO MEET CERTIFICATION AND TRAINING NEEDS FOR MEDICAL RESIDENTS

Download PDF version:

ABSTRACT
To meet institutional requirements for resident education in a core curriculum, the UIC College of Medicine implemented an online educational program called the UIC Online Core Curriculum. The UIC residency is composed of 1,000 physicians in training in 60 programs at 18 training sites. Its size and distribution create substantial obstacles to classroom-based delivery. An online format offered a viable alternative to meet the college’s need to present uniform content, document participation, and confirm resident achievement while addressing the residents’ need for flexibility with the Internet’s anytime-anywhere availability.

Development of an online core curriculum included a focus group of program directors, residents, medical directors, and other faculty who were asked to suggest course topics that met not only the "letter" of the requirement, but also the spirit of the general competencies. These discussions resulted in a list of 13 topics, or modules. In July 1999, a pilot study involving first year UIC residents demonstrated the feasibility of the program. The following year, all UIC residents began participating. In the past two years residents from other institutions have been added to the program, resulting in a current participant base of approximately 3,800 residents nationwide.

An analysis of 2,544 anonymous end-of-module surveys demonstrates that most residents can effectively participate with minimal technical problems. Consistent with the "anytime-anywhere" approach to web-based instruction, residents complete the modules not only at home but also while at clinical and other university sites. Overall satisfaction as reported by residents is high, and the majority agrees that the material presented is useful.

Based on findings to date, the online core curriculum is an efficient and cost-effective method of providing a required program to a large, distributed population of learners while maintaining a high level of participant satisfaction.

I. INTRODUCTION

The University of Illinois at Chicago (UIC) online GME Core Curriculum was conceived in response to a rule set forth by the Accreditation Council for Graduate Medical Education requiring institutions that sponsor medical residency programs to provide a " regular review of ethical, socioeconomic, medical/legal, and cost-containment issues that affect GME and medical practice” [1]. Referred to as the core curriculum, the intent of this program is to help prepare physicians in the post-graduate training period of their career to function more competently in the modern healthcare environment. Despite this universal requirement, few institutions have published reports on development and implementation of a core curriculum [2, 3], and none has described an online approach, with the exception of a preliminary report from this department [4].

The UIC College of Medicine, with one of the largest residency programs in the United States, faced a particular logistical challenge, to present the core curriculum to 1,000 residents in 60 discrete training programs, at 18 training sites scattered through the Chicago area. Because medical residents are in fact practicing physicians, they work long hours and carry heavy patient loads. Furthermore, the physicians who supervise this program, referred to as residency directors, also carry patient loads and may not have the time to develop expertise in a variety of topics such as those encompassed in a core curriculum. The opportunities to meet with groups of residents face-to-face are rare, and attempts to present live conferences usually have only limited success. Discussions between the Department of Medical Education at the UIC College of Medicine and the Graduate Medical Education Office were spurred by a call for proposals from UI Online, a project developed in the late 1990's by the University of Illinois at Champaign-Urbana, and funded by the Sloan Foundation. Receipt of a grant from UI Online allowed planning for an online core curriculum to go forward in October 1998.

II. DEVELOPMENT

The initial step was to assemble a focus group of residency program directors, medical residents, medical directors of group practices, and key faculty to refine the definition of the core curriculum. What emerged was a list of thirteen broad topics that addressed not only the "letter of the law" but also its spirit, by including information that the group felt all residents should know by the time they finish their training (Titles of the resultant modules are listed in Table 1). Course designers were challenged to apply concepts of active learning and keep the residents' time commitment to a minimum. The planning group also agreed early on that the content presentation and discussion would be asynchronous, meaning that participants could log on and complete work at their own pace and schedule.

Development of the course material also began in early 1999. Faculty with expertise in the specific topics were drawn from throughout the University of Illinois. A single course presentation program, CourseInfo Blackboard, was selected to minimize learning time and to streamline the development process. Each topic is presented in a short course, referred to as a module, which can be completed in 3-6 hours. The modules themselves are segmented into shorter sessions that can be completed independently of one another. Each session includes at minimum a reading as well as an exercise to demonstrate the participant's grasp of the concepts. Although the core curriculum is not intended to assess mastery of the content, it was felt that completion of an exercise would help reinforce learning as well as indicate that the resident had in fact comprehended the material. Typically, the exercise is a case study that describes a situation requiring application of the session concepts in order to solve the problem. There is usually no single right or wrong answer, and residents are encouraged to respond to each other's postings in the conference board. Residency program directors (who supervise and train the residents) are also provided access to the modules, and encouraged to use the lessons as departure points for discussions on the core topics within their own specialties. Finally, every module ends with an evaluation by the resident of the content, ease of use, and applicability to their career, with a section for free-form comments. The course evaluation module in Blackboard is used to compile these responses to ensure confidentiality. A database is also maintained for the residency program directors so they can view their own residents' progress in the core curriculum. A portal, or home page, was designed to be complementary to other web sites managed by the Department of Medical Education, and a simple URL was selected, www.gme-core.org.

The pilot commenced with a single module, Medical Professionalism, which was presented to all 300 incoming UIC residents beginning in June 1999. The following year, all UIC residents were enrolled in the core curriculum. Residents are assigned to discussion groups of 15-20 to encourage collaborative learning. These groups initially included mixed groups of residents from various specialties, but later were organized by specialty to allow residency program directors to work more closely with their own trainees. Residents usually have up to three years to complete the core curriculum. While each institution establishes its own policy for completion, UIC requires residents to complete a minimum of four modules for each year of their training. Residents who fail to meet this standard are not issued graduation certificates.

At new resident orientation (an on-ground activity that occurs each June), each resident is given a brochure that describes the core curriculum and contains all the information they need to log on and get started. In addition, each resident receives a personalized business card that included the URL, an e-mail help line address, toll-free phone number, and their sign in name and password. Residents are encouraged to keep these cards with their wallets, pocket organizers, etc., so that they can log on anytime-anywhere and take advantage of down time to work on the program.

III. RESULTS

A. Evaluation of the Core Curriculum

An important aspect of presenting an online program to such a diverse and dispersed audience is the analysis of feedback from participants and the continuous quality improvement of the program. From its inception, residents have been required to complete an evaluation of every module they've completed, in order to receive credit. The evaluation consists of ten questions with Likert-scale scoring that address technical issues as well as satisfaction with the module content. Following are compilations of 2,544 replies for all modules from the programs’ inception in July 1999 through December 2002.

1. How would you rate your skills with personal computers?

2. What was your experience with the technical aspects of the core curriculum?

3. What was your experience getting help from the support staff?

4. Where did you complete the bulk of the online module?

5. How many total hours did you spend completing this module?

6. The material presented was well organized and easy to understand.

7. The questions and cases were helpful for clarifying points made in the readings.

8. I enjoyed reading the replies to the questions posted by my peers.

9. The material presented will probably be helpful to me in my professional career.

10. How much did you learn as a result of taking this module?

B. Help Desk Experience

The following table describes the breakdown in requests for assistance by residents over the same period of time:

Login ID and password problems refer to resident questions about their IDs. Enrollment refers to the opposite problem, where information on the resident has not been received by the technical support group. Progress reports and Program and Schedule likewise represent resident queries about their progress and the availability of certain modules. Assessments and forms cover primarily the end-of-module survey.

C. Program Growth

The initial aim for the online core curriculum was to provide training to residents in UIC-sponsored training programs. However, word of mouth regarding the core curriculum, combined with increased pressure by the accrediting agency, brought inquiries from other institutions regarding the availability of the core curriculum for their own residents. Between July 2000 and May 2003, 20 institutions enrolled approximately 2,800 residents in the core curriculum, in addition to the approximately 1,000 already in UIC programs. Charges to outside institutions have been kept low ($5-15/month/participant) as a result of the economies of scale available in an asynchronous program. These charges are considerably less than what a teaching hospital would incur to hire faculty to develop and present an on-ground program of this scope.

IV. DISCUSSION

The results of this report demonstrate the feasibility of a large-scale education program for healthcare professionals, using the Internet as the delivery system. Over the course of three years, the system grew from an initial pilot of 300 local participants to one that accommodates almost 4,000 learners in nine states. This growth was accomplished with minimal additional staff and faculty, reflecting the economies of scale possible in asynchronous learning networks. Evidence for the success of this program comes in large part from feedback from the participants. An important aspect of presenting an online program to such a diverse and dispersed audience is the analysis of feedback from participants and the continuous quality improvement of the program. From its inception, residents have been required to complete an evaluation of every module they've completed in order to receive credit. Resident responses to various questions are compiled by the Blackboard program and reported in aggregate. With over 2,500 module completions, responses illustrate the overall success of the program: In general, the replies are approximately 80:20 favorable to unfavorable on various aspects of the program. Note that for discussion purposes, responses identified as “agree” include both Strongly Agree and Agree, and those referring to “Disagree,” include Disagree and Strongly Disagree, are presented below.

Evaluation of the program is ongoing, but the experience of a large number of participants provides some estimate of the program’s effectiveness. The results reported in this paper can be used to address three critical questions:

Can technical barriers be overcome with a large dispersed learner group?
Does the program provide a quality experience?
Is the program cost-effective?

A. Technical barriers

Simple and trouble-free participation is a key consideration when working with a large and dispersed population. Technical problems have not been a hindrance. For instance, about 85% of the residents were able to complete the module without help from the technical support staff. Only 11% of residents have made use of the support desk, through e-mail or toll-free telephone. As seen in the help desk results, 41% of the requests are for password and login information, which is easily dispensed, while another 15% relate to omissions in rosters provided by residency program staff. Along with the queries about progress and module availability, few of the help desk contacts actually relate to technical support. Participant satisfaction with support is illustrated by the low numbers of residents who experienced “significant delays” in getting help, approximately 3%, or had to ask more than once for help (4%). Self-reporting of their computer skills indicates that most residents consider themselves “average for their peers”, albeit a high-achieving group. Only a minority considered themselves either “very strong” (25%), or “not skillful at all” (7%). The low overall numbers of help requests allow the support staff to focus on the small minority of residents who are not well versed in computer operation.

Another concern related to technical barriers is access. When the system was first introduced, some residents objected because they did not own personal computers, and therefore could not complete the work at home. In fact, one of the goals of the system was that it not significantly cut into personal time, and residents are encouraged to make use of down time during clinical or administrative duties. The diversity of venues used to complete the modules is also interesting. Almost 50% of the modules are being completed either in the resident's department office or while on clinical duty. The hope of minimizing time-starved resident use of home time was partially borne out with a 42% participant rate from home. (Ideally, the number would be even smaller).

Finally, residents were asked how long it took to complete each module. During the design phase, faculty were instructed to keep the volume of the material, reading and exercises, to a level that could be completed within 5-6 hours. 80% of the residents are completing modules within 4 hours, and only a few exceed the 6 hour limit. This is encouraging because it addresses both the technical barrier question as well as the volume of material presented, and will allow faculty to include additional material if warranted.

B. Quality of Experience

Beyond technical functionality, it’s important that residents have a positive experience and regard the content as useful. Overall, 88% of the residents agree that the modules are well organized and easy to understand. They also respond positively to the instructional design. Most residents (83%) find the questions and cases to be helpful in clarifying points made in the readings, and 79% report that they enjoy reading the replies to questions posted by peers in their small groups. These results reinforce the initial aim to make the modules student-centered and interactive. On the other hand, a third of the residents report that they learned “very little” from the module. This particular result appears to be related to responses from particular modules, and may not a wholly negative response, because some of this material is already familiar to residents from their medical school days. When residents are asked whether they think that the material presented will be helpful to them in their careers, their response is 78% agreed. Again, this appears to relate to the content of individual modules.

C. Cost Effectiveness

Cost effectiveness of the online core curriculum is difficult to quantify, because no on-ground alternative exists. However, a qualitative comparison can be made. To duplicate a 3-4 hour presentation, along with small group activities, would require a team of instructors and assistants on a full time basis. With 13 modules, presented each year, the UIC College of Medicine would have to make a presentation every four weeks, to a third of the total resident population. Additional sessions would probably be necessary for those who didn’t make the regular session. Residents would have to factor in travel time and programs would have to ensure clinical coverage while the residents were attending the training sessions. Finally, printed material would have to be duplicated and distributed at each session. Without putting a price tag on this effort, it would appear to be cost-prohibitive for most large institutions to pursue an on-ground program that included the quality of faculty and interactive aspects provided by the online core curriculum.

On the other hand, in the online presentation faculty need only prepare material once for loading into the module web site. They monitor group discussions and certify resident completion, but as with the residents, these activities are performed at times and places convenient to the faculty. Support is provided by a team of three full time academic professionals and several part-time graduate students. However, it should be noted that this team also supports an active online Master’s program as well as continuing professional education programs, so their time is only partially assignable to the core curriculum.

Finally, the program’s cost effectiveness is implicit in the number of outside institutions who are participating in the online core curriculum. Each pays a small fee, $5 to $15 per month per enrolled resident, depending on program size, affiliation status, and services provided. For that fee, the residents as well as their directors have complete access to the program. The outside institutions have selected the UIC program because the cost is very attractive compared to hiring staff to develop and present a face-to-face program.

D. Faculty Satisfaction

Faculty are recruited for the core curriculum based on their expertise in given areas. Because of the online presentation, faculty can be recruited from outside the institution, and several are situated outside the city of Chicago. Technical staff work closely with them to develop the content and presentation, and scale their effort based on the faculty member’s interest and expertise in the technical part of online course development. Because of this close relationship, no formal survey of faculty satisfaction has been conducted. Anecdotally, faculty reaction to the modules has been positive, with comments noting the creativity and intelligence of the residents in their responses to questions and exercises.

Also important to note is that in order to sustain the high levels of satisfaction expressed by participants, an annual review of the modules has been instituted. Most modules have been in use for less than two years, so data are just now being accumulated to evaluate the effectiveness of the presentation. A report on each module is shared with the module's author, along with a request to review and update the content. This will ensure that information is current and relevant to residents' learning needs.

V. CONCLUSIONS

The UIC online core curriculum demonstrates the feasibility of presenting an ongoing asynchronous educational program to a diverse and distributed learner base. With careful planning and strong user support, technical barriers are eliminated for all but a few participants. Paying attention to good online teaching practices helps ensure a positive experience. Nation-wide, institutions are struggling to meet new educational requirements for their physicians-in-training. Technology-mediated delivery provides access to participants whose clinical responsibilities make classroom meetings impractical.

VI. REFERENCES

1. Graduate Medical Education Directory, 1998-1999, p. 29.
2. Collins, J., Alderson, PO, Amsel, S. GME Core Curriculum: A Pilot Program in Radiology. Acad. Med. Vol. 75, p.547, 2000.
3. Medio FJ, Arana GW, McCurdy L. Implementation of a College-wide GME Core Curriculum. Acad. Med. Vol. 76, pp. 331-336, 2001.
4. Sandlow LJ, York JW, Hammett WH. Development of a Web-based GME Core Curriculum. Acad Med. Vol. 75 p. 137, 2000.
   

VII. ACKNOWLEDGEMENTS

The authors acknowledge the generous support of two grants from the University of Illinois (UI Online, http://www.online.uillinois.edu/) for development of the online core curriculum as well as creation of a technical support team.

VIII. ABOUT THE AUTHORS

Joseph York, PhD, is Project Director for the UIC Online GME Core Curriculum, as well as Associate Dean for Educational Administration for the University of Illinois at Chicago College of Medicine.

Gerald Stapleton, MS, is Associate Director for Online Learning in the Department of Medical Education, University of Illinois at Chicago College of Medicine. In addition to the online core curriculum, Mr. Stapleton oversees the delivery of the online Master of Health Professions Education program (www.mhpe-online.org) as well as Continuing Medical Education Online (www.cme-online.org).

Leslie J. Sandlow, MD, is Professor and Head of the Department of Medical Education, University of Illinois at Chicago College of Medicine, as well as Senior Associate Dean for the College.