Presenters: Julia Aquino, Elisabeth Schainker, Michelle Hofmann

Summary: The panel began by laying the foundation for why competency assessment of community pediatric hospitalists is important, based on the Pediatric Hospital Medicine Core Competencies published in the Journal of Hospital Medicine. Throughout the presentation, the panel used their own personal experience of developing a neonatal competency assessment program to demonstrate how to build a successful, comprehensive program for community pediatric hospitalists.

The presenters outlined the key goals of a competency assessment program, including:

• Feedback about both strengths and weaknesses should be provided in order to reinforce learned knowledge and guide future learning;

• Programs should assess not just clinical, but also non-clinical skills, such as communication, patient safety, and continuous quality improvement; and

• Assessment should include formative assessment, or “low stakes” monitoring of learning, as well as summative assessment, in which there is a “high stakes” consequence for not demonstrating competency. An example of summative assessment would be the current NRP testing: if you do not pass the test, you do not receive NRP certification.

The panel then described the need to build a competency assessment program based on the concepts of Miller’s Triangle of assessment. The bottom two layers of the triangle include standardized evaluations of a user’s knowledge, while the top two layers are comprised of practice-based assessment.

The bottom layer of the triangle assesses a user’s knowledge through test questions. The next level up tests a user’s ability to apply knowledge, for example, the current PALS case-based learning. The third level evaluates a user’s ability to show or demonstrate knowledge, such as through simulation training. The final or top level assesses the user actually doing, or performing the task, such as through direct observation.

The presenters pointed out the relative strengths and weaknesses of each level, namely, as one moves up the pyramid, the levels of assessment demonstrate increasing levels of competency, but come at the expense of increasing work or cost.

Key Takeaways

A comprehensive competency assessment program should:

(1) Evaluate clinical and non-clinical skills, based on the Pediatric Hospital Medicine Core Competencies;

(2) Incorporate both “low stakes” formative and “high stakes” summative assessment; and

(3) Include standardized assessments as well as practice-based evaluations, based on the concepts of Miller’s Triangle. TH

Dr. O’Callaghan is a clinical assistant professor of pediatrics at the University of Washington and a member of Team Hospitalist.