- Identifies and analyzes learning and performance problems.
- Designs, plans, and develops instructional interventions using appropriate strategies and techniques.
- Develops an evaluation plan for a project based on stated goals and recognized standards.
Return to RB Purdue LDT Portfolio Homepage
Artifacts: EDCI 577 E-Learning Module Evaluation Results Fall 2016
EDCI 566 Digital Storytelling Spring 2017
EDCI 566 Digital Job Aid Spring 2017
Narrative:
My evaluation of a module that I built during the previous summer provides an effective venue for highlighting my application of instructional design principles. The focus of this particular project was to effectively apply Kirkpatrick's 4 Levels of Evaluation (Kirkpatrick & Kirkpatrick, 2006) by developing an evaluation plan for a project based on a projected need driven by customer goals and state standards. The customer requests were to provide a digital, basic finances module which improved student financial understanding while requiring limited in-class administration. The key customer metric was retention beyond the lesson exit assessment.
Learning and performance problems were identified and analyzed with an intent to further address these challenges via targeted website support options during the module instruction and as a part of Level 2 evaluation. "Scores on the post-test should exceed 75%. Students scoring below 75% were directed to instructor-designed intervention tools and may be subject to retesting. The...responses...of these students were analyzed...as a group to identify any elements of the study materials that might be hindering their progress." The planned intervention elements are learning-style specific websites providing training and practice on the finance topic.
The premise was to create a scenario whereby an organization contracted us to perform an evaluation of a learning program. We then design, plan, and develop instructional interventions using appropriate strategies and techniques for that scenario. Next we projected implementation and resultant findings, and reported the data analysis results to the contracting organization. As I evaluated a module that I had previously designed, I was also able to highlight the instructional design strategies used in the course through the evaluation lens. Below are the instructional activities involved with varying venues:
"The short course consists of numerous media selections:
- Two slideshows addressing budgeting basics,
- An online budgeting game,
- A guided practice on investment calculations,
- A set of practice problems about investment calculations,
- A practice quiz using investment knowledge and budgeting skills,
- An online test covering investment calculation skills,
- An artifact production exercise using the investment knowledge and skills acquired in this lesson."
As part of the evaluation activity I included a simulated question where identifying and analyzing a potential performance problem yielded the need for artifact adjustment. "The question with the lowest response rate was analyzed by the instructors and found to have language that would be confusing to their particular demographic based on how ‘biweekly’ is interpreted in the Allen, Texas area. ‘Biweekly’ is considered twice a week, not every two weeks."
This study of evaluation applications drove key adjustments to my module and reshaped my views regarding the construction of future models. As I venture into on-line course development I will be better attuned to viewing my course from the structured evaluation perspective. I am considering constructing the evaluation artifacts near the beginning of a project and then ensuring that I design the project elements to drive success in those evaluations. We often do that with the Level 2 artifacts (pre-test/post-test), but incorporating the goals of the other levels might create a module that sings.
The educational applications of multimedia course (EDCI 566) in the spring of 2017 allowed me to explore the world of creating digital artifacts with significant review of research to guide me to the most efficient presentation of learning. This is an area of passion for me and one that I expect to contribute to during the next two decades.
Kirkpatrick, D. L. & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). San Francisco: Berrett-Koehler Publishers, Inc.