AI Does The Heavy Lifting

By Jerry D. Fife, M.Ed.  |  ProTect Athletics  |  September 25, 2026

The National Athletic Trainers’ Association certification exam used to have three parts: a written simulation, an oral practical, and a multiple-choice section.

Passed the first two without difficulty. Clinical competence was not the issue.

Missed the multiple-choice section by one question. One question ended a path to becoming a certified athletic trainer.

What stayed was not the disappointment. It was the design of the exam itself. The two components that demonstrated clinical competence were open-text and performance-based: scenarios, judgment calls, real-world reasoning under conditions that looked like actual practice. The component that closed the door was a test of whether you could identify the one answer the author decided was correct.

Those are not the same skill. In high-stakes safety credentialing, they are not measuring the same thing.

That realization eventually became the design principle behind everything ProTect Athletics built.

The Problem with Multiple Choice

A standard multiple-choice exam measures two things well: recall and recognition. The learner sees a question, matches it to something they memorized, and selects the answer that corresponds. That is a legitimate assessment for some purposes.

It is not a legitimate assessment for a safety professional making real-time decisions on a sideline.

The person who needs to identify an emergency, communicate a referral without overstepping their clinical authority, document the incident correctly, and manage the situation until qualified personnel arrive is not operating in a multiple-choice environment. They are in an open-ended, ambiguous, high-pressure context where the right action is not printed on a card. Their ability to perform in that context is what a safety credential is supposed to verify.

Research bears this out. The National Safety Council identifies human behavior as a contributing factor in 80 to 95 percent of workplace incidents (NSC, 2021). Knowledge is necessary but not sufficient. The gap between knowing a protocol and executing it under pressure is exactly where incidents happen. A test that measures only knowledge cannot close that gap.

We built the DAS assessment design around open-text responses: incident reports, scenario analyses, reflective assessments. When a learner writes about what they would do in a situation, they cannot hide behind answer choices. They have to think, organize, and express. And that writing tells us something that a circled letter cannot.

What We Learned from How People Write

Reading those open-text responses over time, a pattern became clear. Different learners express their knowledge differently.

A learner with strong logical reasoning structures their incident report like a chain of cause and effect. A learner who processes interpersonally describes the social dynamics of the situation before anything else. A kinesthetic processor describes the physical sequence of their actions in precise detail. An intrapersonal thinker reflects on their own decision-making process before describing what happened externally.

These are not random differences. They map to established research on how people process and express information. Howard Gardner’s foundational work on multiple intelligences identified seven distinct intelligence types: linguistic, logical-mathematical, spatial, musical, bodily-kinesthetic, interpersonal, and intrapersonal. The DAS cognitive framework draws from those dimensions, adapted for instructional application in professional safety training. The practical application for adult learning is not to restrict instruction to a single modality. Adults bring prior experience and established cognitive patterns to every learning encounter. Knowles documented this in his foundational andragogy research. Meeting the learner where those patterns live is good instructional design.

That recognition became the foundation of Dynamic Cognitive Profiling.

The Problem with Handing It to the Machine

ProTect Athletics runs a professional safety credentialing program called the Director of Athletic Safety, or DAS. The people who earn this credential are responsible for identifying hazards, managing emergency action plans, and making real-time safety decisions on behalf of student-athletes. These are not simulations. These are sideline decisions, and the consequences of getting them wrong are real.

That means the training system has to work. Not in the compliance sense, where a certificate gets printed and a box gets checked. In the actual sense, where the person on the sideline knows what to do when something goes wrong.

The instinct in EdTech right now is to automate everything you can. Quiz grading. Feedback delivery. Coaching recommendations. Content personalization. Hand it to the machine, the machine handles it, and you call that innovation. In most industries, that is exactly the right call.

In high-stakes safety training, it is the wrong one.

We use AI throughout our credentialing system. But every coaching recommendation that system generates gets reviewed by a human instructor before it reaches the learner. The human reviews it, owns it, and delivers it. The AI does the analysis and the synthesis. The instructor does the coaching.

That choice is not accidental. It is the architecture.

What We Built Instead

We built a system with four core capabilities. None of them replace the instructor. All of them make the instructor more effective.

Dynamic Cognitive Profiling

Every learner who goes through our DAS curriculum submits open-text written responses: incident reports, scenario analyses, reflective assessments. A standard LMS would run those through a rubric and return a score. We do something different.

We analyze how a learner writes, not just what they say. The word choices, the sentence structure, the way they organize information, how they describe what they observed versus what they concluded. From that analysis, we map each learner to a cognitive profile across Gardner’s seven dimensions. And then the coaching they receive is written to connect with how their brain already organizes information.

A logically-dominant learner who needs to see the cause-and-effect chain gets a different coaching response than a kinesthetically-dominant learner who needs to see the decision sequence played out in physical terms. The instructional content is the same. The communication approach is not.

This is not the same as the contested learning styles matching theory, which claims learners perform better when taught exclusively in their preferred modality. The evidence on that specific claim is mixed. What the DAS system does is different: it reads how a learner expresses themselves and adapts coaching communication to meet them where their language already lives. Adult learning research supports that approach.

To the author’s knowledge, no comparable safety credentialing platform does this from open-text analysis. Many adaptive platforms adjust content based on quiz scores and click-through rates. We are analyzing narrative writing and adapting coaching tone to match cognitive expression. The distinction matters because the goal is not to pass a quiz. The goal is to reach the actual person taking it.

Longitudinal Behavioral Tracking

Most credentialing systems track pass and fail. We track something different.

We track specific tactical errors across every module in the curriculum. If a learner consistently mishandles scope-of-practice language in their written assessments, that pattern does not disappear when they pass Module 2 and move to Module 3. It follows them. The instructor sees it, across time, across the full course.

That lets us close the loop. We can connect a coaching intervention in Module 2 to whether the tactical error appears again in Module 4. If it does not appear, we have evidence that the feedback worked. If it does, we adjust the approach.

Donald Kirkpatrick and James Kirkpatrick describe this as Level 3 evaluation: the measurement of actual behavior change, documented over time, not inferred from a post-course survey or a final exam score. Level 3 is the most meaningful indicator of training effectiveness and the least implemented. Most organizations measure Level 1 (did they like the training?) or Level 2 (did they pass the test?). Very few build systems that document whether the training actually changed what people do.

We built a system to document the behavioral patterns Level 3 requires: tracking whether specific tactical errors recur across the course and creating the longitudinal record that post-training measurement depends on.

Scope and Liability Enforcement

The scope boundary in the DAS credential is not just an instructional design decision. It is a legal line, and crossing it has consequences that go well beyond a course grade.

The DAS credential authorizes its holders to observe, assess, and refer. It does not authorize clinical diagnosis. Those are not the same thing. The line between them is precise, legally significant, and easy to cross without knowing you are crossing it.

Our system uses semantic AI to read open-text responses and flag the language. When a learner writes that an athlete “appears to show signs consistent with a concussion,” that is an observation. When they write that an athlete “has a concussion,” that is a diagnosis. The words are close. The legal implications are not.

The AI catches the distinction. It surfaces the flag. The human instructor reviews it, makes the call, and delivers the coaching. The scope boundary is defended not by a keyword filter but by a system that understands context, with a trained human in the decision seat.

To the author’s knowledge as of this writing, no comparable credentialing organization in the athletic safety space has described a similar scope-flag system in published literature.

The Human in the Loop

The fourth piece is the one that holds all the others together.

Every profile the system builds, every behavioral pattern it tracks, every scope flag it surfaces: all of it goes to a human instructor before it reaches the learner. The instructor reviews the AI’s synthesis, adjusts where professional judgment calls for it, and delivers the coaching in their own voice.

ProTect Athletics published a LinkedIn post about NOCSAE helmet certification standards. The post stated that NOCSAE does not evaluate rotational force. That claim was wrong. NOCSAE ND002 sets a peak rotational acceleration limit of 6,000 rad/s squared for high school and above. A reader with deep knowledge of the standards corrected it publicly. He was right. We acknowledged it, corrected the post, and updated the content.

That is not a failure story. That is what happens when humans are in the loop. The error surfaced, a human caught it, and a human owned the correction. The credibility held because the accountability was clear.

In a fully automated system, the error just sits there. Nobody owns it. Nobody corrects it. Nobody is responsible for it.

The human in the loop is not just a teaching philosophy. It is an accountability structure. And in high-stakes safety training, accountability is the point.

What This Is Really About

ProTect Athletics exists because the most dangerous thing in youth athletics is not a bad helmet or a wet field. It is a qualified person who is not there.

The DAS credential, the SSE curriculum, the YSO and ASM programs: all of it is built to close that gap. To put a trained, credentialed professional in the room before something goes wrong, not after.

We use AI because it makes our instructors capable of doing more. Reaching more learners. Adapting to more cognitive profiles. Catching more scope problems. Tracking more behavioral patterns than any human could manage alone. We do not use it to replace the instructor. We use it to make the instructor impossible to replace.

This started with one missed question on a certification exam: one question on the component that had the least to do with clinical competence. The system we built exists because the kids on those sidelines cannot afford for us to miss the people the format misses.

That is the system. That is what we built.

If you are running a credentialing program in any high-stakes field and you are handing the coaching to the machine, take a hard look at what you are giving up. The machine does not own the outcome. Someone has to.

That someone is the point.

---

References

1. Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. Basic Books.
2. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler Publishers.
3. Knowles, M. S., Holton, E. F., & Swanson, R. A. (2005). The adult learner: The definitive classic in adult education and human resource development (6th ed.). Elsevier.
4. National Safety Council. (2021). Injury facts: Work safety. NSC Press.

---

Jerry D. Fife, M.Ed. is the COO and Co-Founder of Athletic Safety Organization dba ProTect Athletics, a 501(c)(3) nonprofit building the first OSHA-equivalent safety credentialing framework for athletics. He is the developer of the Director of Athletic Safety (DAS) credential, the Youth Safety Officer (YSO) credential, the Athletic Safety Manager (ASM) credential, and the Sports Safety Essentials (SSE) compliance curriculum. Contact: jerry@protectathletics.org