General Manager
An AI character can be convincing and still give a learning system the wrong evidence. If the character invents a fact, offers the answer or misattributes a statement, the resulting conversation may look successful while the assessment becomes difficult to trust.
I want AI to serve the learning design in Practice-Native Learning, the methodology we are developing at Altaius. That means defining what each component may do and what must remain inspectable.
A character represents a person or role inside the situation. A coach helps the learner reflect or prepare. An evaluator interprets eligible evidence against criteria. These roles can share infrastructure, but their responsibilities should remain distinct.
If a coach suggests the decisive action and the evaluator later treats it as independent performance, the record overstates what the learner demonstrated. If a character's description of the learner is accepted as fact, the system may score a claim rather than an action.
I would document the inputs each role may use, the outputs it may produce and the review process when those boundaries fail.
NIST's AI Risk Management Framework 1.0, published in 2023, organises its Core around four functions: Govern, Map, Measure and Manage. They are connected activities, not a four-step certification checklist. NIST currently notes that a revision is in progress. I use the published 1.0 framework here as a reference, not as a claim of Altaius certification. [1]
For a learning system, my application would include defining permitted uses, mapping the risks of inaccurate feedback, measuring behaviour on relevant cases and managing failures after release. The exact controls need to reflect the system and its consequences.
Before relying on an AI-supported scenario, I would include these illustrative tests:
These are proposed tests. Listing them does not mean the full shared engine has passed them or that they exhaust the risks.
The same learner response may receive different feedback after a model or rule update. Without the relevant versions, a reviewer can struggle to determine whether the learner changed or the system changed.
I would retain enough information to reconstruct the conditions of a result, while applying an appropriate data-retention and access policy. More stored conversation data is not automatically better. Collection should serve the agreed learning and evaluation purpose.
The programme also needs a named person or team that can respond when the system fails. A flag with no reviewer is not an effective review process.
Altaius's Simulation Intelligence Engine remains in development with partial capability. The intended architecture and these acceptance questions should not be read as a statement that a general-purpose engine or all controls are already released.
Practice-Native Learning does not require AI for every interaction. Where a facilitated exercise or a simpler mechanism provides better control for the intended task, it can be the appropriate choice. The methodology's commitment is to meaningful practice and defensible feedback. The technology has to earn its place in that design.
Read the Altaius definition of Practice-Native Learning
Previous in this series: Practice-Native Learning in Arabic and English
