AI’s Action and Learning
Asks how AI carries out plans, learns from experience, and how far things can be entrusted to it.
What should be checked when AI makes plans and acts?
AI that acts changes the state of its environment. Which tools it can call and in which situations it should call them are different questions, and if a review is marked complete simply because a tool responded, later judgments may accept an unconfirmed state as fact. Evaluation, too, should look separately at the sentences of a plan and the actual actions, checking in turn whether a correct plan was put forward, whether the necessary actions were carried out and whether those actions produced the expected results.
Current state
Goals · evidence · unconfirmed items
Choosing an action
Authority · conditions for starting
Running a tool
Look-up · calculation · recording
Checking the result
Completion · errors · effects
Confirmed
Recording experience and conditions
In what situations it held
From checking the result back to choosing an action — choose again if anything incomplete · conflicting · erroneous remains
From recording experience and conditions back to the current state — reused in a new task after the conditions are checked
A request to execute is distinguished from actual completion. If incompletion or an error is confirmed, the action is chosen again, and confirmed experience is re-verified in other tasks with its conditions of application recorded.
Read more — From plan to execution
The Scholars’ Council for AI examines the technical possibility of an action together with the appropriateness of the judgment. AI that acts changes the state of its environment. Actions vary in kind, from reading documents or performing calculations to creating records in business systems. Which tools can be called and in which situations a tool should be called are different questions. A plan for reviewing a purchase request can consist of gathering materials, comparing specifications, checking exceptions and writing up the review results. Each step needs conditions for starting and criteria for confirming completion. If a review is marked complete simply because a tool responded, later judgments may accept an unconfirmed state as fact.
Evaluation, too, should look separately at the sentences of a plan and the actual actions. Whether a correct plan was presented, whether the necessary actions were carried out and whether those actions produced the expected results are each checked.
What does it mean for AI to learn from experience?
To build AI that does not repeat the same errors, past experience has to affect the next performance, and that effect can arise in several ways. It is useful for memory to keep the situation that occurred, the solution used, the confirmed result and the conditions for reuse together, and only when the scope in which memory is used is clear can the problem of treating old experience as present fact be reduced. The results of learning should be checked on new tasks, and getting a previously failed case right again does not mean that general ability has improved.
To use a method that worked once in another situation, the conditions that made it succeed should be checked. If a purchase review AI missed specification information in a certain document format, it can remember to check that place in the same format next time, but generalizing that the same place exists in every document produces new errors. That is why it is useful for memory to keep the situation that occurred, the method used, the confirmed result and the conditions under which it can be reused together.
The situation that occurred
The method used
The confirmed result
Conditions for reuse
Read more — AI that learns from experience
To build AI that does not repeat the same errors, past experience should affect the next performance. That effect can arise in several ways. Earlier records can be read again in the next input, business rules can be updated, or a model’s weights can be changed through a separate training process. It is useful for memory to keep the situation that occurred, the solution used, the confirmed result and the conditions for reuse together. Only when the scope in which memory is used is clear can the problem of treating old experience as present fact be reduced. The results of learning should be checked on new tasks. Getting a previously failed case right again does not mean that general ability has improved.
Looking together at what was stored and in what situations it was usefully applied is the task of continual learning.
How far should things be left to AI, and when should one intervene?
Autonomy is hard to express as a single degree. The scope in which AI looks for materials on its own, the scope in which it makes final decisions and the scope in which it changes external systems can differ from one another; in a purchase review, broad authority to look for additional documents can be given while the authority to finalize the review results is kept with people. To check the effect of an intervention, it should be compared with cases where there was no intervention.
Scope for finding materials
Scope for making final decisions
Scope for changing external systems
A 2016 study co-authored by Professor Seokhwa Yun examined, in the relationship between empowering leadership and members’ job performance, both a positive path through self-efficacy and a burdening path through job-induced tension.
Read more — How far to leave things to AI, and when to intervene
Autonomy is hard to express as a single degree. The scope in which AI looks for materials on its own, the scope in which it makes final decisions and the scope in which it changes external systems can differ from one another. In a purchase review, broad authority to search for additional documents can be given, while the authority to finalize the review results is kept with people. An intervention is an action that supports or corrects autonomous performance. It can request missing materials, flag conflicting evidence and have alternatives reviewed again. For an intervention to be useful, it has to make the change the current state needs.
A 2016 study co-authored by Professor Seokhwa Yun examined, in the relationship between empowering leadership and members’ job performance, both a positive path through self-efficacy and a burdening path through job-induced tension. When this is connected to AI, human tension or psychological burden should not simply be attributed to it. A separate hypothesis the Scholars’ Council for AI can set up concerns whether widening the range of actions that can be chosen increases search costs or opportunities for error under certain conditions. The subject of the original research and that of the new hypothesis are distinct.
To check the effect of an intervention, it should be compared with cases where there was no intervention. Cases in which additional information was requested, cases in which the review was repeated and cases in which the process continued as it was can be examined separately from the same state. Not only improvements in results but also delays and costs are recorded. Only by including situations in which intervention was unnecessary can the problem of choosing only the cases with good effects be avoided.
The purpose of control lies in making it clear who decides what. Under conditions the system cannot check, judgment is requested from people, and wrong changes are made traceable or reversible.
The Scholars’ Council for AI can build up evidence to explain which structures of autonomy and intervention are appropriate depending on the nature of a task.