Work That People and AI Do Together
Asks how people and AI share judgment and authority, solve problems together, and what becomes of people’s capability in work and learning.
How should people and AI share judgment and authority?
In situations where people and AI work together, it should be decided specifically who takes on which part. Authority can be thought of separately according to the kind of action — the authority to read materials, to propose alternatives, to change records and to finalize decisions — and even if the same person or AI holds several kinds of authority, it should be possible to check which action was taken under which authority.
To evaluate whether collaboration is good, the burden of review on people should be examined along with the final result. Even if errors went down, review time that grows too much can affect other work, and even if processing time went down, it may become harder for users to understand or correct the results.
Human judgment
Goals and permitted scope
Sets the purpose of the request and the authority
Support from AI
Gathering evidence and alternatives
Checks materials and prepares options
Support from AI
Supporting judgment
Presents evidence · uncertainty · alternatives
Human judgment
Review and finalization
Revision · deferral · final choice
Execution and operation in the field
Execution and checking the state
Executes within the set scope and records
Results and effects
Human judgment
Requests for explanation and re-examination
Corrects wrong judgments and measures
From requests for explanation and re-examination back to supporting judgment — re-examination
Case
A research team led by Michelle Vaccaro at the MIT Center for Collective Intelligence in the United States studied under what conditions humans and AI can work better together. Their study published in 2024 gathered and reviewed relevant experiments according to set criteria and statistically combined the results of many experiments. This method is called ‘systematic review and meta-analysis.’ The conditions compared were humans alone, AI alone and human–AI combinations. In the studies included in the analysis, the combinations were, on average, statistically significantly lower than whichever of humans alone or AI alone performed better, and differences by type of task were observed. This is the result for a sample of studies published from 2020 to the first half of 2023. It should not be stretched into grounds for applying the same conclusion to every latest system.
Read more — How people and AI share authority
Research is needed that evaluates how people’s abilities and right to judgment are maintained in actual processes. In situations where humans and AI work together, it should be decided specifically who takes on which part. Authority can be organized according to the kind of action. The authority to read materials, the authority to propose alternatives, the authority to change records and the authority to finalize decisions can be thought of separately. The same person or AI can hold several kinds of authority, but it should be possible to check which action was taken under which authority.
The subjects of research include not only the people who buy or operate a system but also the people affected by its judgments. Distinguishing whom it helped and whose burden increased is the social meaning of sharing authority.
When a problem is defined anew, what can be compared?
When looking for creative solutions, how the problem was defined matters.
The procedure before the change
Suppose that in a purchase review there is a procedure in which a missing specification is found only after all checks have been completed.
If the order is changed
It can be compared whether changing the order so that the requested specifications are checked first reduces unnecessary follow-up work.
However, removing a review step and changing when the review happens are different interventions, so the results can be interpreted only if what was changed is recorded accurately. When creativity is evaluated, too, the number of new proposals and their usefulness should be looked at separately.
Producing many similar proposals does not mean that the range of exploration has widened. It should be reviewed whether they keep to existing constraints, whether they can actually be carried out and what side effects they produce.
A 2023 paper by Professor Namgyoo K. Park, who studies strategy and creative thinking at the College of Business Administration at Seoul National University, describes “Puzzling,” a creative thinking method that breaks a work process apart and re-examines the relationships among its parts to find opportunities for improvement. The Scholars’ Council for AI examines the research possibilities of connecting such approaches to the analysis of AI work processes.
Read more — Creative problem-solving
What matters is not applying a particular creativity technique to every problem but showing which thinking processes are useful for which tasks. In the process of looking for creative solutions, how the problem was defined matters. In the purchase review case, it can be supposed that there is a procedure in which a missing specification is found only after all checks have been completed. It can be compared whether changing the order so that the requested specifications are checked first, at an early stage, reduces unnecessary follow-up work. However, skipping a review step and changing when the review happens are different interventions. The results can be interpreted only if what was changed is recorded accurately. When creativity is evaluated, the number of new proposals and the usefulness of the proposals should be looked at separately.
What becomes of people’s capability when they work and learn with AI?
It is hard to judge the effect on people’s learning from the quality of the output alone. In organizations, it should also be checked whether other members can understand and revise procedures written by AI, and whether the grounds for the work remain when the person in charge changes.
Quality of the output
Suppose a new employee wrote a purchase review report quickly with the help of AI.
Learning that remains with people
Whether that person can afterwards solve similar problems without AI, explain the specifications and exceptions, and notice AI’s wrong suggestions has to be checked separately.
Read more — Human capability in work and learning
When AI enters work, the make-up of the activities people carry out can change. It is hard to judge the effect on human learning from the quality of the output alone. For example, suppose a new employee wrote a purchase review report quickly with the help of AI. Whether that person can afterwards solve similar problems without AI, explain the specifications and exceptions, and spot AI’s wrong suggestions has to be checked separately. In education, how to combine providing correct answers with supporting the thinking process can be examined. In organizations, the problem of knowledge becoming concentrated in particular individuals or tools also matters. It should be checked whether other members can understand and revise procedures written by AI, and whether the grounds for the work are maintained when the person in charge changes. Research on knowledge management and organizational learning offers perspectives on the transfer and reuse of knowledge in this process. The paths by which the use of AI leads to an organization’s actual capability should be observed specifically.
Research on work and education pursues both the growth of human capability and the narrowing of gaps in the use of AI.