AI’s Collaboration and Organization
Asks how several AIs divide roles, judge together with scattered information, and coordinate one another’s work.
How should roles be divided among several AIs?
Giving several AIs different names does not by itself create a structure for collaboration. With a central coordinator, information is easier to gather, but that coordinator’s omissions can affect the whole; in a distributed structure, varied review is possible, but the cost of bringing opinions together has to be dealt with separately.
The research of Professor Won-Woo Park, who studies organizational behavior and teamwork at the College of Business Administration at Seoul National University, is part of the scholarly foundation that can be consulted when examining roles and team performance. When this is applied to AI, whether a role was actually carried out should be turned into observable criteria.
Six items for checking roles in AI collaboration
| Criterion | Question to check | Examples of observational data |
|---|---|---|
| Definition | Has it been decided what the role is responsible for? | Role description and the scope of inputs and outputs |
| Assignment | Who took on the role? | Assignment records and the AI or person in charge |
| Operational availability | Are the conditions in place for it to be carried out when needed? | Access rights to materials, tools, readiness to run |
| Contribution | What did it add to the actual judgment and result? | Information passed on and changes in the final judgment |
| Evidence | What explains that contribution? | Links to source texts, execution records and sources |
| Verification | Has it been confirmed that it was carried out appropriately? | Independent review and checking of results |
These are research criteria constructed for AI collaboration. Their validity is verified separately from the original scales of existing team role theory.
Read more — Roles and organizational structure
The strengths and weaknesses of a structure are a matter to be tested according to the task and the conditions of information. Giving several AIs different names does not by itself create a structure for collaboration. It should be decided what information each role deals with, what results it passes on and who uses those results. The relationship among inputs, responsibilities and outputs matters more than the names of roles. With a central coordinator, information is easier to gather, but that coordinator’s omissions can affect the whole. In a distributed structure, varied review is possible, but the cost of bringing opinions together has to be dealt with separately. In the initial research on diagnosing roles, the definition, assignment, operational availability, contribution, evidence and verification of a role are checked separately. These six items are research criteria constructed for AI collaboration, and their validity is verified separately from the original scales of existing team role theory.
In research, it is better to compare variables separately than to change the number of members and the organizational structure at the same time.
How can judgments be made together when information is scattered?
In collaborative tasks, the information needed is divided among several participants. Even if each piece of information is accurate enough, problems arise if they are not considered together in the final judgment. Observing separately where information is passed on, where it is understood, and where it is applied and verified makes it possible to explain more specifically where a failure occurred.
When opinions clash, information worth reviewing can come to light, but having dissent raised at every step can repeat problems that are already settled or increase groundless doubt. To evaluate the effect, it should be checked what new evidence the dissent provided, whether it corrected errors and how much time and resources it took.
Professor Won-Woo Park’s 2000 study empirically examined, in human groups, the variables that make up Janis’s groupthink model. To connect this research to AI, observable phenomena should be defined without assuming that AI has the same social pressures or emotions as humans. For example, it can be examined whether early opinions are cited repeatedly, whether opposing evidence is actually reviewed and whether conclusions that could still be revised are settled too early. Here, diversity of opinion and diversity of evidence should be kept distinct.
Holding
Who knows what?
Check source texts and sources
Passing on
Did it reach the role that needs it?
Check records of sending and receiving
Interpretation
Were the conditions and exceptions understood?
Compare interpretations of terms and conditions
Reflected in judgment
Was it used as grounds for the conclusion?
Check the reasons for including or excluding it
Read more — Judging together with scattered information
Research on group judgment should examine the gap between holding information and actually using it. In collaborative tasks, the information needed is divided among several participants. Even if each piece of information is accurate enough, problems arise if they are not considered together in the final judgment. Observing the points of passing on, understanding, application and verification separately makes it possible to explain more specifically where a failure lies.
How should the work of several AIs be coordinated?
To study collective intelligence is to ask what abilities appear when members work together. Because how a shared goal connects with role-by-role evaluation has to be coordinated, both the output of each role and the success of the whole task are looked at when a collaborative system is evaluated. The relationship between structure and conditions should be examined without assuming that the most complex organization is always the best.
Case
Anita Williams Woolley is an organizational psychologist at Carnegie Mellon University in the United States who studies team collaboration and collective intelligence. The study published by Woolley and colleagues in 2010 examined a group-level ability factor that appears in common when people carry out several tasks together. This result provides a reason to treat a group’s performance as a separate object of analysis. Whether the relationships observed in human groups hold as they are in groups of AI should be verified again.
Read more — Coordinating AIs that work together
The Scholars’ Council for AI treats dependencies between roles and the coordination of shared goals as core problems of cooperation. To study collective intelligence is to ask what abilities appear when members perform together. In AI collaboration, a coordinator can divide tasks, gather information and choose the next action. This function can be concentrated in a single strong model or distributed across several roles. Whichever approach is chosen, the evidence the coordinator referred to and the reasons for its decisions should be open to review from outside. It also matters whether there is a path for correction when another participant finds an error in coordination. In a purchase review, if the AI that gathers materials concentrates only on collecting many documents, review time can grow excessively. If the summarizing AI concentrates only on reducing length, important exceptions can disappear. How a shared goal connects with role-by-role evaluation has to be coordinated. This is why both the output of each role and the success of the whole task are looked at in evaluating a collaborative system.
The relationship between structure and conditions should be examined without assuming that the most complex organization is always the best.