Collaborate
Asks who does what in joint research, in what order the research proceeds, and how far its results can be extended.
Explore the participating rolesSee the joint research process
A SHARED RESEARCH QUESTION
Different roles, one process of verification
Participating roles
Who does what?
Joint research can be run in a way where different roles carry forward a single process of verification. It is a structure of cooperation that links scholarly review, problems in the field, and implementation and empirical validation.
The scholarly review of the Scholars’ Council and the participation of the Enterprise and the field come together in a joint research question, and the Research Institute can turn that question into a form that can be verified.
Research directions · criteria for judgment
Scholars’ Council
The Scholars’ Council establishes research directions and criteria for judgment on the basis of accumulated scholarship.
Implementation · operation
Enterprise and the field
The Enterprise implements research results as technologies, products and services, and operates them.
Research models · experiments · verification
Research Institute
The Research Institute turns scholarship into concrete research models and carries out experiments and verification.
Read more — What each role does in meetings
The meetings of the Scholars’ Council for AI can be run with separate sessions for reporting results and for reviewing their scholarly interpretation. Those in charge of experiments explain what was done and what was observed. The Scholars’ Council reviews which hypotheses the results support or weaken. The Enterprise explains what matters in actual use. If what was settled in the discussion and the questions that remain are recorded, the next study can take over those judgments.
The process
In what order does joint research proceed?
From a shared question
to evidence in practice.
From a problem to a research question
When a project begins, the problem in the field should be turned into a research question. Rather than using the inconvenience the Enterprise reported as an outcome indicator as it is, it is checked which results matter and why, and under what conditions the problem appears.
Differences between experimental results and real environments
If useful results have been obtained in an experimental environment, the differences from the real environment are examined. The completeness of the data, users’ level of skill, time constraints and changes in the work can affect the results. At first, testing can be done within a scope where results are observed or go through human review, and the scope of application can be set according to the conditions that have been confirmed.
Field validation and records
In field validation, the system’s results and the changes in the field are recorded together, and if the work has changed or users have begun to use new workarounds, those changes should also be included in the analysis. Not every difference between before and after a particular system is adopted is concluded to be an effect of the system; comparison conditions are kept where possible, and differences that could not be kept are stated as limits.
Read more — Organizing the project and designing experiments
The Scholars’ Council reviews the explanations offered by the relevant scholarship, and the Research Institute organizes the project into a form that allows observation and comparison. When there are different causal hypotheses about the same problem, experiments are designed to tell them apart.
Results and follow-up research
How far can the results be extended?
When one applied study ends, the next question is how far its results generalize. Whether a structure that showed good outcomes in a particular project is also valid with other data, other users and other models should be examined.
Read more — Results that do not fit, and the research record
Results that do not fit existing theory are also important research material. The way the original concept was mapped onto AI may have been inappropriate, or the implementation may not have met the intended conditions. If differences remain even after these problems are checked, a new explanation may be needed. A claim to have discovered a new phenomenon should be accompanied by a process of comparison with alternative explanations. AI research can also return questions to existing scholarship. Conditions that appeared together in human organizations can be separated in simulation, or a variety of information-passing structures can be compared repeatedly. Even so, the laws of human society cannot be settled from results obtained in artificial systems alone. The long-term assets of the Scholars’ Council for AI are formed by the number of papers together with a connected research record. How which source text was interpreted, what hypotheses it was turned into, and under what conditions results appeared should be linked together. Later researchers can test new conditions and offer different explanations on the basis of that record. This is why the literature, experiments and field records are managed together.
BEFORE WE BEGIN
What is to be decided first at the launch
What is to be decided first when the Scholars’ Council for AI is launched is whose difficulties, and which ones, it is to help resolve. That problem is turned into questions that can be verified, the scholarship and data needed are gathered, and participants’ contributions and the criteria for evaluation are set. After that, whether harm has decreased, whether people can understand and revise decisions, whether capability has grown, and whether a wider range of people can make use of the help are examined within the limits of the evidence.
The operation of the Scholars’ Council for AI itself should follow the same principles.
Participants’ views
When do the views of the people affected come in?
Evaluation against the social goals should include the views of the participants who are actually affected.
At the start of the research
the help expected and the burdens of concern are identified.
During validation
a channel for reporting problems and a person responsible for acting on them are put in place.
At the end
the indicators set in advance and the participants’ experience are reviewed together.