Researchers warn AI could speed up its own development

More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio and OpenAI research lead Jakub Pachocki, warn that AI could soon help speed up the development of more capable AI. In a new paper, they say this could lead to an “intelligence explosion” — a rapid cycle in which AI helps build better AI, which then speeds up the process further.

The authors stress that the timing and outcome are uncertain. But they say AI systems already write most of the code at companies developing them, and that AI could automate the whole research-and-development process within years.

Why are they concerned?

If AI takes on more of the work of designing and testing new systems, progress that now takes years could happen in months, the researchers warn. They fear governments and society may struggle to respond at that pace. They also raise the possibility that people could lose control over highly capable AI, and that power could shift among companies, governments and countries.

These are risks described by the paper’s authors, not a prediction that such an outcome is certain. The article does not provide a timeline more specific than “within years,” or details about which parts of AI research would be automated first.

What do they want policymakers to do?

The researchers call for policymakers to get much better visibility into how AI research is being automated. That could help decision-makers understand how quickly the technology is changing, though the paper’s specific policy proposals are not detailed in the source.