Politic?

This is a blog dedicated to a personal interpretation of political news of the day. I attempt to be as knowledgeable as possible before commenting and committing my thoughts to a day's communication.

Sunday, November 15, 2026

Controlling Accelerating SuperIntelligent AI Before It Controls Us

"[A] very benevolent, superintelligent AI [would only push humans out of the way when it was essential to accomplishing its mission]."
"But if it’s so much smarter than us, a lot of the time it just will take control away from us because that’s the way to get stuff done."
"But even if it’s not a bad actor, it may derive subgoals that cause it to want to get rid of people."  
"The whole point of regulation is not to stop people developing things, not to stop people getting rich by developing things. It’s to make sure that if you want to get rich by developing things, you develop in a direction that helps people, not hurts people." 
"If you make it more intelligent and its main concern is our well-being, then maybe we’re safer. But at present, their main concern is not our well-being. Their main concern is to achieve whatever goal you give them."
Geoffrey Hinton, Nobel Laureate, Professor Emeritus, Computer Science Department, University of Toronto  
https://www.forbes.com.au/wp-content/uploads/2026/11/DSC_8868_2a21cb-e1794373905486.jpg?w=1024
Geoffrey Hinton, the researcher widely known as the “godfather of AI,” spoke at the inaugural Hinton Lectures in Toronto. Credit: AI Safety Foundation
 
Nobel Laureate Geoffrey Hinton, speaking of the future threat that artificial intelligence could pose in its development beyond human control as it matures into what appears to be a machine self-agency organized at lightning speed, felt not so long ago that a crisis point would still be at least several years into the future. This year's AI Safety Foundation's 'Hinton Lecture' at University of Toronto, appears to have shaken Mr. Hinton off that several years'-hence deadline. What he heard and witnessed has convinced him that the pull-back period he envisioned in the near future has become nearer than he had calculated.
 
After listening to AI researcher and lecturer Paul Christiano the sparse comfort zone of dire warnings leading to more conscientious and conscious manipulation of AI's potential, pulling humanity back from the brink of no return, seems to have evaporated in the immediacy and urgency of reality. "That's terrifying. I'm going to actually tell a bunch of government leaders that on Friday, and we'll see how they react", commented Professor Hinton, when that signal lecture had concluded; one of three lectures examining the risks associated with AI behaviour, maintaining control of 'self-improving AI', along with shared safety standards in scientific oversight and international cooperation in managing those risks.
 
An American AI researcher -- founder of the Alignment Research Centre -- Paul Christiano has been deeply involved in AI safety standards, having previously worked at Open AI. Artificial Intelligence has solved some of the world's most complex math problems, when not all that long ago, in its juvenile iteration, it struggled to compose academic-grade sentences. Now, according to Mr. Christiano, fears have arisen through conjectures that AI agents may develop weapons such as killer robots, may attempt to take control of critical infrastructure, and engineer biological threats leading to pandemics.
 
https://www.forbes.com.au/wp-content/uploads/2026/11/DSC_8606-e1794384385141.jpg?w=1024
Paul Christiano, an architect of RLHF, and Geoffrey Hinton, the researcher widely known as the “godfather of AI,” spoke at the inaugural Hinton Lectures in Toronto. Credit: AI Safety Foundation.
 
"[There] is a high risk of an AI takeover within the next three years. What I mean by an AI takeover is a system situation where there are AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots in what happens on Earth, and where if, say you or me or Donald Trump, wanted to stop them, they would be as ineffective as if you or me [were] trying to stop the U.S. military."
"Many people would die during that transition, or that many people could die shortly thereafter, if we were unable to maintain a habitable environment."
"[Alternatively, a takeover could be the end of human self-determination], something we should be working incredibly hard to avoid."
Paul Christiano, AI Researcher 
In his lecture, Mr. Christiano spoke of the prospect of billions of superhuman AI systems, with "much, much more capacity than any state cyber program", veering toward possible opportunities to undermine human control. He addressed a commonly expressed opinion that to head off what seemed inevitable, address the machine by simply turning it off. Not so simple, he responds; it would be difficult to recognize whether the system at fault is 'ours or the other guy's AI' gone awry. Moreover, the fact that a lot of infrastructure runs on AI, must be addressed. 
 
Existing laws, he argued, could provide governments with control powers of intervention in the development of AI. New laws could also be passed. "You should think of them as taking steps like empowering a regulator to actually turn what are industry best practices into binding requirements". Understood is that there is a certain amount of trepidation in applying the brakes and becoming entrenched, in the process, in regulations. 
 
https://i.insider.com/6af4479fdce316f5167eabc6?width=1000&format=jpeg&auto=webp
Former OpenAI researcher Paul Christiano warns that rogue AI agents could ultimately escape human control and cause catastrophic accidents. VCG/VCG via Getty Images
 
"So I think a goal of transparency is both to understand what is happening, and then for hard scientific questions to understand that the relevant evidence is available publicly for independent review."
"[If you are] looking at how you're going to regulate this technology, you naturally fear falling behind if you slow down. [There is a] concern that if we were to impose safety requirements, it could cause us to fall second place. I think that is one of the most likely ways that we end up in an extremely bad situation."
"[AI development needs to slow down]. My view is now we should probably slow down immediately until we actually have the infrastructure to slow down later, right?"
"Until we have a high enough degree of security, governance, risk management, that we are able to detect and react to future risks as they emerge."
Paul Christiano, AI Researcher 
 

Labels: , , ,

0 Comments:

Post a Comment

<< Home

() Follow @rheytah Tweet