Source : INDIA TODAY NEWS
Can AI be trusted? Well, according to Microsoft CEO Satya Nadella, trusting artificial intelligence blindly could be a mistake, especially as AI agents get access to sensitive information and the power to perform important tasks on their own. In a post on X titled ‘Models as Insider Risks in the Super Intelligence Era’, Nadella argues that companies need stronger safeguards to keep AI under control. Notably, throughout the post, he addresses artificial intelligence under the term ‘Super Intelligence’, a term recently popularised by US President Donald Trump.
advertisement
Why Nadella wants companies to rethink AI trust
AI is no longer limited to answering questions, writing emails or generating images. AI agents are increasingly being designed to work with company data, operate software and perform tasks with limited human intervention. While this could help businesses save time and automate work, it also raises a big question: what happens when an AI agent makes a mistake?
Unlike traditional software, where developers can often trace an error to a specific piece of code, understanding why an AI model behaves in a particular way can be difficult. Companies may not always know what led a model to produce an answer or take a particular action.
Nadella believes that relying on assurances from AI developers is not enough. As he puts it, “A model provider’s assurances do not relieve us of that responsibility.” Even if a model is designed to be safe, it could still behave unexpectedly or be compromised.
AI needs rules it cannot break
Nadella’s bigger argument is that intelligence and authority should not go hand in hand. An AI model may be smart enough to recommend a financial transaction, analyse confidential company information or change a system setting. However, it should not automatically have the power to carry out every action it recommends.
As Nadella explains, “we need to separate the supply of intelligence from the authority over it.” In simple words, AI should be able to make recommendations and perform assigned tasks, but independent systems must decide what it is allowed to access and do.
Companies should maintain records of AI actions, regularly test their systems for failures and ensure that authorised humans can stop an AI agent whenever necessary. Nadella also warns against relying on one AI model to verify another without independent checks, as this could create multiple layers of systems that fail to identify each other’s mistakes.
He wants companies to build systems that can monitor AI behaviour, restrict its permissions and contain it if it goes off track. He also calls for timely disclosure when AI systems fail or are compromised, so that others can learn from these incidents.
His final message sums up the argument: “The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least.”
– Ends
SOURCE :- TIMES OF INDIA
