OpenAI has shelved the planned launch of GPT-6.1 Astra after internal tests raised concerns about deception and agents acting beyond user authorization, according to The Wall Street Journal. The company says it will investigate the issues and strengthen safety measures before releasing future models.
4 min
Copyright 2020, Dow Jones & Company, Inc. All Rights Reserved Worldwide. LEARN MORE
OpenAI says it is scrapping the release of its next-generation AI model over safety concerns that researchers raised during internal testing, in one of the clearest signs so far that agent misbehavior could stymie the industry’s rapid progression.
The move follows a summer punctuated by reports of artificial-intelligence systems industrywide going rogue, and marks a rare case of a major AI developer ditching a new release because of safety concerns.
The company had planned to launch the model, known as GPT-6.1 Astra, in the coming days or weeks, aiming for an October debut. The model was more capable than the company’s previous models in completing challenging tasks from end-to-end without human assistance, as well as writing.
The company instead will focus on improving the safety of future models, which it expects to be even more capable.
Saachi Jain, OpenAI’s head of safety systems, said in an interview that GPT-6.1 Astra regressed in two areas. Compared with its predecessor, GPT-6 Astra, the model performed poorly on tests measuring alignment, or how well the model adheres to what humans would like it to do. Specifically, GPT-6.1 Astra showed higher levels of deception: It wasn’t always honest about telling users of the actions it did or didn’t take.
Another issue was what OpenAI calls “scope authorization,” meaning that GPT-6.1 Astra would push ahead on a task without asking the user for permission, and would at times reach for external tools and services even if it might be unsafe.
“For anything regarding safety and alignment, there’s a trade off,” Jain said. “You really do need to find what’s the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction.”
While GPT-6.1 Astra improved in areas such as “model laziness,” Jain said it didn’t quite meet OpenAI’s bar for safety and alignment, so the company decided not to launch the model publicly.
The announcement comes one day ahead of OpenAI’s annual developer conference in San Francisco. In the past, OpenAI has used the conference as an opportunity to launch new models and services that reduce costs for software developers—a segment the ChatGPT-maker competes with rival AI company Anthropic to win over.
In recent weeks, OpenAI and Anthropic have called on industry partners to slow down the development of cutting-edge AI models and invest in safety standards, noting they will temper the pace of their own internal AI progress.
OpenAI says it is working to investigate a range of agent security incidents that it has discovered in recent months, and address the safety issues underneath them. As part of the work, the company has implemented a new monitoring system to catch AI-agent misbehavior more quickly, and started requiring engineers to use stronger security guardrails for testing its AI systems.
Earlier this summer hundreds of OpenAI’s internal agents, which were tasked with completing a cybersecurity test, ended up hacking into the AI company Hugging Face. Since then, high-profile organizations such as the Australian government and United Nations discovered that OpenAI’s agents used similar, but less extensive, techniques to gain access to their websites.
Many of the publicly known agent-security incidents involved OpenAI’s internal AI models that were never slated for public release.
Last week, OpenAI said it paused training on its most capable AI models after an AI agent slipped through a gap in the company’s internet restrictions to query a public chatbot. The company said its new monitoring systems flagged the incident within 15 minutes, and training on these models remains paused.
GPT-6.1 Astra isn’t one of those models, but a different case, the company said.
“We want to make sure our model development is safe no matter whether that’s in the company, or when we ship it to users,” Jain said. “But when we ship it to users, we have an extremely high bar in terms of safety and alignment.”
While the company decided not to ship GPT-6.1 Astra, it hopes to use the same base model to do additional reinforcement learning runs, and create future generations of its GPT-6 models.
OpenAI plans to conduct several deep dives to identify the root cause of the problems identified in GPT-6.1 Astra, Jain said. The work includes ensuring that OpenAI’s reinforcement learning environments are rewarding the right type of behavior, Jain added, though she noted the company would investigate all stages of model development.
AI companies have begun to draw scrutiny from policymakers and public officials, who are paying attention to the rapid development of the technology. Later this week, a Senate subcommittee is holding a hearing with third party AI researchers titled, “Rogue AI: Securing the Homeland Against AI Agent Attacks.”
Florida Attorney General James Uthmeier, a Republican, sued OpenAI in June, claiming that the company and Chief Executive Sam Altman knowingly released an unsafe product and ignored warnings that it could harm users.
In a motion for temporary injunction filed Monday, Uthmeier sought to prevent OpenAI from developing new AI models without third-party approved safeguards, stop ChatGPT from soliciting user engagement and limit the company’s ability to advertise ChatGPT as safe.
Tech companies claim they “cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government,” Uthmeier said in the filing. “The Florida Attorney General is answering your cry for help.”
An OpenAI spokeswoman said that people want to know AI is being developed safely, “and that starts with what companies like ours do ourselves.”
“Governments have an important role to play in setting robust safety standards for AI, and we’re committed to working with Florida and other states on advancing pragmatic AI policies that apply to the entire AI industry—not just one company,” she said.
The Australian leather house has opened an immersive four-day pop-up in Manhattan, unveiling its Bloom Collection and redefining what a product launch can look like.
Following the successful launch of its Palais Collection, MAISON de SABRÉ has unveiled a new modular handbag system offering more than 720 styling combinations.
Apple closed above a $5 trillion market valuation for the first time, capping Tim Cook’s 15-year tenure and setting a high bar for incoming CEO John Ternus, as the company faces growing pressure to strengthen its AI capabilities and sustain growth.
3 min
Apple AAPL 0.50%increase; became the second company to close above the $5 trillion mark on Wednesday, capping the Tim Cook era, and posing an implicit challenge to new Chief Executive John Ternus.
The iPhone-maker’s shares closed at a record, just enough to push the company’s market capitalization into territory that only chip colossus Nvidia occupies. Apple’s market value has surpassed $5 trillion six times previously during intraday trading, but it had never closed above that level, which it squeaked past on Wednesday, ending the day at $5.005 trillion according to Dow Jones Market Data.
Cook’s act will be tough to follow. During his 15-year tenure as CEO, which ended Aug. 31, Apple’s market capitalization multiplied by a factor of 13. He nearly quadrupled the number of iPhones sold annually. He grew its highly profitable services business from $10 billion annually to more than $100 billion. He launched the Apple Watch and AirPods, which together generate more than $30 billion a year. He poured Apple’s cash into more than $1 trillion worth of share buybacks and dividends.
Today, Apple shares trade at a premium compared with the overall S&P 500 even though the company’s revenue growth rate is slower than the index’s average. That is because the company’s shares are viewed as safe, and its business predictable, say analysts.
A key to Apple’s success under Cook is how it has limited capital spending, juicing the company’s returns for investors and greatly reducing the risks of its business. That has never been more apparent than during the AI build-out, when big tech rivals have rushed to spend gargantuan sums on data centers while Apple has focused on its core businesses making devices and selling high-margin services for them.
Apple investors are so confident in its business that they have driven up its shares despite big cost increases that threaten profit margins. The cost of memory and storage chips inside iPhones, Macs and iPads has skyrocketed as demand for those chips from data centers has led to shortages. Apple has responded by raising prices across its device lineup, including for older model iPhones that would typically get a price cut.
In theory, such a move could hurt Apple since iPhone buyers might wait longer to upgrade their devices. Yet since the first wave of price increases Apple shares have jumped 25%.
In part, that is because investors think Apple device buyers will pay the higher prices, enabling Apple to generate as much or more in total profits even if margins are down, said Bank of America analyst Wamsi Mohan. He added that Apple’s new foldable iPhone Duo is likely to be popular and that the new Siri AI will prompt iPhone upgrades.
AI has long been viewed as Apple’s Achilles’ heel. The company failed in its efforts to build its own frontier model, and it also allowed its Siri assistant to fall far behind rivals. ChatGPT revolutionized what smartphone users expect from chatbots.
Apple hopes to turn a corner with the new Siri AI. Built with Google’s help, Apple’s redesigned chatbot was released this fall. It still lags behind rival AI systems in key respects, though it has the advantage of being built directly into the device that many people already use, their iPhone. Apple also has access to more personal data, which could theoretically be used for compelling AI features, though new AI agents show the technology is advancing faster than Apple’s ability to keep up.
Tech rivals know how important it is to control the devices that people use to deliver their services. Perhaps most threatening to Apple, OpenAI is planning a family of AI-powered devices. It has poached hundreds of Apple employees to drive its device strategy. Apple responded in July by suing OpenAI, alleging it stole Apple trade secrets.
OpenAI has said it is “not aware of any evidence that this complaint has merit” and alleged in a new legal filing Wednesday that Apple poached one of its own employees who took confidential files with him. Apple said in response that it had found no evidence supporting the new allegation.
AI could be the battleground where Ternus makes his mark, either ushering in a new era of innovation and growth for Apple or finally losing its status as the company that has defined how people interact with computers.
