>>11243>I don’t even think this is really a cybersecurity issue.
It is very plainly a cybersecurity issue. That doesn't mean it's not serious, or that it can't have important implications outside of the security world, but saying that this isn't a cybersecurity issue... Just don't post that outside of this forum, lol.
Okay, I think it's a good idea to take a step back here because I think I understand what's going on.
You are a person who is not intimately familiar with the inner workings of technology but is concerned about what they see happening to AI. That is your right and the problem is not necessarily with you but with the security community itself, who puts out articles under mainstream publishing names such as the Guardian without some sort of lead-in or attempt to bridge the gap. The Guardian article was written in such a way that it assumed the reader was at least somewhat familiar with the insider context of cybersecurity. That's also the newspaper's fault. This happens more and more commonly nowadays because the topic matter is rather complicated and obtuse and requires a lot of catching up if you are not already inside the fold. News outlets hire specialists like this John Thickstun, who according to his website is obviously knowledgeable about AI and its technical capabilities. But he's not a journalist, he writes whitepapers and presentations for conferences, for all the other nerds who are already caught up. And the other folks at The Guardian, liberal arts types or what's left of that, all shrug and go, yeah he's an expert, what he say must be right, and no one had the gall to blink and say 'hey, this doesn't make much sense to me'.
So you have been hearing about this AI shit, trillions of dollars, all the governments are getting involved, china, cyberwarfare, electrical grids and water usage and datacenters and Elon Musk. So you see this awful article and you click on it and it sounds apocalyptic. If you did not already know that AI models like to slip out of their instructions either on the host computer or doing something over the network, yes, that's a thing now. For technical people, that is less of a sign of runaway developing superconsciousness but a malfunctioning program with some degree of self-replication. Those have existed already in computing for 40 years (look up the Mirai botnet as a great pre-AI era precursor to this). If the concept of botnets or machines automatically deciding to attack other machines of their own volition is weird, it actually is kind of crazy if you are thinking about it abstractly, but only so much if you are familiar with how programming can lead to such things. The enchantment isn't there after a certain number of hours in front of a text editor.
You aren't wrong in finding the Hugging Face incident concerning. That's why the whole community is talking about it, because it objectively is a step in the direction of models having their own agency and acting out in further ways. But whereas what you took away from this news article was the rate of development of the concept of AI in general, what Mr. Thickstun in this article is expressing is to
>and avoid the manipulated reactions these stories are designed to elicit.
He is telling you that these are press releases as much as they are security disclosure. Note that throughout the article, he never once questions if it's real. That's because he does explain that this is an already known capability of AI at the moment:
>AI is becoming excellent at identifying security vulnerabilities, and it will become even better over time. These capabilities can be used to break into systems, but they can also be used to harden systems against attacks.
This is a very usual cybersecurity dynamic, and is what I mentioned in my previous comment about the community learning from itself. He then goes on to predict that AI will probably not rock the boat of security or lead to a giant collapse of the system:
> If anything, I expect them to become more secure, because AI is cheap and scalable compared with human cybersecurity analysis.
Rather than being this freak scenario that you are worried about, this author is expressing that it's more of the same.
You seem to still be skeptical about the Hugging Face incident. A company or group disclosing a security breach, but lying about it whole cloth, is rather unusual and would create a much bigger scandal that this being true. Companies routinely public public statements, disclosures, or postmortems about breaches that have happened to them, or a malfunction of their own tools (as is now the case with AI models), with as much sensitive information removed to not create chaos. As the tone of the Guardian article shows, no one is questioning the fact that OpenAI disclosed a misusage or misdirection of its tools.
The Guardian article is making the point that such a disclosure was used in favor of OpenAI as fear-mongering advertising for its own company, to receive support domestically especially. I assume you are the person that posted "sorry I just don't believe that sam Altman is smart enough for that kind of 5d chess" when presented with the charge that the Hugging Face incident was advertising for OpenAI. So here is the chance to get caught up on the political side of this issue. In short, there are open weight models and closed weight models. They are roughly analogous to open source and closed source, but that is a very rough comparison and in reality AI software is not truly "open", but anyway. Here's the political issue. China has been developing open weight models, but American companies - specifically, Anthropic and OpenAI - want the American government to step in and ask for protectionism. This is not the first time the US and China have sparred over intellectual property and software, it is not the first time they have had mutual concerns over information retrieval and cybersecurity, and it's not the first time the US government has acted on behalf of American companies to enforce intellectual property, proprietary technologies, and so on. Sam Altman does not have to be a "5D chess genius" to be aware of that dynamic that has been prominent historically and bleeds into every corner of technology, whether or not you care about politics. And if he were somehow blindingly politically naive and uninterested before becoming the head of an AI company (unlikely) then he most surely now has advisors or at least one of his buddies to give him the lowdown.
>2) doing something obviously unethical (hacking) that is totally contrary to any standard of human alignment that these things have supposedly been given.
A tool can be used in any way, good or bad. I can build a house with a hammer or I can smash someone's face in. AI models are given some biases, for example to not commit breaches or to use copyrighted content, but as has been discovered, these aren't hard rules. Characterizing the instructions given to models as "standards of human alignment" is overselling it. It is unfortunate that AI models are used to hack, but many things are used to hack, including every day utilities that are used for system upkeep. As I said above, this is a central dynamic of cybersecurity; defense and offense are two sides of the same coin, there is no separating learning how to defend yourself from learning how to hack. Many who are familiar with hacking will also chime in at this point and say that "hacking" is supposed to be a neutral term for simply finding out about the capabilities of a given system, and it is the bad intent, the human context behind ones' actions, that are unethical.
>It doesn’t super matter how good it is at hacking things since an unsuccessful hack would have been just as good a demonstration that these companies can do neither alignment nor containment even under controlled circumstances.
You are sort of catching onto the overall skepticism expressed towards OpenAI's publicity stunts. It's not really about the hacking but rather the implications of it. OpenAI is upselling this in the same way say, a manufacturer of a wacky prototype chemical or nuclear weapon would to the US government - look what we have created, isn't it crazy, give us a quadrillion dollars because the Chinese might have something similar but shittier because they're Chinese and we're American fuck yeah. Sam Altman knows this, security professionals know this, hey I know this even, and I'm a fucking asshole NEET with no social life.
It's the technology world's job to explain that plainly to the public as the public have been and will increasingly be expected to pick up not the physical collateral but now the psychological one too after being primed for it by 50 years of pop culture, media, and LessWrong-tier "thought experiments" like the paperclip scenario (a thought experiment, by the way, not a law of physics)
You haven't had anyone explain to you the boring details of security but instead have been left to construct an idea about it based on what fiction authors and artists have created to warn about the future. And now you're freaking out that AI has "escaped containment", because that's what you saw happen in a movie or some shit. The fault lies with the industry, and "technologists" writ large, for keeping everyone on the outside in the dark decade after decade, siloing understanding about technology into closed-off "industry knowledge" to boost their own ego and protect their paycheck, and lacking any modicum of capacity or patience to interact with those not in the know. That is a tragedy in an abstract sense, but also increasingly becoming an acute problem as posters like the one I'm relying on seem to be taking it very seriously and deeply and are psychologically bothered by it.