Post by @AwetTesfaiesus@mastodon.social
Let me be advocata diaboli here:
AI is the democratic equalizer, that lifts the general IT crowd to the level of (malicious) state actors and it doesn’t wait to be invited to the party by anybody.
How wrong is that pov?
@AwetTesfaiesus@mastodon.social
Ich versuche, die Rolle von KI bei #php2shell einzuordnen. Ist es das Fieber oder das Fieberthermometer?
Mir scheint: letzteres.
Liege ich falsch?
6 likes
@AwetTesfaiesus Given that running it requires compute and compute costs money, I would say pretty wrong.
@Merovius that I don’t understand. Compute and money seems to be exactly what state actors have plenty of
@AwetTesfaiesus Huh? Yes, that is exactly my point. AI is not a "democratic equalizer", because for that, everyone would need to have the same access to it. I could never run, say, a frontier model on all of GitHub. A state actor can. So I'm clearly not lifted to the level of a state actor.
@Merovius but someone ran it and it was brought to the public eye. So what used to be state only privilege is now in the public domain.
@AwetTesfaiesus But that access isn't equal. Someone could've read the code previously and found a vulnerability. The only thing privileging state actors was that they can use a lot more eyeballs. Now you can use compute. State actors remain privileged, because they have a lot more compute.
AI is definitionally a technology to replace labor with capital. The idea that it's "democratizing" is oxymoronic on the face of it.
@Merovius it seems to be beside the point but: newspapers have to be bought, too.
Maybe put it this way: it serves a purpose in the democratic realm
@AwetTesfaiesus Yes and what happens when news gets put under the control of capital, we are all observing right now.
Again, the *point* of AI is to centralize power. The space is controlled by like three companies in a trenchcoat. Controlling investments comparable to NATOs military spending. It's baffling to me, how one can look at that and think "democracy".
@Merovius I can see the point you are making.
Still: in this particular case, isn’t AI the thermometer, rather then the fever.
@AwetTesfaiesus I don't understand the analogy. What's supposed to be the fever that AI is the thermometer for? Because if anything, I would consider it the disease *causing* the fever.
@Merovius I assume that flawed code is the disease and that AI (at least in this case) merely measures the code‘s quality
@AwetTesfaiesus That *might* be true today (I'm not sure it is) but I certainly don't think it'll be true in five years. I mean, everybody in the industry *knows* all existing code is bad. And *right now* there are enough low-hanging fruit that you can find high-profile problems with comparatively low resource usage.
In five years, all devices will still be vulnerable, but exploiting that will require "auditing" too much code to be feasible for most people.
@AwetTesfaiesus@mastodon.social @Merovius@chaos.social no, it’s in the domain of the likes of OpenAI, google, anthropic. The public domain can’t afford it. We see a shift from state actors to a very few tech companies.
@docwhozee @Merovius granted. But in this particular instance: the news is packed with it. Everyone updates their servers now. Couldn’t possibly/ be more in the public domain.
Remember: not that long ago, the news used to be the public domain
@AwetTesfaiesus @docwhozee Heartbleed was also in the news. I don't understand the argument. And no, not "everyone updates their servers now". The percentage of actually fixed websites will be pretty low. The overwhelming majority of people affected won't even know their website uses PHP, even if they hear about it.
@Merovius @docwhozee chasing the tail now.
From: all servers are open to state actors via this flaw.
To: … many are not.
Wrong?
@AwetTesfaiesus @docwhozee Oh at this point right now, I would say definitely wrong. Every server (except *maybe* the ones from the hyperscalers) is almost definitely open to state actors right now. I would bet almost any amount that any intelligence agency worth its salt has a zero day for Linux.
@AwetTesfaiesus As far as I understand this, state actors were often private contractors before. But yes: AI makes a huge difference.
The fantastic @LiveOverflow shows how AI assist in writing a quick POC Exploit:
@AwetTesfaiesus Here's the more horrifying thing: The cost of attacking critical infrastructure that most of the internet relies upon is in the measly four-five digit sum and the incident response – the type that PMs post about advising people to update – relies on people that don't have contracts with any government.
I'm a bit biased, but I think this raises the need to bankroll those that everyone has been freeloading from.
@n0toose that I totally agree to. Sovereign tech funds should be pouring money into audits for standard software
@AwetTesfaiesus @n0toose Not just into audits but also into funding the people who are invested in improving that bit of software long-term.
@AwetTesfaiesus calling AI "democratic" seems quite a stretch to me, seeing it delivered by some of the most influencal anti-democrats or even fascists (musk) world-wide, and with the stated target to abolish human work and concentrate wealth even more.
Right now they are cheap or even free to use, which may look like a boon for people without money, but there are a lot of hidden costs (enviromental damage will have to be paid by all), and also a lot of obvious costs that are currently being shouldered by risk capital betting on the mentioned target, abolishing human work.
@AwetTesfaiesus If you would have to pay the real cost of a LLM query, it would not be that accessible for everybody anymore.
In addition, the companies controlling the compute could decide to disable your access at any time.
Regarding your initial topic, attacking security flaws, i do not see a surge of hackers. The thousands of CVEs detected by AI analysis are underwritten by hundreds of millions of dollars, if you would pay this money to actual open source developers you would get a lot more in return. Primarily, i see AI companies touting their own horn (aka marketing).
@SimonHain the printing press also costs a lot of money. It still historically facilitated journalism and served the public interest
@AwetTesfaiesus wouldn't you differentiate between who owns the printing press?
Do you see Murdoch in the public interest? Nius? Der Stürmer? X?
@SimonHain first: keep in mind that I am playing the devil’s advocate here.
Secondly: of course I see a power struggle here. But the NYT is privately owned for how many generations now?
So: in relation to the states, does AI in general/php2shell in particular currently enhance the states‘ powers or does it limit the states‘ powers?
@AwetTesfaiesus I don't see the state involved in php2shell, it's a (potential) security flaw in code written by open source maintainers, fixed by open source maintainers.
(potential, because i have not seen an exploit doing something else but crashing the server, except for some fabricated cases that used other local access issues - but i am not a specialist for php)
As for AI in general, i mainly see state actors having massive FOMO in Europe (sovereign clouds), deep entanglement to the corrupt trump government, and chinese companies having state connections is not a surprise as well.
I do not see any issues that states would like to solve with LLMs, it's more a "its the future" and not wanting to be left behind.
And a lot of big news outlets in private hand are currently playing a central role in the endangerment of democracy.
The new generation of LLMs could be a great equalizer, it it wheren't mostly in the hands of autocrats.
They have been corrupted by capitalism from the start. To make them a benefit for democracy, they have to be in the hands of the people.
I recently tried Microsofts Copilot. I had a massive tendency to promote technology that is in the hands of Microsoft or benefits Microsoft, except if explicitely asked, not to. From what I hear, grok massively promptes musk.
It's a lot like social media in that regard. It can be a great tool for democracy, but if it is in the hands of a few rich and powerful people, it also is a great tool for manipulation.
@AwetTesfaiesus I'm a cryptographer so only adjacent to/more specialised than general IT security, and from my perspective this is scarily on point.
LLMs/ML-aided tools are very good at pattern matching in a different way to humans, so they can find a lot of issues we missed. (this might dry up eventually, but it is currently a huge problem)
Additionally they can also help write exploits, which was previously a very specialised developer role, making it cheaper still.
@AwetTesfaiesus Right now, it is kinda-democratic, because you can get cheap AI-in-the-cloud access through various means. This is only the case because the big AI and compute companies are still in the "here's the free offer to hook you" phase. And we can see the end of that phase coming hard.
If their planned trajectory continues, it will be decidedly non-democratic, because the actual costs to run this in a datacenter are quite high, and not everyone (read: Only privileged few) can afford that.
But we also see a lot of innovation and smaller models that can be run locally on sort-of modest hardware. If the Ai-eating-all-the-chips-crises isn't killing off home computing, we have a decent chance of getting the tools into the hands of the masses.
But, again, that still doesn't address the question of how and on what material and for what compensation the models are being trained. That process currently still takes a lot of centralised and costly infrastructure...
@AwetTesfaiesus I'd say "disruptior" instead of "equalizer". AI models are built to give the companies who build them more power. Giving some benefits to individual people or less powerful groups is just a temporary measure to dominate the market 🤷♀️
@AwetTesfaiesus that's just one piece of the landscape, so maybe we just exchange the flavors of bad in one area, but we lose a lot of independence and abilities in others,,, which would make the situation it total worse
@AwetTesfaiesus
In a sense, I guess? AI works as a kind of "brute force multiplier", meaning that if in the past you needed enough expertise to know where to look in order to find something, you now just press the button and thanks to the "efficient randomisation algorithms" that AI uses, it will just automatically find anything there is to find withinin a non-astronomical time frame.
@AwetTesfaiesus 2 remarks:
1. the logic isn’t lost on me, but
2. golf courses in the desert sure make it look green but add very little to biodiversity.
@AwetTesfaiesus You might think so. In reality, however, it seems to be a bit more complicated.
The source provides a lot of other details about the PHP issue also.