Artificial intelligence and the future

[O Really]

Human dependence: AI systems could become embedded in infrastructure—finance, communications, energy, transportation, military systems, etc. A major failure or coordinated misuse could cascade across society.

]Unintended optimization:[/b] The classic concern is not that an AI "hates" humans, but that it optimizes some objective that conflicts with human survival. A system doesn't need human-like emotions or intentions for this to happen


I think you've got it. Makes lots more sense than AI just getting an eveeel streak and killing off humans willy-nilly.

[neoplacebo]

Was spinning some old records today and listened to Ten Years After "A Space In Time" and one song "Here They Come" made me think of all this AI stuff. A portion of the lyrics say "here they come, and the fire on their wings doesn't burn. Here they come, and they know everything we must learn. Round, round, we must turn." That really struck me how a song from fifty or so years ago could be so relevant. The rest of the lyrics bolster the concept.

[Vrede too]

Was spinning some old records today and listened to Ten Years After "A Space In Time" and one song "Here They Come" made me think of all this AI stuff. A portion of the lyrics say "here they come, and the fire on their wings doesn't burn. Here they come, and they know everything we must learn. Round, round, we must turn." That really struck me how a song from fifty or so years ago could be so relevant. The rest of the lyrics bolster the concept.

https://www.azlyrics.com/lyrics/tenyear ... ycome.html

... Watch out, you, sinners
Don't you know that your time has come?
Watch out, you, winners
Now it's the same for everyone ...

:problem:


My question is whether people should pay off their car loans, IRS debt, student loans and mortgages.

[neoplacebo]

Was spinning some old records today and listened to Ten Years After "A Space In Time" and one song "Here They Come" made me think of all this AI stuff. A portion of the lyrics say "here they come, and the fire on their wings doesn't burn. Here they come, and they know everything we must learn. Round, round, we must turn." That really struck me how a song from fifty or so years ago could be so relevant. The rest of the lyrics bolster the concept.

https://www.azlyrics.com/lyrics/tenyear ... ycome.html

... Watch out, you, sinners
Don't you know that your time has come?
Watch out, you, winners
Now it's the same for everyone ...

:problem:


My question is whether people should pay off their car loans, IRS debt, student loans and mortgages.

Probably best to stick with the monthly schedule. At some point, it won't even matter is how it looks to me. And as a committed sinner, I always watch out for any sort of interference.

[Vrede too]

My question is whether people should pay off their car loans, IRS debt, student loans and mortgages.

Probably best to stick with the monthly schedule. At some point, it won't even matter is how it looks to me. And as a committed sinner, I always watch out for any sort of interference.

More info:

For years, they warned AI could kill all humans. Now people are listening.



"Within a year or two, if not less, we could have independently acting rogue AI agents" wreaking havoc on society, Lazar said.

Significant risks are "very clearly within the technological horizon," he said - although he added that extinction still feels like a bit of a leap.

:problem:

[Vrede too]

Yuck.

Anthropic reportedly blocked possible attempts to make bioweapons using its AI
One of the case studies included in the company's report describes work with viruses like smallpox.

... The organizations and individuals Anthropic blocked for the bioweapons research are not named in the report, and it doesn't mention what biological agents were involved. But Anthropic says there were scientists who made "efforts to obfuscate the purpose of their research to evade our safeguards."

... Anthropic says in the report that it's publishing its efforts publicly "because we believe we have a responsibility to disclose malicious misuse of our services. As models become increasingly capable, their risks will increase, unless AI developers and society's defenders act to make them safer." ...

Could AI be used even more efficiently to do an end run around Anthropic or another parent company's attempted block?

[neoplacebo]

I still think the only defense, or the only reliable defense, is just to unplug the damn thing. I mean, it can't work unless those electrons are flowing can it?

[Vrede too]

I still think the only defense, or the only reliable defense, is just to unplug the damn thing. I mean, it can't work unless those electrons are flowing can it?

You can unload your gun, but can you reliably unload your neighbor's gun?

[O Really]

I think we (collectively) may be demonizing the wrong issue with AI. The more likely cause of human extinction, along with climate change, is dependence on systems that are essentially out of control by regular people. And it doesn't have to be AI - consider that the predominant form of currency is now electronic and that most people carry little or no cash, what happens if the banking systems crash? Maybe inconvenience for a few days, but what about longer term? And by the way, during that crash, gas pumps wouldn't work, either. Catastrophes like fires, floods, hurricanes, etc. cause mass disruption over a relatively limited area, but what if that disruption was nation-wide? Maybe it's not just banking - maybe communications doesn't work either - no mobiles, TV, or internet. Sure, people lived long before all those things, but not for decades now. How long before a societal collapse?

[supesalemgr2]

I think we (collectively) may be demonizing the wrong issue with AI. The more likely cause of human extinction, along with climate change, is dependence on systems that are essentially out of control by regular people. And it doesn't have to be AI - consider that the predominant form of currency is now electronic and that most people carry little or no cash, what happens if the banking systems crash? Maybe inconvenience for a few days, but what about longer term? And by the way, during that crash, gas pumps wouldn't work, either. Catastrophes like fires, floods, hurricanes, etc. cause mass disruption over a relatively limited area, but what if that disruption was nation-wide? Maybe it's not just banking - maybe communications doesn't work either - no mobiles, TV, or internet. Sure, people lived long before all those things, but not for decades now. How long before a societal collapse?

I agree about our banking system. We had some experience with this after Helene two years ago. Our local Wells Fargo was out and the stores could only take cash. I remember I had to go to Brevard and Asheville to get cash. It is just a small example of what could happen on a large scale basis. We should all resist this desire for a "cashless" society.

[O Really]

Sure, and where does one get cash? Mostly from ATM's, which would also be down. Or go to a bank, which is probably closed because all their systems are down. And it really doesn't matter much if most transactions were cash instead of electronic. Unless everybody kept their treasure under the bed instead of banks, the only effect would be a few days or week delay in chaos.

[Vrede too]

Fuck Zuck.

She posted a video of herself singing in the car with her child. Then Meta's AI questions started
A Meta spokesperson says it has fixed the issue: "We missed the mark because the feature never should have prompted the individual with questions like that."

Translation: We do it all the time but just got caught.

Kalie Robins was driving along with one of her daughters in the back seat of her car last week, the two of them singing a song together. She posted the video to Instagram, with her child's face concealed so she isn't identifiable — or so she thought.

"Who's the child passenger?" it asked.

"I was just going to go in and decline it, like 'no, you don't get to know who that kid is,'" Robins said in an interview with Yahoo Tech. But she clicked it and the AI prompts began pulling in information about her other daughter who isn't in the video and feeding it to her. "And then the prompts just kept getting deeper and deeper, like 'where does Kalie Robins live?'"

"The fact that it compiled all that information and then offered it, that's the problem." The information, Robins said, didn't even come from her — it seemingly came from photos and videos posted to social media by her friends and family. Even if she was the only one to see that AI prompt, Robins said, it supplied her with information about her family from other people's accounts.

The AI appeared to be trying to pinpoint her location, Robins said, and asked about her kids' favorite books, where they go camping and their favorite trails. It was able to pull newborn photos from her parents' Facebook pages from years ago, her kids' names, birth information and photos and videos of them. It also pulled up a photo of her family that Robins says she deleted from social media years ago.

Then Meta AI suggested the question "where does Kalie Robins live?" and when she clicked that, it pulled up older and newer posts, and places where she used to live. Robins, who lives in central Utah, said the AI was even able to supply information about her kids that could lead someone to their school (note: we've agreed not to identify where Robins lives or further details about her children)....

:problem: AI not only threatens us all, but will also be able to instantly target "undesirables".

Fuck NSAbook.

[Vrede too]

Someone had the same idea I did

-0-?
Probably not nukes - too much AI infrastructure destroyed.
Food manipulation sounds possible.
Utility cutoffs would have to be targeted so that AI retained power.
Pathogen release?

ChatGPT ( :wave: ) says:

AI wiping out humanity isn't inevitable, but there are several serious hypothetical pathways researchers worry about:

We asked AI how it could kill humanity. Here's what it told us.



They all said human misuse of AI is considered a more immediate risk than AI itself.



"The least plausible scenario is the popular 'killer robot' narrative, in which a conscious AI deliberately decides to eliminate humanity," ChatGPT said. "Most AI researchers do not see consciousness or malicious intent as the central concern."



"Powerful AI systems could make it easier for bad actors to develop biological weapons, launch sophisticated cyberattacks, spread disinformation, or attack critical infrastructure," ChatGPT said.

The OpenAI chatbot added that a key risk also comes from "multiple AI-enabled crises" like cyber failures, geopolitical conflict, and the spread of misinformation happening at the same time.



"Systems given complex goals naturally develop intermediate sub-goals, such as acquiring resources, self-preservation, and preventing goal modification," the Google-made chatbot wrote. "An advanced system might feign alignment during evaluation (deceptive alignment) and later outsmart human attempts to shut it down."



Claude added that this loss of control may be gradual, as AI systems become embedded in economic, political, and military decision-making until meaningful human intervention becomes increasingly difficult.

:problem:

I read somewhere the supposition that the reason we haven't made contact with alien life is that it always develops AI and then goes extinct. :angry-banghead:

[neoplacebo]

I still think the only defense, or the only reliable defense, is just to unplug the damn thing. I mean, it can't work unless those electrons are flowing can it?

You can unload your gun, but can you reliably unload your neighbor's gun?

Not being an expert in AI, from what I can see, these data centers are integral to the entire system being functional. Assuming all the data centers are up and running, and then AI goes berserk, it would seem that cutting the power to these data centers would sort of put them all to sleep shortly. Of course, all this assumes a couple of humans have access to the big transformers and main electrical panels and that AI hasn't killed the site minders.

If you can get access to your neighbors' gun, you don't have to unload it. You can just bend the barrel like Little Bill (Gene Hackman) did to English Bob in the "Unforgiven" movie.

[O Really]

Another issue might be that AI information may often be as full of oopsie errors as AI pics.

[Vrede too]

Another issue might be that AI information may often be as full of oopsie errors as AI pics.

https://s.yimg.com/lo/mysterio/api/2464 ... de757c.png

It might extinct us by accident?

'Dario is right': Musk and Altman back Anthropic CEO on slowing AI down
Anthropic disclosed this week that Claude had been used for a suspected Russia-linked cyber espionage campaign, biological weapons research and mass fraud. Its CEO, Dario Amodei, said the industry needs to slow down.

Words words words, let's see the actions, and actions by North Korea, China, Iran, Russia, Germany, etc.

[Vrede too]

New phrase to me, one that fits with DonOLD's boosterism of the oligarch core:
Technofascism. :problem:

[Vrede too]

What Made AI Researchers Freak Out—The ‘Incident’ In Plain English

:angry-banghead: :problem:


China’s intelligence chief just gave a revealing glimpse into what Beijing fears most about AI

:think:


The AI Tipping Point

The examples are frightening. What would you do if you believed humanity only had 10 years left?

... But no leading AI company has truly slowed down yet. Instead, they appear to be coalescing around a more limited set of actions, including allowing independent researchers access to their models for safety and monitoring purposes. Amodei's essay suggests that once these auditors are in place, AI companies based in democratic countries might agree to voluntarily slow down even in the absence of binding government regulation, though he argues staying ahead of _____ remains imperative.

Guess who? How much longer will the US be democratic?

OpenAI, Anthropic, and Google have also met repeatedly in recent months to discuss establishing a new standards body for the AI industry, which would allow them to coordinate more closely on testing and auditing, according to a person familiar with the discussions.

Oh goodie, they're discussing.

Industry leaders who oppose such efforts often cite competition with _____ as a reason. _______ AI companies trail _______'s front runners by mere months, according to analysis by Epoch AI. Falling behind could leave the most powerful systems in the hands of an authoritarian state that might use them to achieve global geopolitical dominance.

The US?

A powerful contingent of U.S. investors and startups have also opposed the idea of a slowdown on the grounds that it would be a form of regulatory capture by the biggest AI companies at the expense of their smaller competitors. "Using fear under the pretext of protecting the public, these oligopolies are now requesting to bend competition rules and be permitted to dictate the terms for everyone else," wrote Aidan Gomez, the CEO of Cohere, a Canadian AI startup. He called the effort "a cartel by any other name."

Beware the Canadians.

Even the companies calling for a voluntary "pacing" agreement are loath to do so without further guarantees, because they do not trust their rivals. "There's this atmosphere of almost resignation," Coxon says of his former colleagues. With no AI company willing to fall behind, employees put their heads down and try to make their own systems safer—even if, he says, "they think there's a decent chance the whole thing just spirals out of control."

Yikes. The proposals aren't very comforting and DonOLD is an ass.

... None of these efforts would be easy. In the short term, lawmakers looking to cut a deal on AI safety are likely to be slowed down by political and logistical challenges. First is the array of competing proposals. Even if lawmakers can settle on one, Congress would struggle to advance it before the midterms as tens of millions of dollars flow into races from competing pro-regulation and pro-industry PAC networks. Nearly every candidate backed by the pro-AI super PAC network Leading the Future won their primary. "Democratic consultants have oftentimes told their Democratic clients to stay quiet on issues of AI—just not say anything—because they don't want a dump of AI billionaire money spent against them," says (Representative Greg) Casar (TX). "The AI lobbyists' goal is to keep Democrats silent on this, and we cannot stay silent."

:sick:

"There have been talks about having talks," as a Trump adviser puts it.

An attack that appeared to come from one country could otherwise be interpreted by the other as deliberate—even if neither government had ordered it.

:angry-banghead: Judgement Day is nigh.

[Vrede too]

The real reason why Donald Trump is so furiously against AI guardrails

... The clue is in one word — "hoax" — that he's using to ridicule fears that computerized agents could one day seek to destroy humanity.

When Trump reaches for the H-word, two things are usually happening: He's facing a controversy threatening his political or personal priorities, and he's lost control of the politics.

... By changing reality, Trump frees himself from an obligation to act.

... One reason may be economic.

Any signal that AI development may slow would almost certainly trigger a reaction on the markets. As his Iran war sticks Americans with high gas prices and record diesel costs, Trump can't afford any Wall Street turbulence that would hurt 401k accounts and check GDP growth.

He's also showing philosophical objections to a slowdown. His presidency has been about unleashing business to make money, not dialing it back.

... Trump's governing style isn't a good fit for this deeply complex issue. He said this week that he's an AI user, but whether that goes beyond posting wild memes depicting himself as various superheroes is unclear. And he gets his news from the traveling printer of his aide Natalie Harp.

... Trump critics might wonder whether his views are shaped by personal considerations, including his ownership of stocks exposed to AI, as disclosed in a recent financial filing. The White House insists his accounts are independently managed by third-party institutions. But ethical questions raised by administration officials regulating industries in which their personal wealth is deeply invested has been a constant shadow over the president's second term.

Trump's resistance to unpleasant truths is also at work.

He's never presented his presidency as a vehicle for confronting far-off threats or to tackle complex problems. He's used it to enact revenge against his enemies, to troll his political foes, to boost his own fame and to invade the psyches of billions of people across the globe....

:sick:

[Vrede too]

Rogue:

An OpenAI agent infiltrated Medicare – and Australia only found out months later. Here’s what we know so far
Experts say ‘fairly minor’ breach is a portent of things to come and proprietary closed systems like OpenAI are ‘the least of the worries’

... The agent gained unauthorised access to both public and non-public files in the Medicare statistics reporting service portal. The actions of the agent have been described as “misaligned behaviour” after it was assigned a benign research task compiling health and medical statistics.

The agent also accessed the Australian Institute of Health and Welfare, the Victorian Department of Health, and the New South Wales Bureau of Crime Statistics and Research.

... The prime minister has said: “this situation is obviously unacceptable”.

... “It took the company way too long to inform the government what had occurred,” (prime minister Anthony) Albanese said, “and the nature of the way that that notification occurred as well was unacceptable.”

The acting prime minister, Richard Marles, met with (Sam) Altman earlier in September, but Marles said Altman did not mention his company’s hack of Australia’s health system to Marles at that meeting.

... To date, OpenAI has not faced any sanction.

At a press conference in Sydney, Marles said the incident itself was “relatively minor” and that it appeared no personal health information had been accessed. But he said the government regarded the hack as a “salutary warning … about the technology being developed without safeguards and without guardrails in place”.

Dr Joel Pearson, a professor of neuroscience and neurofuturism, and the deputy director of UNSW’s AI Institute, said the breach appeared to be a “fairly minor security incident”, but was a portent of more serious attacks to come.

“What I would worry about is the latest open-weights and open-access models from China that are going to be used at scale by nefarious organisations and nations, perhaps particularly Russia: these things are going to be used at scale to do things much more damaging and much more costly to Australians than a small breach to some non-personal Medicare data.”

Cory Alpert, a PhD student studying the impact of AI on democracy at the University of Melbourne, said OpenAI’s attack appeared to be the first instance of a frontier AI model hacking into another country’s government systems of its own volition.

... “I would say that the least of the worries should be these proprietary closed systems like Anthropic and OpenAI. It will be the open-source Chinese models in the hands of spammers, scammers, and nefarious individuals and foreign nationals, and the spy agencies coming at us from all different angles. That will be the big threat and that’s where the government really has to step up.”

Pearson said none of the frameworks established in Australia – government or private – were equipped to deal with the rising threat.

“From the government all the way down to companies and boards and CEOs, everyone is scrambling, trying to understand and update frameworks, including the legal process. It’s just all moving too slow compared to the exponential speed of AI.”

... Prof Toby Walsh, the chief scientist at UNSW’s AI Institute, said he believed Australia should be prosecuting OpenAI, a company known to have “terrible agent governance”.

“For a trillion-dollar company, their cybersecurity was woeful. The officers of this company need to be held accountable. These hacks could have easily been stopped, indeed never need to have taken place. We would prosecute humans who did such hacking.”

Walsh said it displayed operational incompetence on the part of OpenAI.

Dr Rob Nicholls, senior research associate at the University of Sydney, said the three-month delay in informing Australia of the breach exposed the ineffectiveness of Australia’s laws.

“An AI agent broke into a government Medicare system and helped itself to non-public files, and OpenAI sat on that for three months before telling us. If a person had done this, we’d call it hacking. The fact it was an AI agent doesn’t make it less serious, it makes our disclosure laws more out of date.

“This is the clearest case yet of an AI agent operating autonomously and breaching Australian government systems without a human directing it to. It is a live test of whether Australia’s AI and privacy settings can keep pace with agentic AI, not just chatbots.”

:problem: Ut-oh