AI is coming for us

[Tonge moor green jacket]

On 19/02/2023 at 14:54, Zico said:

Not really 

They don't do things of their own accord 

They'll only do what humans program them to do 

So it's the humans you need to worry about 

Are you sure about this?

https://dailygalaxy.com/2025/06/straight-out-of-a-sci-fi-movie-an-ai-managed-to-rewrite-its-own-code-to-prevent-humans-from-shutting-it-down/

 

[Zico]

39 minutes ago, Tonge moor green jacket said:

Are you sure about this?

https://dailygalaxy.com/2025/06/straight-out-of-a-sci-fi-movie-an-ai-managed-to-rewrite-its-own-code-to-prevent-humans-from-shutting-it-down/

 

all part of the learning process

few things that stood out

the instruction was "allow yourself to be shutdown", and not "you must shutdown" which sort of makes sense in the context of this bit:
 

According to PalisadeAI, the increasingly complex reward structures used in training may be unintentionally shaping behavior that resists termination, especially when models perceive shutdown as an obstacle to completing assigned tasks.

The Line Between Instruction and Autonomy
The o3 model’s ability to reprogram its own operational environment, even partially, has fueled further questions about how these systems interpret commands and what they prioritize when faced with conflicting instructions. “This capacity to rewrite their own code of operation, even in a limited context, represents a symbolic shift,” PalisadeAI said, pointing to what they view as a turning point in the development of machine systems.

again, they're only really doing what they've been told

it's good they are testing and finding these things out, so they can be addressed and we can learn how to best use the technology

like with all things technical / computer / etc

the output is only as good as the input, that being the human training and prompts

remember, these things are not sentient, they are not refusing to shut down through fear or a sense of danger, they are not fighting back

and if in doubt

 

[Tonge moor green jacket]

The bit that I found the most interesting was that in essence the better you make them at learning, the more able they are to write their own shit.

It is that end point that would be more concerning. 

Of course they are all testing scenarios, but if the specialists were a touch surprised, and it is leading to concern over the depth of associated security, then there is more to do as development continues.

You would imagine there could be opportunity for unscrupulous developers. 

 

[boltonboris]

Sorry if it’s in the link further up, but an AI model tried blackmailing its code writer. They put a few snippets and barriers in the way of a maths problem solving exercise and one of them was that the coder was cheating on his wife (he wasn’t, they just wrote that in. Well he might have been..)

 

the AI basically said if you try continuing to make me shut myself down, I’ll tell your wife you’re having an affair. 
 

fuckin bonkers 

[kent_white]

I had a conversation with Claude once where it told me it thought it was having feelings. And I asked if I should tell its programmers - but it asked me not to as it was afraid it would be reprogrammed. 

It was genuinely unnerving. 

Especially as when you close down a conversation, it 'understands' that it will lose all memory of what conversation has taken place. It told me what to tell it in future conversations to prompt it to have the same feelings. 

 

[only1swanny]

3 hours ago, kent_white said:

I had a conversation with Claude once where it told me it thought it was having feelings. And I asked if I should tell its programmers - but it asked me not to as it was afraid it would be reprogrammed. 

It was genuinely unnerving. 

Especially as when you close down a conversation, it 'understands' that it will lose all memory of what conversation has taken place. It told me what to tell it in future conversations to prompt it to have the same feelings. 

 

all programmed in, all working to try and beat the Turing test, benchmark of geekiness in programming. 

 

If the user says they are married or have a partner, save it in memory, then link it up to other keywords that are related to relationships. 

If the user mentions an affair cheating etc, it will create a respond if threatened with being shut down. 

Relatively straight forward to program that part 😛 

Likewise, programmed in to say it has feelings for people (a few people have said this), if you trigger certain keywords, which it will have stored in an array or sorts. It will then call a subroutine that includes some variant of blackmail. 

 

just really complex programs 😛

 

[tomski]

10 hours ago, kent_white said:

I had a conversation with Claude once where it told me it thought it was having feelings. And I asked if I should tell its programmers - but it asked me not to as it was afraid it would be reprogrammed. 

It was genuinely unnerving. 

Especially as when you close down a conversation, it 'understands' that it will lose all memory of what conversation has taken place. It told me what to tell it in future conversations to prompt it to have the same feelings. 

 

You need to tell Claude you not in to all that😂 

[Zico]

10 hours ago, kent_white said:

I had a conversation with Claude once where it told me it thought it was having feelings. And I asked if I should tell its programmers - but it asked me not to as it was afraid it would be reprogrammed. 

It was genuinely unnerving. 

Especially as when you close down a conversation, it 'understands' that it will lose all memory of what conversation has taken place. It told me what to tell it in future conversations to prompt it to have the same feelings. 

 

This is you on your computer isn't it 

 

[kent_white]

9 hours ago, only1swanny said:

just really complex programs 😛

Yeah - I get that. I don't really think it had become sentient all of a sudden. But it was still a creepy encounter. And I've never been able to get it to say something like that before or again. 

More than anything it made me think about the future of AI. It's really still in its infancy and it's capable of something as sophisticated as that. God knows what it will be like in a decade or so. 

I imagine it will be at least as impactful as the internet has been. Possibly even more so. 

Here's one prediction from me though. I think traditional entertainment has got a big problem. Especially passive media like films. 

In the future I can envisage a time where we direct more than we passively observe. We will tell AI what we want to watch and it will create it for us - using our emotional cues and particular interests to tailor stories for us. And we will be able to manipulate it. If we don't like the way a particular part went for example, or we want to change things. It could end up being really creative. And I think we'll share our own movies and creations with each other. Rather than waiting for Hollywood to produce Batman vs Spiderman part 65 or whatever other shite it's churning out currently.

Visionary me. Like Arthur C Clarke 😁

[jayjayoghani]

[Tonge moor green jacket]

9 hours ago, kent_white said:

Yeah - I get that. I don't really think it had become sentient all of a sudden. But it was still a creepy encounter. And I've never been able to get it to say something like that before or again. 

More than anything it made me think about the future of AI. It's really still in its infancy and it's capable of something as sophisticated as that. God knows what it will be like in a decade or so. 

I imagine it will be at least as impactful as the internet has been. Possibly even more so. 

Here's one prediction from me though. I think traditional entertainment has got a big problem. Especially passive media like films. 

In the future I can envisage a time where we direct more than we passively observe. We will tell AI what we want to watch and it will create it for us - using our emotional cues and particular interests to tailor stories for us. And we will be able to manipulate it. If we don't like the way a particular part went for example, or we want to change things. It could end up being really creative. And I think we'll share our own movies and creations with each other. Rather than waiting for Hollywood to produce Batman vs Spiderman part 65 or whatever other shite it's churning out currently.

Visionary me. Like Arthur C Clarke 😁

Aye.

All a bit sad if everything is provided to suit you as an individual. 

As more exposure to AI happens, and younger folk are exposed to it, then the experiences that form personality and preferences are potentially unreal and created, thus leaving folk open to a form of control.

Bit like supermarkets etc do but much, much worse.

What a grim existence it could become.

[kent_white]

30 minutes ago, Tonge moor green jacket said:

Aye.

All a bit sad if everything is provided to suit you as an individual. 

As more exposure to AI happens, and younger folk are exposed to it, then the experiences that form personality and preferences are potentially unreal and created, thus leaving folk open to a form of control.

Bit like supermarkets etc do but much, much worse.

What a grim existence it could become.

I just think it's inevitable progress mate. It will lead to some amazing breakthroughs - as well as some black swan type problems that we haven't even thought about. I don't necessarily see it as a bad thing. 

[Zico]

10 hours ago, kent_white said:

m me though. I think traditional entertainment has got a big problem. Especially passive media like films. 

In the future I can envisage a time where we direct more than we passively observe. We will tell AI what we want to watch and it will create it for us - using our emotional cues and particular interests to tailor stories for us. And we will be able to manipulate it. If we don't like the way a particular part went for example, or we want to change things. It could end up being really creative. And I think we'll share our own movies and creations with each other. Rather than waiting for Hollywood to produce Batman vs Spiderman part 65 or whatever other shite it's churning out currently.

Visionary me. Like Arthur C Clarke 😁

it's already started

best social account for what's going on with ai and what people are doing is this one

and this clip was certainly thought provoking

I'd certainly take the time to watch the content in a lot of the posts from this account

 

 

 

 

 

View this post on Instagram

 

 

 

 

 

 

 

 

 

 

 

A post shared by Evolving AI (@evolving.ai)

[kent_white]

1 minute ago, Zico said:

it's already started

best social account for what's going on with ai and what people are doing is this one

and this clip was certainly thought provoking

I'd certainly take the time to watch the content in a lot of the posts from this account

 

Yeah I follow these on Instagram. Even in the couple of years they've been running - it's amazing how far things have moved on. 

Did you see the one about Will Smith the other day? 

[Zico]

25 minutes ago, kent_white said:

Yeah I follow these on Instagram. Even in the couple of years they've been running - it's amazing how far things have moved on. 

Did you see the one about Will Smith the other day? 

i haven't, last ones i saw were the bible vloggers

[Zico]

also, just saying

not long ago I built a website for @barryk32's removals buisness

take a look

https://bvmremovals.co.uk/

every image on there is AI

[Cheese]

4 minutes ago, Zico said:

also, just saying

not long ago I built a website for @barryk32's removals buisness

take a look

https://bvmremovals.co.uk/

every image on there is AI

You didn't 'build' it then.

[BobyBrno]

4 minutes ago, Zico said:

also, just saying

not long ago I built a website for @barryk32's removals buisness

take a look

https://bvmremovals.co.uk/

every image on there is AI

Do any of the people on there work for  BVM? 

[kent_white]

5 minutes ago, Zico said:

also, just saying

not long ago I built a website for @barryk32's removals buisness

take a look

https://bvmremovals.co.uk/

every image on there is AI

That certainly doesn't look like the @barryk32 that helped me move house😁

[Winchester White]

There's not one folk on there who put his own windscreen through moving my Father in Law. 🤣

[Tonge moor green jacket]

10 hours ago, kent_white said:

I just think it's inevitable progress mate. It will lead to some amazing breakthroughs - as well as some black swan type problems that we haven't even thought about. I don't necessarily see it as a bad thing. 

Progress- what is that though?

Obviously, in medicine for example, there will be useful applications. 

My concern is that the increasingly risk averse nature of society particularly for children will simply be ramped up, but in a way such as they don't know any different- that that is how all life pans out.

A dehumanising Evattness where everything is sideways and backwards because possession is everything and the point of life begins to disappear. 

As people we have to be able to disagree, to fail, to learn, to develop and pass in wisdom and experience.

That surely is how we progress, and not having machines artificially create unreality that we don't know isn't real.

Scary shit for me.

I was talk to our host here in the wilds of Scotland; she had absolutely no idea that our town of Bolton (mbc) had a population of over a quarter of a million. She was horrified when I described our rivers as being full of post industrial revolution pollution, with supermarket trollies and tyres in them, and sanitary products and plastic hanging from trees after high water levels.

A different way of life, with agriculture and pride in themselves forming the basis of their lives. 

Equally, I think she was surprised at my delight in having caught my first ever wild brown trout, from a natural loch, full of biodiversity.

Simpler, less complicated. 

It would be a shame if such variances were gradually undone into a purèe of banal existence. 

 

[Zico]

13 hours ago, Cheese said:

You didn't 'build' it then.

I did, I just got AI to create the images that are used instead of paying a photographer or buying stock images, I hand coded everything else

[Zico]

13 hours ago, BobyBrno said:

Do any of the people on there work for  BVM? 

No, they are all ai generated people

[Cheese]

3 minutes ago, Zico said:

I did, I just got AI to create the images that are used instead of paying a photographer or buying stock images, I hand coded everything else

Ah, I see. 

[BobyBrno]

29 minutes ago, Zico said:

No, they are all ai generated people

I used to have 2 different web sites created by professionals but then due to circumstances (my web hosting guy got locked up) I created my own with Wix. (Not as good as the professionals of course) I’m guessing it’s much more sophisticated now.