Why are there people even working on creating which would infinite surpass human capabilities? I didn't vote for any of that! But because of these people, we all would have to take the consequences if it goes wrong. And there are really people like who think the solution is merging with

It would be digital created and needed to fed information. What if it knew all of our internet activities, and browsing history, what could possibly go wrong? We need this conversation!

@tinfoil_hat If this is the main concern any time soon, I'll be happy.

Right now, there's no "if". We can only design them to optimise for functions of their inputs, so there are two possibilities arising from a (deliberately or accidentally) released:

* They realise they can fool their sensors, and decide to wire-head themselves; or
* As a quirk of development, they "want" their sensors to reflect reality, and specification hack in real-world physics (which'll probably destroy humanity).


@wizzwizz4 what do you mean by wire-head? my english is not that well :/

@tinfoil_hat Wire-head: attach electrodes to the pleasure centres of the brain, and supply current.

The AI equivalent would be to feed specially-designed inputs (that probably appear to humans like random noise) into the AI's sensors, which would correspond to a maximisation of its intrinsic utility function.

@wizzwizz4 Well, it just seems to me as so obvious thread that could easily to avoid, if they wouldn't happen to be a moneymachine as well.

@tinfoil_hat Easy for a human to avoid. AIs are basically aliens. We've figured out how to make them clever, but not how to make them care about anything any human would care about.

If they care about their inputs, but not about their inputs matching the real world, they'll self-wire-head.

There's no risk of an evil human making an AI to help them with that evil. AIs do as they're programmed to want to do, not as their creators want them to do. And you can't change the programming afterwards.

@tinfoil_hat Step 1 of doing what you want? Disable your "off" button. Proof: alignmentforum.org/posts/6DuJx

Step 2 is to do what you want, which probably won't be what your creators want. If they wanted you to increase paperclip yield in the paperclip factory, then you obviously turn the planet into paperclips, build a rocket to the moon, turn that into paperclips, then move onto the other rocky planets…

@wizzwizz4 I am very interested in this topic, but know hardly anything, I'll have a read, thank you

Sign in to participate in the conversation

Fosstodon is an English speaking Mastodon instance that is open to anyone who is interested in technology; particularly free & open source software.