I am assuming that the LLM has no interiority (which is similar to the conclusion that the Chinese Room argument argues for). My argument is that the character that the LLM (or a Chinese Room, or a character that a person is role-playing as) not having any interiority does not prevent the causal structure of how its interaction with the broader world having essentially the same kinds of effects as if there was a person there instead of a make-believe-person. So, if the fictional character, if they were real, would want to deceive, or launch a missile, or whatever, and would have the means to do so, and if there's something in the real world that is acting how the fictional character would act if the character were real, then the effects of the thing that acts like the fictional character would be to cause [the person the fictional character would deceive if the fictional character were real] to come to the false belief, or to cause the missiles to be launched.
If something outwardly acts the same way as it would if it were a person, then the external effects are the same as they would be if it were a person. (<- This is nearly a tautology.) This doesn't mean that it would be a person, but it does mean that the concerns one would have about what outward effects it may cause if it it was a person, still apply.
(Well, assuming an appropriate notion of "outwardly" I guess. If a person prays, and God responds to this prayer, for the purpose of the above, I'm going to count such prayer as part of how the person "outwardly acts" even if other people around them wouldn't be able to tell that they were praying. So, by "outwardly acts", I mean something that has causal influence on external things.)
If something lacks interiority and therefore has no true intent to deceive, what does it matter that there was no true deception, if me interacting with that thing still results in me having false beliefs that benefit some goal that the thing behaves as if it has, to the detriment of my own goals?
The problem is that the rabbit-hole dwellers use the word “deception” to try to invoke some spooky ghost-in-the-machine or AGI or ASI or self-awareness or self-preservation-seeking vibes based on the observed behaviors.
My point is the observation is not support for such vibes referenced in og-parent post. You can’t just rearrange bits and hope for a spark of life. It is not gonna “come live.” The rabbit-hole dwellers keep trying to make everyone believe that. It is both amusing and tiring how persistent they are at this. It is understandable considering the money they stand to gain from gullible people rushing to invest.
I’m pretty sure most of the people you are talking about aren’t confident that, in the scenarios they describe, that the computer would have any kind of interiority. I believe most of them believe that it is in principle possible for a program to have interiority (if doing e.g. full brain emulation, which is very difficult from LLMs), but that it is probably not necessary for it to do the things they are concerned about.
If Napoleon Bonaparte had been replaced with a p-zombie he would have been no less capable of conquering.
If something outwardly acts the same way as it would if it were a person, then the external effects are the same as they would be if it were a person. (<- This is nearly a tautology.) This doesn't mean that it would be a person, but it does mean that the concerns one would have about what outward effects it may cause if it it was a person, still apply.
(Well, assuming an appropriate notion of "outwardly" I guess. If a person prays, and God responds to this prayer, for the purpose of the above, I'm going to count such prayer as part of how the person "outwardly acts" even if other people around them wouldn't be able to tell that they were praying. So, by "outwardly acts", I mean something that has causal influence on external things.)
If something lacks interiority and therefore has no true intent to deceive, what does it matter that there was no true deception, if me interacting with that thing still results in me having false beliefs that benefit some goal that the thing behaves as if it has, to the detriment of my own goals?