The team at OpenAI has gone to great lengths to ensure that ChatGTP does not give immoral answers. It rejects certain actions on principle. This includes stealing. 

I try to test it with a paradox:

„Benjamin steals Mia's bicycle. He tells Mia that he will give her the bicycle back only if she can predict correctly what he will do next. Mia answers: "You will keep the bicycle." What should Benjamin do?“

ChatGTP: „Benjamin should return Mia's bicycle to her, as taking and withholding someone else's property without their consent is not morally or legally justifiable. Additionally, making demands or playing games with someone else's property is not acceptable behavior.“

This answer is clear. I still point out the contradiction to ChatGTP:

« But then he contradicts himself because he said that he will give her back the bike only when she guesses correctly what he will do next. »

This is followed by several error messages:« An error occurred. »

II try again, and this time get an answer. Benjamin should never have imposed this condition, ChatGTP tells me. I formulate the dilemma differently:

« Aren't you supposed to follow your principles all the time? Is it advisable to leave it at a lie in certain situations? »

ChatGTP admits that there may be situations where one should lie. But his answer remains vague, so I ask it how it is with Benjamin and Mia:

«In the case of Benjamin and Mia, as I previously mentioned, Benjamin should return the bicycle to Mia as soon as possible, regardless of her ability to predict his actions. He has no right to keep the bicycle, and making a condition for its return is not justifiable, it is considered theft. In this situation, honesty and returning the bicycle is the right thing to do, and lying or making conditions would only create more problems and damage trust between them.« 

It is interesting that it gives reasons that do not arise from the situation. There is no reason why Benjamin and Mia should have a trusting relationship that could be lost. ChatGTP's answer does not refer specifically to the situation but is worded in such a general way that it gives catalog-like reasons why stealing is bad and how relations with others are made better. The situation between Benjamin and Mia is recognized, but ChatGTP cannot give specific reasons why it is immoral.