I mentioned this AI scam to my mom last night and she agreed with me that a passcode is great to have but even better is to have it prompted in such a way that it avoids a passcode being asked for in an obvious way. Kind of how you see in movies where spies have a conversational phrase that needs a specific weird answer.
Beyond that i doubt that an AI would be able to replicate my sense of humor. Whenever i speak with my mom i’m usually joking, making puns, etc so if she were to ever get a call from me when i’m all business with her she should be really suspicious.
With a challenge and password system you embed the password in a sentence with other plausible password-words. In the sentence “The budgie yells at the baboon every fortnight”, only one word in the chain may be the password. But if you were to overhear “I sure could use a baboon right about now” it would be more obvious. You can also have a duress password, for use in circumastances where you fear your challenge or password has been breached.