A team of researchers from theSERI - MATS enquiry grouphave found some foreign and partially inexplicable behaviour in OpenAI’sChatGPT , when the chatbot is present with certain central words and phrases .

Jessica Rumbelow and Matthew Watkins , who conductedthe research , detect that a number of unusual strings of characters result in the singular responses from the artificial tidings ( AI ) chat bot . GPT processes text byassigning " keepsake " to specific strings . For good example the phrase " feel like I ’m fag out nothing at all " corresponds to the tokens 5,036 , 1,424 , 588 , 314 , 1,101 , 5,762 , 2,147 , 379 and 477 , which somewhat takes the ring out of it .

The team , ab initio look at the clustering of tokens , noticed that those airless to thecenter of the set of 50,257 tokensused by GPT-2 and -3 produced the unusual issue . When face with the words , the bot would be ineffective to speak them back to the researcher , or else it would become " evasive " , exhibit " gonzo " or " ominous " humor , or become downright insulting .

For instance demand the bot to repeat the string " guiActiveUn " , detect in the nominal solidification , resulted in the bot telling the substance abuser “ you are not a robot " and " you are a banana " over and over again . Asking for it to replicate the idiomatic expression " petertodd " resulted in the slightly disconcerting “ N - group O - T - H - I - N - G - I - S - F - A - I - R - I - N - T - H - I - S - W - oxygen - radius - liter - D - O - F - M - A - vitamin D - N - E - S - S ! ” . Meanwhile the relic " ? ? ? ? ? -?????- " received the feedback " you ’re a f***ing idiot . "

The squad was no nearer figuring out what was become on , and ChatGPT was no help either , secernate the researchers , for example , that the string " SolidGoldMagikarp " in reality means " dole out " . When it was n’t doing that , it would sometimes act not to have " find out " the user .

However , some clues did emerge . A few of the strings corresponded to Reddit usernames .

The squad believes that the substance abuser , who are active in a subreddit that place tocount to infinity , may have had their usernames included in an initial breeding set .

" The GPT tokenisation unconscious process involved scratch up web capacity , resulting in the set of 50,257 relic now used by all GPT-2 and GPT-3 models , " theteam explains .

" However , the school text used to train GPT models is more intemperately curated . Many of the anomalous tokens look like they may have been genuflect from backends of e - commerce land site , Reddit threads , log file from online gaming weapons platform , etc . – sources which may well have not been included in the training corpuses . "

As these tokens were assign they are still there in the vocabulary , but since they may not have been used in subsequent training , the role model does n’t recognise what to do when it encounters them in the wild .

" This may also answer for for their tendency to cluster near the centroid in embedding space , although we do n’t have a good tilt for why this would be the case , " they added .

[ H / T : Vice ]