Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Sure, but GPT-3 doesn't attach semantics to text, it just learns how to produce text patterns that are similar to text patterns it has seen before.


"Similar" is an inherently semantic property.


Would you say that something like string.GetSimilarity(string), which tells you by what number of characters two strings differ, is interpreting the text?

Basically what GPT-3 does is to find a string X of a particular length such that it maximizes concat(userInput, X).GetSimilarity(someStringInTrainingSet).

Edit: to be clear, I'm not suggesting it's looking up the training set at runtime, X.GetSimilarity(someStringInTrainingSet) is basically what got baked in during training.


That is not what GPT does.


Words with the same length are similar to one another but not well correlated semantically. E.g. bog vs dog.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: