How does the NN encode the fact that when you eat "with" a fork, the word "with" isn't the same sort of "with" as eating something "with" anchovies.
By looking at the noun in the NP embedded in the PP rather than the preposition. Whether this actually happens depends on various factors:
- In some dependency annotation schemes it's actually the head of the NP that attaches to the head with the prepositional phrase relation. Though this is uncommon, since typically the preposition is the direct dependent of the head.
- In some transition systems, attachment of a dependent is done when all its dependents are attached. So, at the moment the preposition is attached to a head, you already know what the preposition governs.
- In some transition systems, previously-made attachments can be changed (e.g. when you encounter a noun in a PP which suggests a different attachment).
- You might already look ahead in the buffer, either by having a small number of buffer positions as the input of the NN or by forming a representation of the current buffer using an RNN.
tl;dr NN can definitely learn such things, but you may have to help a bit by picking the right transition system or classifier inputs.
By looking at the noun in the NP embedded in the PP rather than the preposition. Whether this actually happens depends on various factors:
- In some dependency annotation schemes it's actually the head of the NP that attaches to the head with the prepositional phrase relation. Though this is uncommon, since typically the preposition is the direct dependent of the head.
- In some transition systems, attachment of a dependent is done when all its dependents are attached. So, at the moment the preposition is attached to a head, you already know what the preposition governs.
- In some transition systems, previously-made attachments can be changed (e.g. when you encounter a noun in a PP which suggests a different attachment).
- You might already look ahead in the buffer, either by having a small number of buffer positions as the input of the NN or by forming a representation of the current buffer using an RNN.
tl;dr NN can definitely learn such things, but you may have to help a bit by picking the right transition system or classifier inputs.