a unified theory of language acquisition and evolution
Here are some thoughts on linguistics. I haven't checked all the details, but it all seems to hang together and I think it's a good solid foundation to build a new theory of linguistics on.
Please help me finish the theory!
Erica: I've added you to this thread so that you can contribute some knowledge of classical theoretical linguistics to the discussion. Feel free to rope in any other linguists who you think might be better able to kick the tires of the theory, so to speak. If you want to make sure you're current with all the latest stuff, maybe go ahead and subscribe to math@moomers.org, my official listhost for sharing my mathematical ideas with the world.
Some more thoughts: probably the easiest way to get validation for this theory is to build a state-of-the-art chatbot using a dead simple architecture. If the theory is as good as I think it is, this should lead to a terrifyingly good chatbot that will (in the limit over time) approach passing the Turing test.
I recommend using the tensor2tensor library with the transformer model and byte-pair encoding. The chatbot should be hooked up to a public-facing website such that people can converse with it and can reward or punish the chatbot whenever they like or do not like what it says. Since people are terrible, we will need to have some sort of trusted set of people who can log in with a password and chat with it; these "known good" interlocutors can dish out rewards that are of much higher value than that of randos. In fact, it might be the case that the reward signal from randos should be compared to the known-good reward signal and should be flipped if the system thinks the rando is trying to train the chatbot to say offensive shit.
Training data should be fed in in batches over a period of perhaps one hour, and then replaced with the next batch when the hour is up. Data should simply be transcripts... need to think about this more, but essentially the task we want to train the system on is to produce the next line of dialogue given the entire previous conversation. There is some work to do to figure out how to encode all of this in a way that works with tensor2tensor's existing shit without having to reinvent the motherfucking wheel all the time.
On Wed, Apr 4, 2018 at 8:17 PM, Eric Purdy epurdy@uchicago.edu wrote:
Here are some thoughts on linguistics. I haven't checked all the details, but it all seems to hang together and I think it's a good solid foundation to build a new theory of linguistics on.
Please help me finish the theory!
Erica: I've added you to this thread so that you can contribute some knowledge of classical theoretical linguistics to the discussion. Feel free to rope in any other linguists who you think might be better able to kick the tires of the theory, so to speak. If you want to make sure you're current with all the latest stuff, maybe go ahead and subscribe to math@moomers.org, my official listhost for sharing my mathematical ideas with the world.
-- -Eric
An updated draft of the paper is attached.
On Thu, Apr 5, 2018 at 12:04 AM, Eric Purdy epurdy@uchicago.edu wrote:
Some more thoughts: probably the easiest way to get validation for this theory is to build a state-of-the-art chatbot using a dead simple architecture. If the theory is as good as I think it is, this should lead to a terrifyingly good chatbot that will (in the limit over time) approach passing the Turing test.
I recommend using the tensor2tensor library with the transformer model and byte-pair encoding. The chatbot should be hooked up to a public-facing website such that people can converse with it and can reward or punish the chatbot whenever they like or do not like what it says. Since people are terrible, we will need to have some sort of trusted set of people who can log in with a password and chat with it; these "known good" interlocutors can dish out rewards that are of much higher value than that of randos. In fact, it might be the case that the reward signal from randos should be compared to the known-good reward signal and should be flipped if the system thinks the rando is trying to train the chatbot to say offensive shit.
Training data should be fed in in batches over a period of perhaps one hour, and then replaced with the next batch when the hour is up. Data should simply be transcripts... need to think about this more, but essentially the task we want to train the system on is to produce the next line of dialogue given the entire previous conversation. There is some work to do to figure out how to encode all of this in a way that works with tensor2tensor's existing shit without having to reinvent the motherfucking wheel all the time.
On Wed, Apr 4, 2018 at 8:17 PM, Eric Purdy epurdy@uchicago.edu wrote:
Here are some thoughts on linguistics. I haven't checked all the details, but it all seems to hang together and I think it's a good solid foundation to build a new theory of linguistics on.
Please help me finish the theory!
Erica: I've added you to this thread so that you can contribute some knowledge of classical theoretical linguistics to the discussion. Feel free to rope in any other linguists who you think might be better able to kick the tires of the theory, so to speak. If you want to make sure you're current with all the latest stuff, maybe go ahead and subscribe to math@moomers.org, my official listhost for sharing my mathematical ideas with the world.
-- -Eric
-- -Eric
participants (1)
-
Eric Purdy