Richard Feynman famously wrote: “You have to keep a dozen of your favorite problems constantly present in your mind, although by and large they will lay in a dormant state. Every time you hear or read a new trick or result, test it against each of your twelve problems to see whether it helps. Every… Continue reading On Richard Feynman and your core questions
Artificial Intelligence and Cognitive Science researchers have taken for granted that compositionality exists in the human brain. Fodor , Chomsky, and many others have spoken about the human brain ability to create infinite meaning from a finite set of tokens (words). There are people who argue that the brain is not compositional. You can read… Continue reading Evidence of Compositionality in the human brain
The formal definition of “to move” is: “to change from one place or position to another” or “to set or keep in motion”. This moving usually happens by their own volition, but things can be moved by others as a “patient”. It should be easy to know if something is moving and all animal intelligence… Continue reading Representing Moving vs Doing
I just learned Gottfried Wilhelm Leibniz (1646-1716), famous for inventing calculus, was also interested in the philosophy of mind. He proposed a concept called the Alphabet of human thought, which is an idea that says there is a universal way to represent all human thought with small primitive components. With these primitive components, we should… Continue reading The Language of Thought hypothesis and Compositionality was worked on hundreds of years before computers existed!
can we just represent each prime as a point in a vector? We would need to have a programming language that could interpret the vector. So we are moving the meaning of the vector to the interpreter. We need to test compositionality. Can you have multiple vectors as a sequence to represent new concepts/words? The… Continue reading Testing compositionality on GCCKR
I’ve been studying Neural Module Networks from Jacob Andreas. He focuses on compositionality and grounding, my 2 favorite subjects. I just read 2 of his papers: https://openaccess.thecvf.com/content_cvpr_2016/html/Andreas_Neural_Module_Networks_CVPR_2016_paper.html https://openaccess.thecvf.com/content_iccv_2017/html/Hu_Learning_to_Reason_ICCV_2017_paper.html I’ll go over the first paper: “Deep Compositional Question Answering with Neural Module Networks” He has created a new type of neural network architecture where he builds… Continue reading Notes on Neural Module Networks (NMN)
Just read this short and sweet paper from Guy Davison et al. Brenden Lake is also a co-author, I have read several of his papers. paper link: https://psyarxiv.com/byzs5/ They built mini programs as DSLs that represent game rules. The programs act as reward generating functions (goals as reward-generating programs). Their line of reasoning follow the… Continue reading Notes on “Creativity, Compositionality, and Common Sense in Human Goal Generation”
“The meaning of a whole is a function of the meaning of the parts and of the way they are syntactically combined”. It seems there is not a clear consensus on how we would implement compositionality in computers. I have gone through several research papers and collected several different definitions. Systematic The most common definition… Continue reading A computational definition of grounded compositionality
Some of the most interesting ways to represent knowledge in computers using Mathematical structures. Graphs and graph neural networks multi-spacial grid cell representations grid representations probabilistic programming holograms Voronoi tesselations – they allow distance calculations between any concepts https://towardsdatascience.com/the-geometry-of-thought-700047775956 word2vec – its such as simple model, its worth thinking about how to scale this up.
I keep reading grounded language learning papers trying to figure out what is the minimal ingredient to achieve a grounded representation in computers. One theme I keep seeing and coming back to is that while everything is grounded, compositionality may need more abstract thinking. It seems like there definitely must be some recursive computation or… Continue reading Variables in grounded language learning