These 2 terms are talked often in software development, philosophy of language, and machine learning. Sometimes they are used interchangeably and I’m not sure if that is on purpose or not, but their definitions are related and similar, but with slight differences.
Compositional means that the meaning of the whole expression or structure of something is made up of the sum of its parts. Natural language, programming languages, and mathematics are said to be powerful because they are compositional. You can take their finite atoms and combine them in infinite ways to generate new meaning.
Examples:
- “George Washington gave a speech to 30 red dogs on the deck of yacht while floating down the Colorado River.”
- 7 x 3 + 337 x 73.37 / 7 + 333777 / 73737.33773
- return x if (foo > bar && bar < alpha) || this_is_too_long == true
Complex meaning is emerging from the base parts (atoms) of those systems and if you understand each atom individually, you are supposed to understand the sum of them.
Composable means the parts of a system can be broken down into smaller components and that those components can be used in different contexts. Think about a water pipe system, the pieces are interchangeable and you can go to Home Depot and get standard parts to modify it. In software, code is often organized into modules and then those modules are called directly. Then the implementation details do not matter and we can change out the implementation details without effecting the program.
An analogy: Your house in a system with many electronic devices running. You can change out your utility provider and all your devices will continue to run and you don’t need to care how the electricity is being generated. It could be from coal, solar, or hydro, it wouldn’t matter. Composable in man-made systems often just means that the interfaces or connectors are standard and if they follow that, then everything can be connected without worrying about the details.
Something being composable does not guarantee compositional. For example you can combine: “:” and “)” to make a smiley face of ” :)”. Individually understanding those parts won’t give you understanding of a smiley face. The sum is greater than the whole.
Most often in AI research, people are talking about how complex meaning is derived from the atoms of simple expressions, in other words compositional. When I think about how concepts in the mind are stored, I imagine a large database of amorphous concepts that are interchangeably linked to each other. Each concept relies on many other concepts and the implementation details of those concepts don’t matter, but you can combine those concepts together to get new meaning. For example, think about the concept of “a marathon”. You must understand the concepts of “training”, “sports”, “running”, “recovering”, “competition”, “hydrating”, and more. If you understand the basics of recovering and running, you can potentially complete the marathon, but you do not need to plan in detail each of those steps, you can just follow a general plan. The training can be switched out to be treadmill running or outdoor running. Recovering could be sleeping more, taking running days off, jogging sometimes, or drinking a beer.
Some of those same base concepts for “marathon” can be used for other concepts like “hanging out with friends” or “something to do while off work”.
All of these concepts are built on top of other concepts and the details of those concepts don’t matter. They are loosely connected and as long as you know the high level idea of the concept (the interface or connector), then you don’t need to focus on the implementation details. (Is there an actual interface that these concepts must adhere to in order to be interchangeable?) You can just say “I need to drive to work” and not have to worry about traffic lights, accelerating, decelerating, breaking, pedestrians crossing the street, a car accident, how to cross main st, potholes, temporary route closures, trash in the middle of the road, sudden traffic, etc. Your mind will move into motion and deal with those details as they come up. Contrast that with a computer where every one of those potential scenarios will need to be built for ahead of time and tested before going into production.
The mind seems to be storing its information in a compositional and composable way that allow for infinite combinations. As of now, neural networks and machine learning systems are unable to do composition and it is thought to be one of the key problems in machine learning. Understanding how the mind is able to connect and store these concepts in an amorphous, abstract, and amodal way is key to understanding intelligence.