Note — Sep 26, 2021

Making Common Sense

Seen in → No.189

Source →

I’m somewhat dubious of the level of enthusiasm the author shows for OpenAI’s neural networks CLIP and DALL-E. But beyond that, it’s an intriguing consideration of logical, iconic, and distributed representation, how humans use them in different ways, how perhaps they are a good framing for different machine learning implementations, how common sense is used in these reflections but also in our daily lives.

This way of representing the world is less familiar than logical and iconic, but arguably the most common. It treats common sense not as a matter of knowing things about the world, but a matter of doing things in the world. […]

Both kinds of representation are expressive, but logical representations cannot capture relations between elements without adding more information, whereas iconic representations cannot depict elements non-relationally. […]

These distributed representations, in this sense, have a kind of tunnel vision — they represent what elements are most essential for the task and leave out the rest. But this goes for both biological and artificial networks, as well as logical and iconic representations; no representation can represent everything.