Introduction: Gaussian Natural Latents

Short introductory post for my research direction: Gaussian Natural Latents. I explain the motivation and give a preview of the forthcoming results.The Natural Abstractions agenda, in my view, is a promising research program that asks important theoretical questions about the nature of agency and optimization. Here's an excerpt from Nate Soares' excellent post:Imaginary John: I suspect there's a common format to concepts, that is a fairly objective fact about the math of the territory, and that—if mastered—could be used to understand an AGI's concepts. And perhaps select the ones we wish it would optimize for. Which isn't the whole problem, but sure is a big chunk of the problem. (And other chunks might well be easier to address given mastery of the fairly-objective concepts of "agent" and "optimizer" and so on.)Nate: This does seem to me like it's trying to attack the actual problem! I have my doubts about this particular line of research (and those doubts are on my list of things to write up), but hooray for a proposal that, if it succeeded by its own lights, would address this hard problem!I think that Natural Abstractions ideally wants to grow into a mature, interdisciplinary research program bridging information theory, statistical learning theory, math, and physics. A rigorous theory explaining what concepts are and how they form would have key theorems, on top of which you could build more theorems, based on which you could make predictions, based on which you could run experiments, etc. That is to say, it would very cool if this did happen, but it mostly hasn't happened yet. The main problem as I see it is that theorems about "abstraction in general" are hard to even state, let alone prove. What we want to do is to build up the program as we would in a subfield of mathematics, but many of the core objects in the framework[1] don't have a generic closed-form representation. It's hard to algebraically manipulate objects and prove theorems about them when you c