Beyond Hardcoded Evolutionary Psychology

Steven Byrnes has written quite a lot on brain-like AGI algorithms. I'll reiterate here of a small part of his work, but you'd be better off reading his stuff directly.For the ideas which inspired me, see here, here, and here.This post has good handles for intuitively applying neuroscience to rationality. Don't mistake my "simulator" cluster for a distilled and reduced concept; it's "lumpy rock", not "skewed prism of seafloor impactite from the late Devonian".My (loose) extension of Steve's models of human minds predicts that we'll get stuck in local minima of self-reinforcing nonreality; with this knowledge, we can more directly target the underlying mechanisms of cognitive biases.Consciously tracking what you're actually optimizing for is one example skill which seems worth developing despite its time cost. In fact, many rationality skills (like noticing your confusion and scout mindset) are species of this overlying stance.Let's take scout mindset as an example.While discussing something, the goal for rationalists usually isn't to convince the other person of their ideas, but rather to come to truthier beliefs.One should therefore be vigilant, lest they fall into a combative posture, since this makes them slow to update.Cool! Is there a principled way to train this?Do you mean trigger-action patterns?No; TAPs take conscious effort to implement. How do you make it an unconscious System 1 response, the same way I keep my balance while walking?Lots of practice.But what if I have stronger subconscious pressures pushing me away from scout mindset? Like, sometimes I want some idea to work, and then I notice myself defending it. But then it feels like something blocks me from seeing that defensive posture. It's like a part of my brain is trying to keep me blissfully unaware. You're telling me to run uphill, but I want to remove the hill entirely.If you keep practicing it, you'll learn to do it subconsciously! Then it won't take any effort, just like walking.But as an in