Our Work is Low Skill Expression

The amount of skill behind an outcome is often impossible to discern by scrutinizing the outcome itself, and our goal of making AI go well may be an extreme case. Shaping history is like poker: high variance, small edges, low skill expression. If that is right, two things follow: 1) we cannot trust how things seem to be going, good or bad; and 2) we should stop concentrating effort on whatever looks best right now, and instead should hold a diversified portfolio of bets, including those which seem suboptimal.Beginning with poker, where the link between skill and results is actually measured: A strong professional’s edge over a merely competent player is something like five percent of the money wagered. The edge is small because most of the game’s skill sits in basic competence. A player who folds weak hands and bets strong ones has already captured most of what reckless play gives away. Expert skill is a thin refinement on top. The variance, meanwhile, is huge relative to the edge. Below roughly a hundred thousand hands, luck swamps skill, and the weaker player wins sessions all the time. So even in a game with a perfect scoring rule, it is difficult to discriminate between good players in all but the largest sample sizes. It is difficult to determine which habits and individual choices make money. It is hard to determine whether one’s approach needs to change, and in which direction.Our goal is harder than poker in three ways. First, we get one hand. Poker's remedy for variance is volume, and history offers none. Humanity lives through the emergence of powerful AI once. Second, the variance is worse. Philip Tetlock spent decades scoring expert predictions about world events. Forecasting skill is real at short range and decays with distance; a few years out, even the best forecasters drift toward chance. Reality is often very surprising. Third, many of the challenges on the road to our goal are not technical, but political, and thereby often i