A recent study in PNAS suggests that we can at least start thinking about that through inversion – the study of what intolerance is. By looking at the areas of the brain that activate during polarized responses etc a group of researchers are now arguing that intolerance is strongly correlated with a need for certainty.
If polarization is the response to uncertainty in our society, we can readily see how the increasing complexity of societies, and the weaker ability we have to predict the future, contribute to dividing us.
But why do we react this way?
If we take an evolutionary stance to the problem it is not hard to see situations in which the best reply to uncertainty is not Julia Galef’s scout mind, but rather the solider mind that narrows the options and favors belief over doubt – any kind of threat, still unidentified, from the forest or, say, a sound in the middle of the night, should not awake curiosity as much as a strong set of beliefs about that threat – wrong or right.

What is intriguing is that it does not matter what the belief is. If you meet social uncertainty with a strong national-conservative set of beliefs or if you meet it with socialist convictions; nature does not care what you believe, it is only interested in making sure you believe something, because doubt will kill you.
Beliefs will kill you too, if you believe the wrong thing, but in many cases it does not matter if you believe it is a sabre tooth tiger in the forest or an enemy tribe, it only matters that your response is the same. The content of the belief is immaterial to the evolutionary outcome – you leave and survive.
If you choose to believe it is just a branch snapping, and stay – then you run the risk of being eaten, of course, but here is the beauty of the fact that both left and right-wing polarization exists – could evolution have favored randomized belief as to divide groups up in the face of uncertainty?
Whoa! This is speculation across many dimensions – not least suggesting that there is a group selection element here – but entertain the thought for a bit, acknowledging that it is fanciful. What if evolution divides groups in the face of uncertainty to ensure that survival is not dependent on a single belief?
And you really do not need to imagine that there is group selection going on; you could just assume that we have a threshold to beliefs in the face of uncertainty that triggers if too many people seem to believe the same thing or not many people enough seem to believe it. It would be the opposite of peer pressure — peer opposition, built into our natures – an evolve tendency to disagree to a degree.
If nothing else it suggests that uncertainty – not probability – is still dominating our societies, now through polarization, even if the economist profession has done away with uncertainty as a concept and replaced it with calculable risk.
I have not read Galef etc, but regarding the concepts of scout and soldier, as I understand them from a couple of things you have written, I think the optimal course, for those who are able, is to cultivate both attitudes/reaction patterns, and awareness of them, in your personality. So as to have the choice of how to balance them depending on the situation. For the last maybe thirty years (I used to be mostly scout) I have done that, and find it very profitable. In general I run them in parallell as much as I can, and when it seems appropriate I tilt to one side or the other. Also, I find the Big Five personality model, and the the fact that where you as an individual end up on those dimensions seems to be roughly 50/50 nature and nurture, to be a useful tool in trying to understand most aspects of human behaviour.