When Civilization II came out, I spent an entire summer playing it for several hours a day. The only check on my binging was the fact that my parents would eventually come home and force me to pretend, for a few hours at least, that I cared about things other than Civilization II.
I was a senior in college when Civilization IV arrived. I'd barely played strategy games at all for the previous four years, and "senioritis" brought with it a case of intense nostalgia. I bought it in the spring before graduation. It was still consuming my days and nights when the leaves fell later that year.
That was probably the last time my enjoyment of a 4X game was pure and uncomplicated. Lately, I've been wondering where that joy has gone, and why so few games seem to add anything essential to those old experiences.
Few 4X games have a point of view or attempt to tell a story beyond "claim territory, make progress, get stronger than your neighbors". They're not even very interested in what one victory path implies versus the others. These endgames are supposed to represent the culmination of meaningful choices, but are themselves almost entirely meaningless. That lack of purpose or vision is palpable in games like Beyond Earth, the 4X equivalent of a shrug, or in the dull fantasy worlds of a half-dozen 4X pretenders.
Fundamentally, 4X games are stagnating because they are ultimately games about progress that nevertheless have nothing whatsoever to say on the subject. Their version of progress is almost universally boring.
Most 4X games tell the following story: a society gets more and more advanced tools and infrastructure to satisfy the same basic needs throughout its history. Its competitors go through a similar or identical progression so that relative gains remain fairly small. Meanwhile, the society itself changes hardly at all. Where it once needed "5 food" now it needs "50 food" but it's fundamental relationship to the resource or the means of production hasn't evolved one iota.
Progress is more interesting and complicated than that in reality, and games can embrace that rather than reduce it to a series of buffs. Two important games provide examples of promising approaches.
The first, predictably, is Alpha Centauri. Now, Alpha Centauri is fundamentally a successful con: it's using flavor text and cutscenes to create a complicated and ambiguous context around the theme of progress, despite the fact that its (very traditional) mechanics do little to support its narrative. It gets away with it because that narrative flavoring can make the humdrum building-blocks of 4X progress feel alternately thought-provoking, chilling, or humorous.
This is what Alpha Centauri says about a building whose fundamental purpose is to reduce unrest / disorder in a city:
"The entire character of a base and its inhabitants can be absorbed in a quick trip to the Rec Commons. The sweaty arenas of Fort Legion, the glittering gambling halls of Morgan Bank, the sunny lovers’ trysts in Gaia’s High Garden, or the somber reading rooms of U.N. Headquarters. Even the feeding bay at the Hive gives stark insight into the sleeping demons of Yang’s communal utopia."
Look at the levels on which that bit of text is operating. It's telling me what the hell I just built, beyond its mechanical effect. If I don't get that kind of feedback, then the building might as well be called "unrest reducer, Tier 2" and suddenly the game could be any other 4X. Second, the flavor text ties into the game's factional identities. It's another window into who the Morganites really are besides just the "corporate faction". By telling me what the Gaians are building besides a hippie commune, this passage adds detail to its setting while letting me decide what I think about the Gaians' ambitions. Finally, it forces a bit of reflection on both what these structures imply about each faction, but also what your own entertainments say about me.
And all I did was build a new building.
Progress is also hard to control or predict. Few games understand this like Victoria 2, which isn't a traditional 4X but is such a trove of exciting ideas that it should be better-known and appreciated than it is. Set during the height of industrialization and the run-up to World War 1, Victoria is game that encompasses both classic 4X themes (improving technology and economic capacity) and the things 4X games struggle to address (civil society, politics, colonialism). Instead of giving you generic citizens who work resources and produce goods, Victoria creates a vibrant ecosystem of social classes, political beliefs, and economic needs. As citizens sort themselves into these categories, you get benefits but also a new set of needs and costs.
A rising capitalist class may help you build factories, but they'll also want to reap the benefits and exert some political influence over the direction of the country. Staffing those factories requires a mix of blue-collar workers and middle-class managers… both of whom require greater investment in education. However, as "workers" become a greater portion of the population, and education rises across the country, new political movements awaken. Your own citizens start competing for larger slices of the pie that you were growing for yourself. You just wanted more factories to generate more wealth (and maybe more guns) but you ended up with Socialist and Liberal revolutionaries at the palace gates. Progress changed both the nature of the country you're leading and your ability to direct and shape it.
Victoria's fundamental insight is an important one: progress reshapes societies around the new needs it creates. That's a fascinating theme that's applicable to a lot of 4X games, but that hardly any of them touch.
Dull, rote progress undercuts every aspect of 4X games. Combat scarcely changes except that the statistics on units and weapons get a little larger, so that the war you fight at the dawn of history is almost exactly the same as the war you'll fight at the end. Once settlement is over and development is underway, it becomes all too easy to hit "end turn" a few hundred times until the score screen, because there's little to admire or enjoy in the process of building. Friends and rivals become abstract collections of math to be manipulated with trade offers and military strength values, rather than fellow actors on a heroic stage. You're all there competing to build a better civilization, but with no idea of what you're actually building towards.
In the end, the 4X genre needs to do more than get out of the shadow of Civilization and Master of Orion. Each new 4X needs to find a story to tell about conquest, survival, exploration, discovery, or all of the above. A game will be a lot more exciting if it's built around those ideas and concepts rather than around the bones of old, familiar games.