Promethean Shame

There is a well-documented scholarly theory about humanity’s relationship with technology: that people are so ashamed of their own failures compared to the ‘perfection’ of machines, that humans experience Promethean Shame. As a result, in engaging with new technologies, societies and institutions are more likely to adapt their own behaviours to those that are optimised for a machine (as compared to making whatever is new adapt to their way of life, and the outcomes they want). Over the past 40 years, humans have adopted behaviours and forms of interaction that suit computers rather than human-centred goals: Meetings are run by powerpoint slides; emails are exchanged with speed rather than clarity and thought; extremist behaviour has become normalised on social media; disinformation is passively accepted as a persistent presence. Each of these is a reflection on the perceived benefits that societies believed that each new technology would bring. The military is no different to other parts of society in this regard: many processes have changed because of the arrival of computers – from acquisition and procurement, teaching and strategy, to decision making and targeting. Uniformed personnel, like the civilian societies on which they are drawn, have optimised their human behaviours to suit the performance of machines.

It hasn’t always been this way: the arrival of horse archers, the long bow, rifles, artillery, tanks, submarines, or aircraft did not bring immediate revolution as espoused by their makers, but rather through military commanders determining how they would be used for warfare and combat, not in a way that optimised their own performance. They were commanded rather than simply used.

The arrival of Artificial Intelligence within the last 5 years continues to shape societies in unpredictable ways, primarily because humans are using AI rather than morphing it to suit themselves and their needs. Agentic AI, and soon Artificial General Intelligence, are following the same form as users (military officers) are given tools created by technology companies and told what they can deliver. Processes, procedures, and functions change to fit around these new software products. Often, they do not serve the demands of war, of combat, or of the people they are supposed to assist. Much like management consultants, squeezing an organisation around a process that is perceived to have wide ubiquity is accepted and implemented without considering the long-term implications, or even the medium-term consequences that arise. Regarding AI systems, the immediate implications stretch from faster but worse decision-making to increasing the number of people required to make the entirety of the process function. Longer term consequences are likely to have more significant impacts: how to train commanders when there are no staff duties left for them to cut their teeth on. The idea that a commander understands the processes that provide them with the evidence for their decisions has been an accepted characteristic of ‘good’ commanders for millennia. When staffs are replaced by AI, agents, or AGI, future leaders (in as little as a decade) will lack the experience to understand and judge recommendations as feasible, desirable, or verifiable. Accepting machine-determined answers will become normalised, because there will be no alternative.

That does not mean that command and control systems should not exploit the opportunities offered by AI. Instead, it means that C2 needs to be conducted as a function where the commander commands AI and does not simply use it. Reversing the accepted doctrine of how militaries (and societies) adapt around technology, to one where technologies (including AI and agents) become part of a staff process that suits the commander’s intent and demands. By reversing the dynamic around the relationship between humans and technologies more useful and effective operations are likely to emerge.

How to do this is important too. This is not simply an issue with using better (cleaner) databases or adapting the programming of agents. Rather this about the foundational code to be employed. Instead of accepting what is offered by technology companies as a ‘good’ product, operators need to define what they want and demand it is delivered – not something that is interpreted by tech leaders and adapted to what they believe militaries want and then adapting around it. A deeper relationship between commanders and C2 systems is needed if command is to become as technology centric as modern narratives seem to be demanding.