It remains a vivid memory, the astonishment I experienced when a volume authored by evolutionary biologist Peter Lawrence, titled The Making of a Fly, was listed on Amazon with a staggering price tag of $23,698,655.93 (plus a nominal $3.99 shipping fee).
While my global academic peers likely felt a sense of profound dismay at such an academic publication achieving such a monetary valuation, this exorbitant cost was, in reality, a consequence of automated systems interacting and escalating uncontrollably. It became evident that this was not a matter of creative pricing by sales personnel, but rather algorithmic decision-making dictating the outcome.
This particularly striking illustration was identified and rectified. However, one must consider the possibility that such algorithmic interferences transpire routinely, perhaps even in ways that elude our detection. If our lived experiences are increasingly being architected by algorithms, what then becomes of our human agency?
Motivated by these observations, my esteemed colleague, Prof Allen Lee, and I embarked on an investigation into the more profound ramifications of algorithmic technology. Our findings were subsequently published in a scholarly paper featured in the Journal of the Association for Information Systems.
Our research led us to a critical insight: the traditional dynamic between information technology and human involvement has undergone a fundamental inversion over time. Historically, humanity utilized technology as a mere instrument. Today, technological sophistication has advanced to a degree where it actively employs and, in some instances, governs us.
We are not only detached from the decisions being made on our behalf by automated systems but are also profoundly impacted by them in ways that are difficult to anticipate. Rather than occupying a central position within the decision-making frameworks that influence us, we have been relegated to the periphery of their operational environment.
We have incrementally diminished our own capacity for independent decision-making, ceding control to algorithmic processes. In essence, we are transforming into artificial constructs, or human artifacts, whose very essence is shaped and leveraged by the technology itself.
Numerous examples underscore this paradigm shift. Within the legal domain, legal analysis is gradually being supplanted by artificial intelligence, signifying that the successful adjudication of legal cases may partially depend on algorithmic evaluations.
Software has even been entrusted with the responsibility to forecast potential future criminality, thereby exerting influence over individual liberty by dictating the terms under which parole is either granted or denied to incarcerated individuals. In this capacity, judicial decision-making is being subtly influenced by computational engines whose complexity and data-intensive nature render them incomprehensible to human understanding.
In the professional sphere, an overdependence on technological solutions has prompted some of the world’s foremost corporations to process résumés through automated software, consequently preventing human recruiters from ever reviewing the credentials of certain prospective candidates. This not only places individuals’ career prospects at the discretion of machines but can also embed inherent hiring biases that the organization never intended to perpetuate, as was notably observed at Amazon.
Within the news industry, a process known as automated sentiment analysis evaluates public opinion regarding corporations by examining various online sources for positive and negative commentary. Subsequently, this analysis is leveraged by trading algorithms to execute automated financial transactions, obviating the need for human agents to interpret the news content directly.
Unintended Ramifications
Indeed, algorithms operating independently of human oversight now exert substantial influence over financial markets. For instance, an impressive 85 percent of all transactions conducted within foreign exchange markets are executed exclusively by algorithms.
The escalating algorithmic competition to develop increasingly sophisticated systems for market participation results in vast capital allocations being determined by machine-driven judgments.
On a more granular level, the developers of these algorithms possess the capacity to shape their functionalities and operational parameters.
However, a significant aspect of artificial intelligence involves programming software to autonomously ascertain methods for task completion. Consequently, we frequently lack a complete understanding of the underlying mechanisms driving their decision-making processes. As is characteristic of all technological advancements, this can precipitate unforeseen consequences that may vastly exceed the initial intentions of their creators.
Consider the “Flash Crash” that impacted the Dow Jones Industrial Average Index in 2010. The actions of algorithms contributed to the index’s most precipitous decline in its history, eroding nearly 9 percent of its value within mere minutes (though a substantial portion of this loss was recouped by the day’s close). A comprehensive five-month inquiry could only offer speculative explanations for the catalyst of the downturn (and diverse alternative hypotheses have been put forth).
Crucially, the algorithms that exacerbated the initial issues did not operate based on an error. There was no programming defect. The emergent behavior stemmed from the intricate interplay of millions of algorithmic decisions, each influencing the others in unpredictable patterns, ultimately following their own inherent logic to precipitate a market downturn.
The environmental conditions that enabled this scenario arose because, over time, the individuals overseeing the trading infrastructure had begun to perceive human judgment as an impediment to market efficiency. As far back as 1987, during a period when the US stock market experienced a significant 22.61 percent drop, some Wall Street brokers deliberately ceased answering their telephones to avoid receiving customer orders to sell securities.
This marked the commencement of a trajectory that, as articulated by author Michael Lewis in his seminal work Flash Boys, “culminated in computers entirely supplanting human involvement.”
The financial sector has made substantial investments in high-speed network infrastructure and microwave communication systems, all aimed at shaving infinitesimal fractions of a second from the time it takes for algorithms to transmit their directives. When such extreme speed is paramount, a human operator, requiring a considerable 215 milliseconds to execute a button click, becomes virtually obsolete.
Our sole remaining function may be to periodically recalibrate the algorithms whenever the existing technological decision-making framework falters.
As novel delineations emerge between human capabilities and technological prowess, it is imperative that we engage in careful contemplation regarding the trajectory dictated by our profound reliance on software. With human decisions increasingly being superseded by algorithmic ones, and our lives becoming shaped by machines and their unforeseen consequences, we are inadvertently positioning ourselves for technological subjugation.
It is incumbent upon us to determine, while the opportunity remains, the implications of this evolving dynamic for us both as individuals and as a collective society. 
Dionysios Demetis, Lecturer in Management Systems, University of Hull.
