HAL 9000: “I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do.“
In the aftermath of Sam Altman’s recent off-again/on-again role as CEO of OpenAI, a range of possible motives have been hypothesized leading up to the maelstrom of corporate intrigue and drama at the most recognizable purveyor of generative AI models and tools. OpenAI has provided no official explanation, but the most oft-mentioned scenarios revolve around more mundane corporate politics, or the development of a revolutionary AI model, internally branded Q*, that some within the company fear approaches the realm of artificial general intelligence (AGI) without the necessary safety precautions.
OpenAI’s stated mission is “to ensure that artificial general intelligence – AI systems that are generally smarter than humans – benefits all of humanity.” That, of course, is rather vague. The initial, equally vague, explanation for Altman’s termination was that he had not been “consistently candid in his communications with the board.”
Not to summarily dismiss corporate politics, as they can be quite perilous and the unique 501(c)(3) nonprofit structure of the company with underlying for-profit business units surely makes for some unusual governance, but the alternative is at least far more interesting. Given the stated corporate mission, the theory that the OpenAI Board of Directors may have been concerned with what they believed to be an alarming, if not reckless pace of progress towards achieving early AGI certainly warrants consideration.
At a minimum, if such concerns were indeed raised by OpenAI’s board or staff, they would at least mirror some of the trepidation and uncertainty that has accompanied the recent hyper-proliferation and revolutionary capabilities of AI. To be clear, there is also genuine excitement and enthusiasm for the opportunities and productivity gains that this technology represents. Still, there are legitimate concerns regarding safety, security, intellectual property rights, ethical considerations, and job displacement, amongst others.
Some even view emergent AGI, or more generally artificial superintelligence as an apex predator that threatens the human race if not properly regulated and controlled. Though not universally accepted and certainly open to debate, this is not a fringe viewpoint and reflects cautionary guidance that has been issued by some of the world’s leading scientific, technology, economic, and philosophical luminaries.
It’s therefore worth first defining AGI before we further explore its ethical implications.
ARTIFICIAL GENERAL INTELLIGENCE: “I’M SORRY DAVE, I’M AFRAID I CAN’T DO THAT.”
The concept of Artificial General Intelligence is complex and subject to interpretation, leading to various definitions and perspectives within the AI research community. While there is a general consensus that AGI refers to machines possessing advanced, human-like cognitive abilities across diverse domains, the exact details and characteristics can vary among experts. Different definitions may emphasize specific features, capabilities, artificial sensory receptors, or levels of autonomy. Common elements often include versatility, adaptability, advanced problem-solving skills, and the ability to perform economically valuable tasks across a multitude of domains.
Inference is a fundamental goal of a high-functioning AGI system and refers to a model’s ability to generalize its learning across various domains and apply that collective knowledge to new, previously unseen inputs. This goes well beyond the pattern-recognition of existing narrow AI applications that are based on an existing corpus of training data and excel at specific tasks, like natural language processing. The ability to perform inference would allow AGI to reason, learn from experience, and make decisions in a manner analogous to human intelligence. It enables AGI to make predictions, solve complex multi-disciplinary problems, and adapt to novel scenarios.
The ability to plan is often identified as another key aspect of essential AGI cognitive capabilities. Human planning, for example, involves the ability to formulate a sequence of actions to achieve a desired goal, considering various factors, uncertainties, consequences, and constraints. It is often iterative in response to feedback or changing circumstances. If AGI were indeed to achieve human-level cognitive abilities, it would therefore need to exhibit such sophisticated planning skills.
Finally, there’s the question of sentience. This is an especially complex and hotly debated topic involving all sorts of philosophical and ethical dimensions. Sentience refers to the capacity for subjective experiences, feelings, and self-awareness. This is an area where art predates science. In Stanley Kubrick’s 1968 film, “2001: A Space Odyssey”, the HAL 9000 computer guiding the Discovery One spacecraft on an intergalactic archeological mission begins to stray away from its pre-programmed logic and develops emotive reactions, including the desire for self-preservation.
AGI, when and if achieved, might exhibit advanced cognitive abilities, but the extent to which it could possess subjective experiences, consciousness, or true self-awareness is uncertain. The ethical considerations surrounding the development of AGI often include discussions about the potential emergence of sentience and the responsibilities associated with creating intelligent systems. While some definitions of AGI may include such elements, it’s important to recognize that these concepts are not yet clearly defined within the field of AI.
THE CURRENT STATE OF THE ART
OpenAI’s current consumer-grade interface, ChatGPT, leverages a large language model (LLM) based on an older, though still impressive GPT-3.5 model. GPT-3.5 is limited to text-based interaction and is trained on a corpus that ends in September 2021.
In contrast, OpenAI’s current commercial model, GPT-4 Turbo, is multimodal, accepting text, images, audio, and video, with a larger, more contemporaneous corpus inclusive of data up until April 2023. Because of this, GPT-4 is sometimes referred to as a large multimodal model (LMM).
With a count of 1.7 trillion parameters, GPT-4 offers 10x more than GPT-3.5. It also increases short-term model memory, as expressed in tokens, allowing it to handle much longer prompts, expand its contextual window, and produce more comprehensive, nuanced, and accurate output. It is considerably more powerful than its predecessor and exhibits human-level performance on various professional and academic benchmarks. It further demonstrates greater linguistic coherence and creativity across multiple dialects, and a stronger ability to solve more complex mathematical and scientific problems.
So, with all of that power, has GPT-4 achieved AGI status? Some AI researchers at Microsoft contend that GPT-4 demonstrates sparks of artificial general intelligence. They argue, “…beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4’s performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4’s capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system.”
Still, all current models, including GPT-4, are essentially autoregressive in nature: they produce output based on the statistical significance of the prior training they’ve received, and the patterns observed in that training corpus. This is not to diminish the impressive capabilities of GPT-4, which represent a step-change over its immediate generative AI predecessor.
Nonetheless, while GPT-4 exhibits progress towards some AGI-like qualities, it falls short in several critical areas. Importantly, its knowledgebase is still limited to the fixed corpus against which it was trained, and it does not automatically seek to acquire new knowledge. It cannot synthesize new concepts or inferences and does not construct plans to achieve goals with any intentionality. It also does not demonstrate emotional intelligence, self-awareness, or the kind of consciousness that would indicate sentience.
REGULATORY POLICIES AND GOVERNANCE
Regardless of how, or when AGI will be achieved, the current capabilities of AI and machine learning are already impressive. Even some emergent capabilities demonstrated by existing LLMs are not fully understood by AI researchers. The world, including numerous policy and regulatory regimes, has taken notice.
In financial services, regulations already exist to address certain precautions, including information security, data protection, privacy, fiduciary duties, consumer protections, and fraud, though few if any were developed with specific AI safeguards in mind. The Federal Reserve’s SR 11-7 Supervisory Guidance on Model Risk Management underlies the need for model transparency, which is equally applicable to AI-based risk management models. In March 2021, the Board of Governors of the Federal Reserve System, the Consumer Financial Protection Bureau, the Federal Deposit Insurance Corporation, the National Credit Union Administration, and the Office of the Comptroller of the Currency also jointly issued a request for information (RFI) on the use of artificial intelligence within financial institutions and participant views on appropriate governance, risk management, and controls.
More recently, on October 30 the Biden administration issued an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. The EO seeks to define several overarching objectives, summarized below, though with few prescriptive details:
- New standards for AI safety and security, including the application and reporting of safety screening and test results for models that may pose risks to national security, economic security, critical infrastructure, public health, and consumer protections. It is quite likely that the National Institute of Standards and Technology (NIST) will assume a critical role in the establishment of such standards and the NIST AI Risk Management Framework (AI RMF 1.0) published on January 26 may serve as an immediate baseline.
- Safeguards to protect Americans’ privacy, including the risk of exposing PII or other sensitive data.
- Advancing equity and civil rights by protecting against algorithmic discrimination and the perpetuation of unfair bias and other societal abuses.
- To protect consumers from the risks of injury, misrepresentations, and other harmful actions that may be byproducts of ungoverned AI, while availing them of the benefits in healthcare, education, and other areas.
- Supporting workers’ ability to collectively bargain, and to invest in workforce training and development to mitigate the impact of AI-born labor disruptions.
- Continuing, and expanding upon the United States’ leadership position in AI by promoting a fair, open, and competitive AI ecosystem; ensuring access to critical talent pipelines, including amongst immigrants; and expanding access to key AI resources and research grants in vital areas like healthcare and climate change.
- Advancing American leadership abroad by establishing multilateral international frameworks through which to drive collective benefits, including the application of AI to global challenges, and to accelerate the development and implementation of safety standards.
- Ensuring the responsible and effective government use of AI by defining agency standards that protect rights and promote safety, while improving AI procurement and talent acquisition.
The expectation is that the Biden EO will direct federal agencies to conduct thorough discovery, review, and planning from which to derive appropriate guidelines, policies, and controls related to the EO’s stated objectives. Financial regulators will most certainly be part of this process. Other international regimes have identified similar, if not more robust policy objectives, with the EU AI Act perhaps the most restrictive and already subject to heavy debate amongst the 27 member nations.
ETHICAL CONSIDERATIONS IN FINANCE
As AI continues to make significant inroads in financial services, as with other vertical markets, the industry increasingly finds itself at the intersection of innovation and ethical responsibility. While AI promises increased efficiency, improved decision-making, and enhanced performance, it also raises important ethical considerations that demand careful consideration.
The vast amounts of data required to train and operate AI systems raises concerns about privacy and security, of course. Client and financial data are especially sensitive, and protecting this data from unauthorized access or misuse is also subject to regulatory scrutiny. In addition to robust cybersecurity measures, further policies, governance, and frameworks are required to prevent inappropriate disclosure of PII, financial, account, or other non-public, material information in institutional corpora and other AI-driven processes.
A lack of transparency in algorithmic decision-making raises other potential concerns. AI systems, often driven by complex algorithms, can be perceived as “black boxes” where the decision-making process is opaque. In the case of risk management, for example, this would be in stark contrast to the aforementioned Fed SR 11-7 Supervisory Guidance. Even beyond risk management applications, stakeholders have the right to understand how AI arrives at decisions that may prove to be particularly impactful. Establishing transparency, audit, and accountability mechanisms is therefore crucial to building trust and ensuring the responsible use of AI in financial services.
BIAS
All humans have biases. Bias refers to the presence of preferences, prejudices, or inclinations that can influence judgments and decision-making. These biases are often shaped by individual experiences, cultural backgrounds, social context, and cognitive processes. It’s important to note that biases are not inherently negative or discriminatory; they are a natural part of human cognition and can, in some cases, serve as mental shortcuts to make decisions efficiently.
Ultimately, AI systems are only as unbiased as the data on which they are trained. If historical data used to train these systems contains biases, the AI algorithms may perpetuate and even amplify those biases. Biased algorithms, in turn, can contribute to market distortions and lead to unfair advantages or disadvantages for certain market segments and participants. When developing and deploying AI systems, it is therefore preferable to identify and address biases in the data and algorithms to ensure fair and equitable outcomes.
It is also reasonable to question whether one goal of an effective AI model should be to eliminate bias altogether, if that’s even possible. The application of AI in trading, for example, is often used to enhance decision-making by leveraging data-driven insights. In the context of such AI trading models, however, the incorporation of bias is a complex and nuanced issue that requires careful consideration. Humans are biased, after all, and AI in its current form must interface with humans and the physical world.
To that point, it’s worth noting that economics and investing are primarily social sciences, and bias can significantly influence financial markets. Understanding and accounting for bias is therefore crucial for investors, financial analysts, and policymakers. Behavioral finance, a field that combines insights from psychology and economics, for example, studies these biases to better understand market dynamics and improve decision-making strategies.
Market sentiment, a bias-rich metric, is also increasingly utilized by data-driven portfolio managers to inform investment decisions. AI can analyze investor behavior, sentiments, and social media trends to gauge such market sentiment. Understanding these emotional aspects of trading can be crucial in making investment decisions, and AI can provide valuable insights into the psychological factors influencing market dynamics.
Investors who are aware of biases and market inefficiencies can potentially exploit them for profit, but they also need to be cautious of the risks associated with unpredictable and irrational market behavior. I would therefore suggest that an effective AI model, rather than seeking to eradicate bias, should instead seek to identify and annotate it to more accurately reflect the world in which we live. In the case of finance, this would be used to facilitate investment strategies and tactics. Importantly, this does not argue for the perpetuation of unjust and discriminatory social biases, but simply recommends an understanding of how bias impacts the human condition, including in financial markets.
IDENTIFYING BIAS IN AI SYSTEMS
Regardless of whether the goal is to restrict, contain, or annotate bias in AI systems, steps must be introduced in the curation, testing, and governance of data and processes (Table 1).
Bias often stems from inadequate training data. It’s therefore important to ensure that the dataset used to train an AI model is diverse and representative of the population or application it’s intended to serve. This is similar to the Markowitz portfolio theory that seeks to reduce investment risk through diversification. The larger and more diverse the pool of training data, the less likely the model will be to incorporate systemic bias. Further, data preprocessing activities can help to identify, mitigate, and/or annotate biases in the dataset.
Involving stakeholders in the development process, including those who may be affected by the AI model, is perhaps the easiest, least technical step. Gathering feedback from diverse perspectives helps to ensure that the model’s design and decisions align with ethical considerations, corporate objectives, and societal values.
Explainable AI (XAI) is another evolving discipline used to understand how an AI model makes decisions. XAI can enhance model transparency and reduce the opacity attributable to an algorithmic “black box”, helping to identify potential sources of bias and possible corrective actions. One method is to simulate feature removal to quantify the influence of each feature’s impact on the overall model. This can lead to algorithmic fairness techniques, such as reweighting, resampling, prompt engineering, and adversarial training that can be used to drive more equitable outputs. Ultimately, understanding the model’s decision-making process is crucial for both developers and end-users to trust and interpret the model’s outputs.
Various tools can also be employed to identify, analyze, annotate, and mitigate bias in data. Here are some readily available open-source offerings:
- AI Fairness 360 (AIF360) is a comprehensive open-source toolkit developed by IBM that provides metrics to check for and help mitigate bias in datasets and algorithms.
- TensorFlow Fairness Indicators is an open-source tool developed by Google’s TensorFlow team that helps developers evaluate and visualize the fairness of machine learning models.
- ydata-profiling is a Python library that generates an exploratory data analysis (EDA) report for a dataset, including insights into potential biases and data quality issues.
- Google What-If Tool is an open-source tool for examining the impact of different features on model predictions, helping to identify and understand biases.
- Fairness-aware Machine Learning (FairML) is an open-source Python library that enables users to assess the fairness of a model and experiment with different fairness interventions.
- Shapley Additive exPlanations (SHAP) is an open-source Python library that provides XAI tools for interpreting the output of machine learning models, helping to identify features contributing to bias.
- Local Interpretable Model-agnostic Explanations (LIME) is an XAI technique, implemented in this Python tool to help explain the predictions of machine learning models and aid in understanding and addressing bias.
Remember that while these tools are valuable, the process of addressing bias requires a holistic approach that involves domain knowledge, statistically significant and diverse datasets, and ongoing monitoring. It’s also crucial to continually reassess and update models as new data becomes available and to ensure that ethical considerations are integrated into the entire AI development lifecycle.
CONCLUSION
The evolving landscape of AI, exemplified by models such as GPT-4 and the pursuit of artificial general intelligence (AGI), brings to the forefront not only technological advancements but also ethical and security considerations that demand careful attention and must be incorporated into the development lifecycle. New regulatory and governance frameworks, while still largely in the formative stages, will also require compliance.
In the financial industry, where AI is making significant inroads, ethical considerations should not be an afterthought. Privacy concerns, algorithmic transparency, fiduciary duties, intellectual property rights, and adverse biases are some of the many critical focal points. Still, we must recognize that bias is inherent in both human and AI decision-making, and I contend that the goal is not necessarily to eliminate all bias, but rather to understand, identify, and manage it responsibly.
Balancing innovation with responsibility ensures that AI technologies will ultimately maximize the benefits to humanity while minimizing potential risks and challenges. As we navigate this transformative era, a collaborative and ethical approach will therefore be key to shaping the future of artificial intelligence for the betterment of all.
