The AI Watchdogs are Coming

Please share the love with your friends and colleagues who want to stay up to date on the latest in artificial intelligence and generative apps!🙏🤖

Share Aspiring for Intelligence

There’s a point in every new platform shift when the tech hype begins to capture the zeitgeist, and the regulators come knocking. The Telecommunications Act of 1996, the first significant overhaul of United States telecommunications law in 60 years, coincided with the birth of modern browsers and the resulting explosion in Internet usage. As social media grew in scale and reach (not to mention the Facebook / Cambridge Analytica scandal), the EU implemented major data privacy laws in 2018, followed shortly by California in 2020.

Now today, with ChatGPT and similar AI applications being used daily by millions, artificial intelligence is again in the crosshairs of policymakers and regulators.

In the past two weeks alone:

Clearly, there is plenty of excitement around the benefits of AI (of which this Substack has been a proponent!). But we would be wilfully negligent if we didn’t balance that excitement by detailing some of the risks and harms associated with AI proliferation, and what the resulting policy could look like. While there are plenty of good arguments against AI regulation, particularly that foreign adversaries are unlikely to halt or even slow down their advancements in the space, our view is that it is better for founders and operators to be aware of what may come and prepare accordingly. We agree with David below that the government should help shepherd a path, not “create roadblocks”.

In this post, we will discuss the significance of data privacy in the context of LLMs, the potential associated risks, and how companies can maintain a competitive edge by utilizing their own proprietary data without it getting incorporated into other LLMs.

Let’s dig in.

twitter.com/davidu/status/1645813574361837568?s=20

Two of the primary arguments in favor of AI regulation revolve around:

Of course, there are other larger concerns raised about the specter of AI, particularly the existential risks potentially posed by AGI, but we’ll focus on the above two real and immediate concerns in this post.

Data privacy refers to the protection of personally identifiable information (PII), which could include financial information, health information, social security numbers, home addresses, and more. In the digital age, data privacy has become increasingly important as more companies and organizations collect information and share it across online platforms (think about all those targeted ads you receive!).

In the specific context of LLMs, these models are trained on large public datasets (e.g., C4, The Pile, The Bigscience Roots Corpus, OpenWebText, LAION, and more). However, in order to improve the models, the LLMs also collect data from user interactions. For example, when a user asks a question to ChatGPT, they often input additional context or information to the model. In doing so, ChatGPT is collecting potentially proprietary data or information, which the LLM then feeds back into its system to create better outputs.

Misuse of data can result in:

Infringement of data privacy is ultimately what led Italy to take the drastic measure of banning ChatGPT, citing that OpenAI was processing users’ local data in breach of GDPR. OpenAI responded by geoblocking ChatGPT in Italy, and there’s no clear indication as to when it will be unblocked. While we believe Italy is on the extreme side of the spectrum for fully banning ChatGPT, it’s clear that perception of data privacy infringement can lead countries (mostly in Europe) to take drastic action (e.g. see Germany).

A second major concern raised by AI skeptics is the potential for heightened discrimination and bias. AI algorithms are only as unbiased as the data sets on which they are trained. If the training data reflects historical discrimination or biases, the AI system may perpetuate and amplify these biases. For example, AI systems used in hiring processes may inadvertently discriminate against certain groups of people, such as women or minorities, if the training data is biased toward past hiring patterns. In 2022, the US Equal Employment Opportunity Commission (EEOC) issued guidance for employers on the shortcomings of AI-based hiring tools (such as evaluating a candidate's facial expressions which can perpetuate bias). Similarly, AI systems used in criminal justice may reflect historical biases in sentencing, leading to unjust outcomes.

AI can also enlarge and hasten the spread of misinformation in a variety of ways:

A recent real-life example of AI run amok happened in Venezuela, where AI video tools like Synthesia were used to create fake American newscasters which were spreading disinformation. It’s not out of the realm of possibility that we will see more of these kinds of “fake campaigns” run globally.

“Noah”, an AI avatar posing as a journalist on Youtube promoting Venezuela

There are several regulatory frameworks related to data privacy around the world, including the European Union's General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) in the United States. These laws have been created to protect individuals' personal data from being misused or mishandled by companies; however, LLMs are now increasing the scope and impact of data privacy regulations are likely to expand. A few laws and frameworks include:

twitter.com/nonmayorpete/status/1643012255884386305

As enterprises consider how they can create a competitive advantage against their competitors, much of it comes down to establishing a data moat, or a data flywheel. However, with LLMs, it has become increasingly easy to inadvertently share information. This can happen when proprietary source code is accidentally shared for a scan in GitHub CoPilot, or when an employee asks ChatGPT to create an internal memo for a new product launch.

As companies start leveraging these LLMs across a variety of different use cases including 1) Leveraging LLMs off the shelf (e.g., OpenAI API); 2) Leveraging LLMs but fine-tuning the models by bringing in their own proprietary datasets (e.g., HuggingFace models + proprietary datasets); or 3) using GenerativeAI like tools such as ChatGPT, GitHub CoPilot, and JasperAI in-house, companies should consider how to create an AI governance strategy.

Here are a few key questions to consider:

We are starting to see companies emerge in this category that are helping create frameworks around AI Governance strategies and to ensure AI systems are safe:

Credo AI Framework

As of today, only Italy has taken the step of actually banning an AI application (ChatGPT), and most countries do not have an AI-specific regulatory structure in place. Some nations like India have explicitly opted against regulating AI from a government level today, identifying AI as too “significant and strategic”.

While there are no concrete indications of exactly what regulations will look like, we believe there are still several steps AI companies can take to prepare ahead of time and stay in the “good books” of policymakers:

Below we highlight select private funding announcements across the Intelligent Applications sector. These deals include private Intelligent Application companies who have raised in the last three weeks, are HQ’d in the U.S. or Canada, and have raised a Seed - Series E round.

We hope you enjoyed this edition of Aspiring for Intelligence, and we will see you again in two weeks! This is a quickly evolving category, and we welcome any and all feedback around the viewpoints and theses expressed in this newsletter (as well as what you would like us to cover in future writeups). And it goes without saying but if you are building the next great intelligent application and want to chat, drop us a line!