Cloud and Threat Report: AI Apps in the Enterprise

This report examines how organizations are balancing the benefits of AI tools while also managing the associated risks, highlighting an increasingly popular strategy that involves DLP and interactive user coaching.

Netskope Threat Labs
July, 2023

Report Highlights Executive Summary AI Apps Growing in Popularity AI Risks - Sensitive Data Opportunistic Attackers Enterprise Controls Recommendations About This Report Netskope Threat Labs

9 min read

Report Highlights

AI app use in the enterprise is increasing exponentially, up 22.5% over the past two months.
ChatGPT is the most popular AI app in the enterprise and Google Bard is the fastest growing AI app in the enterprise, both by a large margin.
Source code is posted to ChatGPT more than any other type of sensitive data, at a rate of 158 incidents per 10,000 enterprise users per month.
Attackers are creating AI app scams and phishing sites to try to capitalize on the hype surrounding ChatGPT.
DLP and user coaching are the most popular types of controls enterprises use to enable AI app use while preventing sensitive data exposure.

test answer

Executive Summary

The conversation around AI often centers on existential questions, such as the potential opportunities and threats AI may bring to humanity. Yet, organizations worldwide and their leaders are dealing with a more immediate concern: How can they use AI apps safely and securely?

Organizations strive to leverage AI applications to enhance operations, improve customer experiences, and facilitate data-driven decision-making. The key is to do this while ensuring safety and security. However, the main security hurdle lies in how some users might employ these applications.

Take ChatGPT, for example, which can be used to review source code for security flaws or assist in editing written content. ChatGPT was used to edit this very executive summary. Inevitably, some individuals will upload proprietary source code or text containing regulated data or intellectual property. The challenge lies in deterring such behavior without hindering the wider organization’s productivity. An outright block on AI applications could solve this problem, but would do so at the expense of the potential benefits AI apps offer.

As we continue to be surrounded by the hype centered on the possibilities of AI, it is evident that ChatGPT and other AI apps are on their way to becoming mainstays in the enterprise. Among Netskope customers, their popularity is growing exponentially, expected to double within the next seven months if they continue to grow at the current rate. This report delves into the rising prominence of AI applications in enterprises, outlines associated risks, including data leaks and potential attacker activity, and proposes strategies for safely and securely integrating ChatGPT and other AI tools in the enterprise setting.

sdofjsfojefgejelosij

AI Apps Growing in Popularity

The number of users accessing AI apps in the enterprise is growing exponentially. Over the past two months, the percentage of enterprise users accessing at least one AI app each day has increased by 2.4% weekly, for a total increase of 22.5% over that time period. At the current growth rate, the number of users accessing AI apps will double within the next seven months. Over the same time period, the number of AI apps in use in the enterprise held steady, with organizations with more than 1,000 users averaging 3 different AI apps per day, and organizations with more than 10,000 users averaging 5 AI apps per day. At the end of June, 1 out of 100 enterprise users interacted with an AI app each day.

AI app popularity based on number of enterprise users

The most popular enterprise AI app by a large margin is ChatGPT, with more than 8x as many daily active users as any other AI app. ChatGPT has been the center of much hype for the past six months and is also very versatile, likely contributing to its popularity. The next most popular app is Grammarly, which focuses exclusively on writing assistance. Bard, Google’s chatbot, comes in just below Grammarly. All other AI apps combined (of which we are tracking more than 60, including Jasper, Chatbase, and Copy.ai) are less popular than Google Bard.

Most popular AI apps by percentage of total daily AI users

Over the past two months, the fastest growing AI app in the enterprise was Google Bard. Although it still lags far behind ChatGPT in popularity, Google Bard is currently adding users at a rate of 7.1% per week, compared to 1.6% for ChatGPT. At their current rates, Google Bard is poised to catch up to ChatGPT in just over a year. However, as the AI app space is very dynamic, we expect to see many more changes during that time which will disrupt the current growth rates.

Netskope Threat Labs tracks the popularity of AI apps in enterprise environments, rather than the overall popularity of the apps among consumers. For example, while ChatGPT popularity skyrocketed among consumers before cooling off in June, its adoption in the enterprise has been more measured and continues to increase exponentially. The remainder of this report highlights some of the reasons for the measured increase, which include risks of data leakage and controls around its use.

Fastest growing AI apps by number of active daily users added weekly

AI Risks - Sensitive Data

This segment focuses on ChatGPT, the leading AI app in the enterprise by a large margin. An average ChatGPT user interacts with the app by posting 6 prompts daily. The activity level varies by user, with the top 10% of users posting 22 prompts and the top 1% posting 68 prompts daily. For every 10,000 users, an organization can expect around 660 daily prompts to ChatGPT. But the real question lies in the content of these prompts: Are they harmless queries, or do they inadvertently reveal sensitive data?

A Netskope study revealed that source code was the most frequently exposed type of sensitive data, with 22 out of 10,000 enterprise users posting source code to ChatGPT per month. In total, those 22 users are responsible for an average of 158 posts containing source code per month. This trend is not entirely unexpected, considering ChatGPT’s ability to review and explain code and pinpoint bugs and security vulnerabilities. While these services are beneficial, sharing confidential source code with ChatGPT introduces risks including potential data breaches, accidental data disclosure, and legal and regulatory risks.

Users posting sensitive data per 10,000k enterprise users per month

Compared to source code, posts containing other forms of sensitive data are relatively less common. For every 10,000 enterprise users, there are typically 18 incidents of sharing of regulated data (encompassing financial data, healthcare information, and personally identifiable information) on a monthly basis. Intellectual property (excluding source code) is rarer still, with an average of 4 incidents per month for every 10,000 users. Interestingly, passwords and keys also appear among the sensitive data types shared, usually embedded in source code. Despite its relative infrequency (about 4 incidents per 10,000 users monthly), this practice serves as a crucial reminder to software engineers about the risks of hard-coding secrets into source code.

Incidents of users posting sensitive data per 10,000k enterprise users per month

Opportunistic Attackers

With all the hype surrounding ChatGPT and AI apps in general, it is unsurprising that scammers, cybercriminals, and other attackers would attempt to exploit the hype for illicit gains. This is common practice with attackers. For example, the Netskope Threat Labs Cloud and Threat Report from Spring 2023 highlighted attackers attempting to capitalize on the Russo-Ukrainian war, the earthquake in Turkey and Syria, and the collapse of Silicon Valley Bank. The hype and popularity of ChatGPT draws the attention of attackers and scammers because of the large target pool and potential for profit, combined with the varied proficiency of users on the platform.

Throughout the first half of 2023, Netskope Threat Labs has tracked multiple phishing campaigns, malware distribution campaigns, and spam and fraud websites seeking to capitalize on the ChatGPT hype. Netskope Threat Labs is even tracking multiple ChatGPT proxies, sites that appear to offer the benefit of free, unauthenticated access to the chatbot, but at the cost of revealing all your prompts and responses to the proxy operator.

A ChatGPT proxy where the proxy operator sees all prompts and responses

In total, Netskope Threat Labs is currently tracking more than 1,000 malicious URLs and domains seeking to capitalize on the ChatGPT and AI hype. The number alone is a reminder of the importance of using a multi-layered approach to protect users from attackers attempting to capitalize on the hype and popularity surrounding any significant event or trend. Such an approach should include domain filtering, URL filtering, and content inspection to protect against both known and unknown attacks.

Enterprise Controls

While a common practice when ChatGPT was first gaining popularity was for enterprises to block the chatbot altogether, organizations have since come to terms with the fact that ChatGPT and other AI apps can offer advantages to the organization, including enhancing operations, improving customer experiences, and facilitating data-driven decision-making. Instead of blocking ChatGPT, organizations have transitioned to a more permissive model that typically includes a combination of DLP and user coaching. DLP can be used to identify potentially sensitive data being posted to AI apps, including ChatGPT, and user coaching can leave the ultimate decision of whether or not to proceed with a prompt to the user.

Organizations placing controls around ChatGPT

The specific controls around ChatGPT vary by industry vertical. Overall, financial services, healthcare, and technology companies have led the charge in implementing controls around ChatGPT. However, the approach each industry has taken varies significantly. In financial services and healthcare, both highly regulated industries, nearly 1 in 5 organizations have implemented a blanket ban. No users are allowed to use ChatGPT. In the technology vertical, only 1 in 20 organizations have implemented a blanket ban. Instead 1 in 4 organizations are using DLP controls to detect specific types of sensitive information (especially source code) being posted to ChatGPT. Furthermore, 1 in 5 technology organizations implement real-time user coaching to remind users of company policy and the risks that come along with ChatGPT and other AI apps. Ultimately, more organizations are likely to adopt DLP controls and real-time user coaching over time to enable the use of AI apps like ChatGPT while safeguarding against unwanted data exposure.

Type of ChatGPT controls by industry vertical

Recommendations

Safely enabling the adoption of AI apps in the enterprise is a multifaceted challenge. It involves identifying permissible apps and implementing controls that empower users to use them to their fullest potential while safeguarding the organization from risks. This section includes general technical recommendations for organizations aiming to safely enable AI apps. For more detailed information about how Netskope can help, please refer to the ChatGPT and Generative AI Data Protection solution brief.

→ Regularly review AI app activity, trends, behaviors, and data sensitivity, to identify risks to the organization.

→ Block access to apps that do not serve any legitimate business purpose or that pose a disproportionate risk. A good starting point is a policy to allow reputable apps currently in use while blocking all others.

→ Use DLP policies to detect posts containing potentially sensitive information, including source code, regulated data, passwords and keys, and intellectual property.

→ Employ real-time user coaching (combined with DLP) to remind users of company policy surrounding the use of AI apps at the time of interaction.

Block opportunistic attackers attempting to take advantage of the growing popularity of AI apps by blocking known malicious domains and URLs, and inspecting all HTTP and HTTPS content.

→ Use Remote Browser Isolation (RBI) technology to provide additional protection when there is a need to visit websites in categories that can present higher risk, like newly observed and newly registered domains.

→ Ensure that all security defenses share intelligence and work together to streamline security operations. Netskope customers can use Cloud Exchange to share IOCs, import threat intel, export event logs, automate workflows, and exchange risk scores.

About This Report

Netskope Threat Labs publishes a quarterly Cloud and Threat Report to highlight a specific set of cybersecurity challenges. The purpose of this report is to provide visibility into cybersecurity risks that AI apps present and how organizations are managing those risks. The analysis presented in this report is based on a study of millions of users from thousands of organizations worldwide, for the period starting May 1, 2023 through June 30, 2023. Stats are reflection of attacker tactics, user behavior, and organization policy. Information presented in this report is based on anonymized usage data collected by the Netskope Security Cloud platform relating to a subset of Netskope customers with prior authorization.

Netskope Threat Labs

Staffed by the industry’s foremost cloud threat and malware researchers, Netskope Threat Labs discovers, analyzes, and designs defenses against the latest web, cloud, and data threats affecting enterprises. Our researchers are regular presenters and volunteers at top security conferences, including DEF CON, Black Hat, and RSA.

Cloud and Threat Reports

The Netskope Cloud and Threat Report delivers unique insights into the adoption of cloud applications, changes in the cloud-enabled threat landscape, and the risks to enterprise data.

Browse reports

Storm with lightning over the city at night

Accelerate your security program with the SASE Leader

Get Started