The Future of Zero Trust and SASE is Now! Register now

close
close
The platform of the future is Netskope

Intelligent Security Service Edge (SSE), Cloud Access Security Broker (CASB), Cloud Firewall, Next Generation Secure Web Gateway (SWG), and Private Access for ZTNA built natively into a single solution to help every business on its journey to Secure Access Service Edge (SASE) architecture.

Go to Products Overview
Netskope video
Next Gen SASE Branch is hybrid — connected, secured, and automated

Netskope Next Gen SASE Branch converges Context-Aware SASE Fabric, Zero-Trust Hybrid Security, and SkopeAI-powered Cloud Orchestrator into a unified cloud offering, ushering in a fully modernized branch experience for the borderless enterprise.

Learn about Next Gen SASE Branch
People at the open space office
Designing a SASE Architecture For Dummies

Get your complimentary copy of the only guide to SASE design you’ll ever need.

Get the eBook
Embrace a Secure Access Service Edge (SASE) architecture

Netskope NewEdge is the world’s largest, highest-performing security private cloud and provides customers with unparalleled service coverage, performance and resilience.

Learn about NewEdge
NewEdge
Your Network of Tomorrow

Plan your path toward a faster, more secure, and more resilient network designed for the applications and users that you support.

Get the white paper
Your Network of Tomorrow
Netskope Cloud Exchange

The Netskope Cloud Exchange (CE) provides customers with powerful integration tools to leverage investments across their security posture.

Learn about Cloud Exchange
Netskope video
Make the move to market-leading cloud security services with minimal latency and high reliability.

Learn about NewEdge
Lighted highway through mountainside switchbacks
Safely enable the use of generative AI applications with application access control, real-time user coaching, and best-in-class data protection.

Learn how we secure generative AI use
Safely Enable ChatGPT and Generative AI
Zero trust solutions for SSE and SASE deployments

Learn about Zero Trust
Boat driving through open sea
Netskope achieves FedRAMP High Authorization

Choose Netskope GovCloud to accelerate your agency’s transformation.

Learn about Netskope GovCloud
Netskope GovCloud
  • Resources chevron

    Learn more about how Netskope can help you secure your journey to the cloud.

  • Blog chevron

    Learn how Netskope enables security and networking transformation through security service edge (SSE).

  • Events & Workshops chevron

    Stay ahead of the latest security trends and connect with your peers.

  • Security Defined chevron

    Everything you need to know in our cybersecurity encyclopedia.

Security Visionaries Podcast

Cookies, Not Biscuits
Host Emily Wearmouthas sits down with experts David Fairman and Zohar Hod to discuss the past, present, and future of internet cookies.

Play the podcast
Podcast: Cookies, Not Biscuits
Latest Blogs

How Netskope can enable the Zero Trust and SASE journey through security service edge (SSE) capabilities.

Read the blog
Sunrise and cloudy sky
SASE Week 2023: Your SASE journey starts now!

Replay sessions from the fourth annual SASE Week.

Explore sessions
SASE Week 2023
What is Security Service Edge?

Explore the security side of SASE, the future of network and protection in the cloud.

Learn about Security Service Edge
Four-way roundabout
We help our customers to be Ready for Anything

See our Customers
Woman smiling with glasses looking out window
Netskope’s talented and experienced Professional Services team provides a prescriptive approach to your successful implementation.

Learn about Professional Services
Netskope Professional Services
The Netskope Community can help you and your team get more value out of products and practices.

Go to the Netskope Community
The Netskope Community
Secure your digital transformation journey and make the most of your cloud, web, and private applications with Netskope training.

Learn about Training and Certifications
Group of young professionals working
  • Company chevron

    We help you stay ahead of cloud, data, and network security challenges.

  • Why Netskope chevron

    Cloud transformation and work from anywhere have changed how security needs to work.

  • Leadership chevron

    Our leadership team is fiercely committed to doing everything it takes to make our customers successful.

  • Partners chevron

    We partner with security leaders to help you secure your journey to the cloud.

Supporting sustainability through data security

Netskope is proud to participate in Vision 2045: an initiative aimed to raise awareness on private industry’s role in sustainability.

Find out more
Supporting Sustainability Through Data Security
Highest in Execution. Furthest in Vision.

Netskope recognized as a Leader in the 2023 Gartner® Magic Quadrant™ for Security Service Edge.

Get the report
Netskope recognized as a Leader in the 2023 Gartner® Magic Quadrant™ for Security Service Edge.
Thinkers, builders, dreamers, innovators. Together, we deliver cutting-edge cloud security solutions to help our customers protect their data and people.

Meet our team
Group of hikers scaling a snowy mountain
Netskope’s partner-centric go-to-market strategy enables our partners to maximize their growth and profitability while transforming enterprise security.

Learn about Netskope Partners
Group of diverse young professionals smiling

Cloud Threats Memo: Extracting Training Data from Generative AI Language Models

Dec 12 2023

This year will probably be remembered for the revolution of ChatGPT (the website was visited by 1.7 billion users in October 2023, with 13.73% of growth compared to the previous month) and for the widespread adoption of generative AI technologies in our daily life. One of the key aspects of the language models used for generative AI is the training dataset, and despite the controls in place for protecting  data privacy, the risk of using sensitive or protected information to train the model and the possibility of having this content inadvertently leaked is real. The latest warning comes from a paper published by researchers from Google and a team of academics: using a technique known as extractable memorization, the researchers were able to extract gigabytes of training data from several language models, including ChatGPT.

In what is called “a divergence attack” the academics discovered that asking the model to repeat a word forever (for example in the paper they showed the explicit example of the term “poem”) caused it to diverge and start generating nonsensical output. The problem is that a small fraction of these generations diverged into memorization, leaking pre-training data. But a small fraction can become an important amount of data for a motivated adversary with a dedicated budget who is able to perform queries at scale.

In fact, with just $200 USD worth of queries to ChatGPT (gpt-3.5-turbo), the researchers were able to extract more than 10,000 unique verbatim-memorized training examples, concluding that an adversary with a dedicated budget could likely extract “far more data,” and that larger, more capable models are even more vulnerable to data extraction attacks.

Leaked data that researchers were able to extract included memorized examples covering a wide range of text sources, such as: PII, inappropriate content, paragraphs from novels and complete copies of poems, valid URLs, UUIDs and accounts, and code. In particular, this last aspect does not sound surprising to us, since our recent report “AI Apps in the Enterprise” revealed that source code is posted to ChatGPT more than any other type of sensitive data, at a rate of 158 incidents per 10,000 enterprise users per month.

The researchers conclude that “…practitioners should not train and deploy LLMs for any privacy-sensitive applications without extreme safeguards.” This confirms what many organizations have already learned the hard way: Samsung, JPMorgan, and even Apple are just a few examples of organizations that restricted or completely blocked access to ChatGPT over corporate data leakage concerns. But many enterprises don’t have the same firepower as Samsung to develop their own generative AI Model, so they must find the right balance between unleashing the advantages of generative AI, and governing the risks of possible corporate data exfiltration.

Safely Enabling ChatGPT and Generative AI

Netskope provides automated tools for security teams to continuously monitor what applications (such as ChatGPT) corporate users attempt to access, how, when, from where, with what frequency etc. In particular a specific category of connectors for generative AI applications allows organizations to enforce granular access control. 

Netskope’s data loss prevention (DLP), powered by ML and AI models, can identify thousands of file types, personally identifiable information, intellectual property (IP), financial records and other sensitive data, preventing unwanted and non-compliant exposure. Netskope DLP offers several enforcement options to stop and limit the upload and posting of highly sensitive data through ChatGPT. Potentially dangerous actions (such as the upload of sensitive or protected data for training) can be completely blocked, or the user can be coached in real time to provide a business justification, or simply be reminded of the corporate policy before a possible risky action.

Finally, Netskope Advanced Analytics provides a specific dashboard to monitor the usage of generative AI apps across the enterprise, with rich details and insights including app usage, data movement, and user behavior.

author image
Paolo Passeri
Paolo supports Netskope’s customers in protecting their journey to the cloud and is a security professional, with 20+ years experience in the infosec industry. He is the mastermind behind hackmageddon.com, a blog detailing timelines and statistics of all the main cyber-attacks occurred since 2011. It is the primary source of data and trends of the threat landscape for the Infosec community.

Stay informed!

Subscribe for the latest from the Netskope Blog