Netskope named a Leader in the 2024 Gartner® Magic Quadrant™ for Security Service Edge. Get the report

close
close
  • Why Netskope chevron

    Changing the way networking and security work together.

  • Our Customers chevron

    Netskope serves more than 3,000 customers worldwide including more than 25 of the Fortune 100

  • Our Partners chevron

    We partner with security leaders to help you secure your journey to the cloud.

Still Highest in Execution.
Still Furthest in Vision.

Learn why 2024 Gartner® Magic Quadrant™ named Netskope a Leader for Security Service Edge the third consecutive year.

Get the report
Netskope Named a Leader in the 2024 Gartner® Magic Quadrant™ for Security Service Edge graphic for menu
We help our customers to be Ready for Anything

See our customers
Woman smiling with glasses looking out window
Netskope’s partner-centric go-to-market strategy enables our partners to maximize their growth and profitability while transforming enterprise security.

Learn about Netskope Partners
Group of diverse young professionals smiling
Your Network of Tomorrow

Plan your path toward a faster, more secure, and more resilient network designed for the applications and users that you support.

Get the white paper
Your Network of Tomorrow
Introducing the Netskope One Platform

Netskope One is a cloud-native platform that offers converged security and networking services to enable your SASE and zero trust transformation.

Learn about Netskope One
Abstract with blue lighting
Embrace a Secure Access Service Edge (SASE) architecture

Netskope NewEdge is the world’s largest, highest-performing security private cloud and provides customers with unparalleled service coverage, performance and resilience.

Learn about NewEdge
NewEdge
Netskope Cloud Exchange

The Netskope Cloud Exchange (CE) provides customers with powerful integration tools to leverage investments across their security posture.

Learn about Cloud Exchange
Netskope video
The platform of the future is Netskope

Intelligent Security Service Edge (SSE), Cloud Access Security Broker (CASB), Cloud Firewall, Next Generation Secure Web Gateway (SWG), and Private Access for ZTNA built natively into a single solution to help every business on its journey to Secure Access Service Edge (SASE) architecture.

Go to Products Overview
Netskope video
Next Gen SASE Branch is hybrid — connected, secured, and automated

Netskope Next Gen SASE Branch converges Context-Aware SASE Fabric, Zero-Trust Hybrid Security, and SkopeAI-powered Cloud Orchestrator into a unified cloud offering, ushering in a fully modernized branch experience for the borderless enterprise.

Learn about Next Gen SASE Branch
People at the open space office
Designing a SASE Architecture For Dummies

Get your complimentary copy of the only guide to SASE design you’ll ever need.

Get the eBook
Make the move to market-leading cloud security services with minimal latency and high reliability.

Learn about NewEdge
Lighted highway through mountainside switchbacks
Safely enable the use of generative AI applications with application access control, real-time user coaching, and best-in-class data protection.

Learn how we secure generative AI use
Safely Enable ChatGPT and Generative AI
Zero trust solutions for SSE and SASE deployments

Learn about Zero Trust
Boat driving through open sea
Netskope achieves FedRAMP High Authorization

Choose Netskope GovCloud to accelerate your agency’s transformation.

Learn about Netskope GovCloud
Netskope GovCloud
  • Resources chevron

    Learn more about how Netskope can help you secure your journey to the cloud.

  • Blog chevron

    Learn how Netskope enables security and networking transformation through security service edge (SSE)

  • Events and Workshops chevron

    Stay ahead of the latest security trends and connect with your peers.

  • Security Defined chevron

    Everything you need to know in our cybersecurity encyclopedia.

Security Visionaries Podcast

The Intersection of Zero Trust and National Security
On the latest episode of Security Visionaries, co-hosts Max Havey and Emily Wearmouth sit down for a conversation with guest Chase Cunningham (AKA Dr. Zero Trust) about zero trust and national security.

Play the podcast
The Intersection of Zero Trust and National Security
Latest Blogs

Read how Netskope can enable the Zero Trust and SASE journey through security service edge (SSE) capabilities.

Read the blog
Sunrise and cloudy sky
SASE Week 2023: Your SASE journey starts now!

Replay sessions from the fourth annual SASE Week.

Explore sessions
SASE Week 2023
What is SASE?

Learn about the future convergence of networking and security tools in today’s cloud dominant business model.

Learn about SASE
  • Company chevron

    We help you stay ahead of cloud, data, and network security challenges.

  • Leadership chevron

    Our leadership team is fiercely committed to doing everything it takes to make our customers successful.

  • Customer Solutions chevron

    We are here for you and with you every step of the way, ensuring your success with Netskope.

  • Training and Certification chevron

    Netskope training will help you become a cloud security expert.

Supporting sustainability through data security

Netskope is proud to participate in Vision 2045: an initiative aimed to raise awareness on private industry’s role in sustainability.

Find out more
Supporting Sustainability Through Data Security
Thinkers, builders, dreamers, innovators. Together, we deliver cutting-edge cloud security solutions to help our customers protect their data and people.

Meet our team
Group of hikers scaling a snowy mountain
Netskope’s talented and experienced Professional Services team provides a prescriptive approach to your successful implementation.

Learn about Professional Services
Netskope Professional Services
Secure your digital transformation journey and make the most of your cloud, web, and private applications with Netskope training.

Learn about Training and Certifications
Group of young professionals working

Train Your Own Classifier (TYOC) for Image Data Protection

May 30 2024

Introduction

Machine learning-based data loss prevention (DLP) file classifiers provide a fast and effective way to identify sensitive data in real-time, empowering organizations with granular, real-time DLP policy controls. Netskope Advanced DLP offers a wide range of predefined file classifiers, such as passports, driver’s licenses, checks, payment cards, screenshots, source code, tax forms, and business agreements. Although these predefined classifiers are remarkable in their own right, they are necessarily somewhat generic when considering the enormous diversity of sensitive data across different industries and organizations. To better address company-specific or industry-specific documents, including identity documents, HR files, or critical infrastructure images, Netskope has developed a novel patented approach that allows customers to train their own classifiers while maintaining data privacy. This innovation enables organizations to focus on protecting their most critical information.

This training process, known as Train Your Own Classifier (TYOC), is designed to be efficient, requiring neither a large amount of labeled data nor time-consuming training of a supervised classification model.This capability is made possible through the use of cutting-edge contrastive learning techniques. Customers can upload a small set of example images (approximately 20-30) to the Netskope Security Cloud. These examples are then used to extract important attributes and train a customized classifier using Netskope’s machine learning engine.

Once the custom classifier is trained, it is deployed into the customer’s own tenant to detect sensitive information anywhere they use Netskope DLP including email and Endpoint DLP. Importantly, the original samples are not retained and the trained classifier is not shared with any other customers, ensuring the protection of the customer’s sensitive data throughout the process.

Image Similarity and Contrastive Learning

TYOC solves a problem of image similarity by using techniques of contrastive learning.

Image similarity addresses the challenge of identifying images that resemble a reference image, even when there are minor differences in aspects such as color, orientation, cropping, and other characteristics. This process can be effectively managed using advanced contrastive learning techniques.

Contrastive learning is designed to extract meaningful representations by contrasting pairs of similar (positive) and dissimilar (negative) instances. It is based on the concept that similar instances should be positioned closer in a learned embedding space, whereas dissimilar instances should be placed further apart. Contrastive learning involves training image models through unsupervised learning by augmenting each image in a manner that preserves its semantic content. This augmentation includes operations such as random rotations, color distortions, and crops, ensuring that the cropped area remains a significant portion of the original image. These augmented samples are used to train a convolutional neural network (CNN)-based image encoder model. This encoder takes an image as input and produces a feature vector, also known as a representation or embedding.

Netskope TYOC combines a pre-trained image encoder built by Netskope with a small number of training images provided by a customer. The combination enables the Netskope security cloud to perform image similarity ranking on customer-relevant files with performance similar to what is achieved by built-in (predefined) file classifiers.

Training with Contrastive Learning

The encoder model learns to identify similarities between images by establishing that matched pairs of images, referred to as positive pairs, exhibit the highest similarity. Conversely, unmatched pairs or negative pairs – drawn from the remainder of the image dataset – are assigned the lowest similarity. We illustrate this concept through examples of positive and negative pairs below.

The image encoder, trained with contrastive learning, maps any image to a high-dimensional embedding for semantic hash, effectively capturing the semantic meaning of the image. The illustration below showcases the application of this pre-trained image encoder on the “Dogs & Muffins” dataset, which comprises eight images of dogs and eight images of muffins, all closely resembling one another in appearance. On the right, we present a three-dimensional projection of the high-dimensional embeddings generated for each image. This visualization clearly demonstrates the distinct segregation of the two categories within the embedding space, underscoring the encoder’s efficacy in capturing and differentiating the semantic essence of the images.

Using Train Your Own Classifier

By utilizing the pre-trained image encoder model, our system allows customers to upload their training or reference images for the purpose of training a bespoke classifier. For optimal performance, it is recommended to provide at least 20-30 reference images for each category. The image encoder processes these reference images, converting them into high-dimensional embeddings. To ensure privacy, the original images are deleted after encoding. These reference embeddings are then utilized to construct an Approximate Nearest Neighbors (ANN) index, which acts as the custom classifier.

During the inference phase, new images undergo encoding to generate embeddings using the same image encoder model. The ANN model then identifies the class label of the nearest reference embedding. If the distance to this nearest embedding falls below a predefined threshold, the image is assigned the corresponding predicted label from the reference embedding. If not, the image is categorized under the predicted label “other.”

Case Studies

Access Cards

In this evaluation, we adopted the TYOC methodology for classifying Access Cards, as detailed below. Initially, our dataset comprised only three authentic (sample) examples, illustrated on the left side. To augment our training data, we generated 30 synthetic images. This augmentation involved substituting the portraits on the sample cards with a variety of random portraits, as depicted on the right side. Subsequently, we allocated 20 of these images for training the classifier, while the remaining 10, along with 1,000 randomly selected negative examples, were used for testing purposes. In the testing phase, the custom classifier demonstrated exceptional performance, achieving a recall or detection rate of 100% with the precision of 99.3%.

Handwritten Signatures

For this experiment, a public dataset of handwritten signatures was used. The dataset includes signatures of 64 individuals, with approximately 25 image instances for each name. Of these, around 10 images per name represent forgeries. All images were transformed into embeddings using the pre-trained image encoder, without any further retraining. For each individual, six embeddings were incorporated into Annoy as reference images, while the remaining approximately 20 embeddings per name served as test samples. When assessing the test images of signatures, they could be accurately matched to the corresponding name with an 87% accuracy, provided that forgeries were considered valid matches. If forgeries were excluded, the accuracy rate slightly decreased to 84%.

Privacy Concerns

Our pre-trained image encoder translates images into high-dimensional semantic embeddings – compact vector representations of an image’s essential meaning and its visually similar counterparts. As these embeddings contain semantic data, there is a theoretical risk that images could be partially or fully reconstructed from their embeddings, potentially compromising user privacy within our system.

To mitigate these concerns, we’ve conducted thorough experiments and found that while it is theoretically possible for images to be reconstructed from high-dimensional embeddings under extremely unlikely situations, the resulting versions would be of very low fidelity. This limitation significantly restricts the amount of recoverable information, providing a robust safeguard against potential privacy breaches.

As a worst case scenario, we attempt to reconstruct images from their high-dimensional TYOC embeddings using a generative AI model that has access to the TYOC encoder. Below are some of the results. Although it is highly unlikely that the model architecture, weight files, and actual image embeddings would be fully accessible, our reconstructions still exhibit very poor quality, failing to reproduce any fine details.

Summary

Train Your Own Classifier is now generally available as part of Netskope’s Advanced DLP (screenshot below). To learn more about the industry’s most comprehensive and most advanced cloud DLP solution, please visit the Netskope Data Loss Prevention page.

author image
Jason Bryslawskyj
At Netskope, Jason has been developing computer vision models for data loss prevention and phishing detection.
author image
Yihua Liao
Dr. Yihua Liao is the Head of AI Labs at Netskope. His team Develops cutting-edge AI/ML technology to tackle many challenging problems in cloud security, including data loss prevention, malware and threat protection, and user/entity behavior analytics. Previously, he led data science teams at Uber and Facebook.

Stay informed!

Subscribe for the latest from the Netskope Blog