Introduction
Phishing is one of the most common online security threats. A phishing website tries to mimic a legitimate page in order to obtain sensitive data such as usernames, passwords, or financial and health-related information from potential victims.
Machine learning (ML) algorithms have been used to detect phishing websites, as a complementary approach to signature matching and heuristics. They usually rely on a set of “domain knowledge” features, for example, the number of days the security certificate in the header is valid, the number of domains under the certificate, the host information, etc. However, many of the domain knowledge features are not available for inline processing, and they can be easily circumvented by sophisticated attackers.
To address the shortcomings of the domain knowledge features and detect zero-day phishing attacks in real time, at Netskope we use the latest deep learning techniques to implicitly learn the patterns of phishing websites. This includes using deep learning-based encoders on the textual content of the HTML page, Javascript and CSS code. We have been awarded three U.S. patents (Patent # 11,336,689, 11,438,377 and 11,444,978) for our innovative approach to phishing detection.
HTML Encoder
We have developed an HTML encoder to learn the proper representation of the entire HTML content (including the text body, Javascript, and CSS scripts) associated with the phishing detection use case. The HTML encoder is trained with the transformer-based deep learning architecture. This is inspired by the recent success of state-of-the-art language models, such as BERT and GPT transformer models. Similar to other transformer-based generative pre-training, we use a large number of web pages to train the HTML encoder in an unsupervised fashion. Unlike the BERT and GPT language models, however, the output of the HTML encoder is a two-dimensional ML-generated image. We chose the image output because phishing attacks are designed to use web pages that look similar to the real login pages. The ML-generated images appear to be effective in capturing features relevant to phishing and ignoring irrelevant parts of a web page. Below is an example of an HTML page and the corresponding ML-generated image from the HTML encoder.
The following GIF file shows more examples of the images generated by the HTML encoder. We should keep in mind that our objective is not to generate realistic images from the HTML content. Instead, it is to learn the suitable HTML representation that will be used to train the classification model for phishing detection.
Classification – phishing or not
Once we generate a suitable numerical representation (a vector of numbers) from the HTML content of a web page using the HTML encoder,