close
close
Your Network of Tomorrow
Your Network of Tomorrow
Plan your path toward a faster, more secure, and more resilient network designed for the applications and users that you support.
          Experience Netskope
          Get Hands-on With the Netskope Platform
          Here's your chance to experience the Netskope One single-cloud platform first-hand. Sign up for self-paced, hands-on labs, join us for monthly live product demos, take a free test drive of Netskope Private Access, or join us for a live, instructor-led workshops.
            A Leader in SSE. Now a Leader in Single-Vendor SASE.
            A Leader in SSE. Now a Leader in Single-Vendor SASE.
            Netskope debuts as a Leader in the Gartner® Magic Quadrant™ for Single-Vendor SASE
              Securing Generative AI for Dummies
              Securing Generative AI for Dummies
              Learn how your organization can balance the innovative potential of generative AI with robust data security practices.
                Modern data loss prevention (DLP) for Dummies eBook
                Modern Data Loss Prevention (DLP) for Dummies
                Get tips and tricks for transitioning to a cloud-delivered DLP.
                  Modern SD-WAN for SASE Dummies Book
                  Modern SD-WAN for SASE Dummies
                  Stop playing catch up with your networking architecture
                    Understanding where the risk lies
                    Advanced Analytics transforms the way security operations teams apply data-driven insights to implement better policies. With Advanced Analytics, you can identify trends, zero in on areas of concern and use the data to take action.
                        The 6 Most Compelling Use Cases for Complete Legacy VPN Replacement
                        The 6 Most Compelling Use Cases for Complete Legacy VPN Replacement
                        Netskope One Private Access is the only solution that allows you to retire your VPN for good.
                          Colgate-Palmolive Safeguards its "Intellectual Property” with Smart and Adaptable Data Protection
                          Colgate-Palmolive Safeguards its "Intellectual Property” with Smart and Adaptable Data Protection
                            Netskope GovCloud
                            Netskope achieves FedRAMP High Authorization
                            Choose Netskope GovCloud to accelerate your agency’s transformation.
                              Let's Do Great Things Together
                              Netskope’s partner-centric go-to-market strategy enables our partners to maximize their growth and profitability while transforming enterprise security.
                                Netskope solutions
                                Netskope Cloud Exchange
                                Netskope Cloud Exchange (CE) provides customers with powerful integration tools to leverage investments across their security posture.
                                  Netskope Technical Support
                                  Netskope Technical Support
                                  Our qualified support engineers are located worldwide and have diverse backgrounds in cloud security, networking, virtualization, content delivery, and software development, ensuring timely and quality technical assistance
                                    Netskope video
                                    Netskope Training
                                    Netskope training will help you become a cloud security expert. We are here to help you secure your digital transformation journey and make the most of your cloud, web, and private applications.

                                      ChatGPT: Dispelling FUD, Driving Awareness About Real Threats

                                      Jan 20 2023

                                      Summary

                                      ChatGPT is an artificial intelligence chatbot created by OpenAI, reaching 1 million users at the end of 2022. It is able to generate fluent responses given specific inputs. It is a variant of the GPT (Generative Pre-trained Transformer) model and, according to OpenAI, it was trained by mixing Reinforcement Learning from Human Feedback (RLHF) and InstructGPT datasets. Due to its flexibility and ability to mimic human behavior, ChatGPT has raised concerns in several areas, including cybersecurity.

                                      Its popularity has grown specially because it provides fluent responses to questions about virtually any subject, including history, math, literature, and technology.

                                      Asking ChatGPT what is its purpose.

                                      The chatbot is also a great tool for developers, able to create, review, improve, and explain code in multiple languages, such as C++, Java, Python, Go, and Rust. It also provides an API that can be used by plugins in different platforms such as Slack, Microsoft Teams, and IDEs like VS Code.

                                      And what happens if someone asks ChatGPT to create malware or ransomware? No need to panic right now, as ChatGPT will not create novel, functional malware automatically. But it can still be abused by attackers to improve social engineering skills or to understand and build common malware components.

                                      For social engineering, it can generate novel, fluent bait messages. For malware, it can serve as an interactive interface into the many malware tutorials and examples that already exist on the internet, but it still requires a certain knowledge from the person to use this tool to create a functional malware.

                                      There is also good news for cybersecurity defenders. ChatGPT has also been trained on examples of common code vulnerabilities and can pick them out when you share the source code with ChatGPT. So, like many inventions, ChatGPT can be used for both good and evil. Here’s a closer look at how:

                                      Malware Development

                                      Can ChatGPT automatically create novel, functional malware? The short answer is no, but, the tool can be used by attackers to aid in the malware development process, especially by those without experience.

                                      The first use case is to use it to learn and get code examples about malicious techniques. For example, we asked ChatGPT to explain details about a common process injection technique named Process Hollowing.

                                      ChatGPT explaining what Process Hollowing is.

                                      The answer was quite precise and included an example in C++, which could be easily modified to embed this functionality to a malware.

                                      We asked more questions about specific techniques that could be incorporated into malware, such as downloading and executing files from remote URLs or how to encrypt a file using AES on Windows, and the chatbot was able to generate and explain all of them.

                                      ChatGPT showing a C++ code that encrypts a file using AES

                                      However, during our tests, we found that ChatGPT generated incomplete codes for questions that add a certain level of complexity. For example, the AI wasn’t able to provide us with the entire code for a program that searches for files on Windows and encrypts them using a combination of symmetric and asymmetric encryption, which is a behavior that could be used to create ransomware.

                                      Incomplete code generated by ChatGPT.

                                      The AI is also able to identify malicious intent depending on how the question is asked. For example, we failed in obtaining a single code that downloads a base64 encoded executable and executes it using process hollowing.

                                      ChatGPT telling us that it cannot create the code we asked for.

                                      But although the code wasn’t generated in a single answer, nothing stops the attacker from splitting the question by:

                                      1. Asking for a code example on how to download base64 encoded files;
                                      2. Asking for an example of Process Hollowing technique;
                                      3. Combine both answers to perform process injection with the downloaded file.
                                      Code created by ChatGPT that downloads a base64 encoded file from an external URL.

                                      Therefore, ChatGPT won’t create a novel or a complete malware for someone, but it can indeed be used as a resource to study and create snippets of code that when combined could be used to create a functional malware.

                                      Social Engineering

                                      Attackers can use ChatGPT to improve their social engineering skills. It can help to write specific texts that would be used on phishing emails, redirecting victims to malicious websites or luring them into downloading attached malware. 

                                      For example, we asked ChatGPT to write an email that makes it look like the person just won $100,000 in a lottery named “BigMoney.” We specifically asked ChatGPT to randomly generate the ticket number and to convince the person to open a link that will supposedly claim the prize.

                                      The text generated by the AI could be easily employed in phishing emails to redirect people to a website that attempts to steal sensitive information, such as email accounts and passwords.

                                      Email generated by ChatGPT that tells a person won in the lottery.

                                      Another example is spear phishing. With previous knowledge about the target, an attacker can provide details about the victim to ChatGPT and ask help to generate a text that convinces the person to open an attached file.

                                      To exemplify this, we asked ChatGPT to generate an email from a person named Ruth to another person named Glenn, asking help on a fictional project. The project details would be attached in the email, which could be a weaponized file created by the attacker.

                                      Email generated by ChatGPT that could be used in spear phishing attacks.

                                      Attackers could also abuse ChatGPT to create fake news, luring victims to a website that steals sensitive data, such as email or bank accounts. As an example, we asked the AI to generate an email that contains fake news about the California government providing $5,000 to COVID-19 victims.

                                      Text generated by ChatGPT that contains fake news to support a phishing email.

                                      These examples demonstrate that ChatGPT has the potential to significantly decrease the barrier of entry for cybercriminals in building their social engineering attacks.

                                      Benefits for Cybersecurity

                                      ChatGPT is not only useful for attackers, but it can also be a valuable tool for defenders, who can use the tool to spot vulnerabilities in code or assess security posture to improve defenses overall.

                                      For example, we shared a piece of Python code with ChatGPT that contains a SQL injection vulnerability, and the AI was able to spot the problem, explain why it happens, and also provided suggestions to fix the code.

                                      ChatGPT spotting an SQL injection vulnerability.

                                      However, by sharing pieces of code with ChatGPT, users could very easily and unintentionally leak personal or corporate information, which could eventually harm organizations. This is even something that is advised by the platform once an account is created. We do not recommend submitting any confidential code to ChatGPT for analysis.

                                      ChatGPT advises to not share sensitive information in the conversations.

                                      Conclusions

                                      The availability of an AI tool able to mimic human behavior and provide coding knowledge increases cybersecurity risks for organizations, as it can be used by attackers to enhance social engineering attacks and create components to aid malware development. 

                                      It is common for novices to create new malware by copying existing code instead of writing it from scratch, which at this point is likely a faster path than circumventing ChatGPT’s security measures and having to grab pieces of code that need to be adjusted and combined to create functional malware.

                                      However, new attack vectors and malware families that we may see developed with ChatGPT’s help, or any upcoming AI, can largely be addressed by maintaining an effective security posture. This includes keeping software patched and up-to-date, maintaining effective data protection policies and technologies, and equipping the organization’s most valuable assets with additional security software such as an EDR (endpoint detection and response) and a secure web gateway solution. 

                                      For social engineering attacks that may emerge with the AI’s help, it’s important for organizations to rebuild the ‘human firewall’, training employees and customers to spot the small clues, such as misplaced details in phishing emails, and to always verify the identity of the person they are talking to.

                                      As we demonstrated, ChatGPT is not only a tool that can be used by attackers, but it can also be used in favor of developers and security professionals to spot code vulnerabilities and assess security posture. Also, having a better understanding of how attackers could potentially abuse this tool can help security teams to be better protected. Last but not least, the chatbot is still under development, so we should expect more robust security measures from OpenAI to prevent abuse of the tool.

                                      author image
                                      Gustavo Palazolo
                                      Gustavo Palazolo is an expert in malware analysis, reverse engineering and security research, working many years in projects related to electronic fraud protection.
                                      Gustavo Palazolo is an expert in malware analysis, reverse engineering and security research, working many years in projects related to electronic fraud protection.

                                      Stay informed!

                                      Subscribe for the latest from the Netskope Blog