The Future of Zero Trust and SASE is Now! Watch on-demand

close
close
  • Why Netskope chevron

    Changing the way networking and security work together.

  • Our Customers chevron

    Netskope serves more than 3,000 customers worldwide including more than 25 of the Fortune 100

  • Our Partners chevron

    We partner with security leaders to help you secure your journey to the cloud.

Highest in Execution. Furthest in Vision.

Netskope recognized as a Leader in the 2023 Gartner® Magic Quadrant™ for Security Service Edge.

Get the report
Netskope recognized as a Leader in the 2023 Gartner® Magic Quadrant™ for Security Service Edge.
We help our customers to be Ready for Anything

See our customers
Woman smiling with glasses looking out window
Netskope’s partner-centric go-to-market strategy enables our partners to maximize their growth and profitability while transforming enterprise security.

Learn about Netskope Partners
Group of diverse young professionals smiling
Your Network of Tomorrow

Plan your path toward a faster, more secure, and more resilient network designed for the applications and users that you support.

Get the white paper
Your Network of Tomorrow
Introducing the Netskope One Platform

Netskope One is a cloud-native platform that offers converged security and networking services to enable your SASE and zero trust transformation.

Learn about Netskope One
Abstract with blue lighting
Embrace a Secure Access Service Edge (SASE) architecture

Netskope NewEdge is the world’s largest, highest-performing security private cloud and provides customers with unparalleled service coverage, performance and resilience.

Learn about NewEdge
NewEdge
Netskope Cloud Exchange

The Netskope Cloud Exchange (CE) provides customers with powerful integration tools to leverage investments across their security posture.

Learn about Cloud Exchange
Netskope video
The platform of the future is Netskope

Intelligent Security Service Edge (SSE), Cloud Access Security Broker (CASB), Cloud Firewall, Next Generation Secure Web Gateway (SWG), and Private Access for ZTNA built natively into a single solution to help every business on its journey to Secure Access Service Edge (SASE) architecture.

Go to Products Overview
Netskope video
Next Gen SASE Branch is hybrid — connected, secured, and automated

Netskope Next Gen SASE Branch converges Context-Aware SASE Fabric, Zero-Trust Hybrid Security, and SkopeAI-powered Cloud Orchestrator into a unified cloud offering, ushering in a fully modernized branch experience for the borderless enterprise.

Learn about Next Gen SASE Branch
People at the open space office
Designing a SASE Architecture For Dummies

Get your complimentary copy of the only guide to SASE design you’ll ever need.

Get the eBook
Make the move to market-leading cloud security services with minimal latency and high reliability.

Learn about NewEdge
Lighted highway through mountainside switchbacks
Safely enable the use of generative AI applications with application access control, real-time user coaching, and best-in-class data protection.

Learn how we secure generative AI use
Safely Enable ChatGPT and Generative AI
Zero trust solutions for SSE and SASE deployments

Learn about Zero Trust
Boat driving through open sea
Netskope achieves FedRAMP High Authorization

Choose Netskope GovCloud to accelerate your agency’s transformation.

Learn about Netskope GovCloud
Netskope GovCloud
  • Resources chevron

    Learn more about how Netskope can help you secure your journey to the cloud.

  • Blog chevron

    Learn how Netskope enables security and networking transformation through security service edge (SSE)

  • Events and Workshops chevron

    Stay ahead of the latest security trends and connect with your peers.

  • Security Defined chevron

    Everything you need to know in our cybersecurity encyclopedia.

Security Visionaries Podcast

Zero Trust: It's More Than Just Identity
On this episode host Emily Wearmouth explores the concept of zero trust with cybersecurity experts John Kindervag, the "Godfather of Zero Trust," and Neil Thacker.

Play the podcast
Zero Trust: It's More Than Just Identity
Latest Blogs

Read how Netskope can enable the Zero Trust and SASE journey through security service edge (SSE) capabilities.

Read the blog
Sunrise and cloudy sky
SASE Week 2023: Your SASE journey starts now!

Replay sessions from the fourth annual SASE Week.

Explore sessions
SASE Week 2023
What is Security Service Edge?

Explore the security side of SASE, the future of network and protection in the cloud.

Learn about Security Service Edge
Four-way roundabout
  • Company chevron

    We help you stay ahead of cloud, data, and network security challenges.

  • Leadership chevron

    Our leadership team is fiercely committed to doing everything it takes to make our customers successful.

  • Customer Solutions chevron

    We are here for you and with you every step of the way, ensuring your success with Netskope.

  • Training and Certification chevron

    Netskope training will help you become a cloud security expert.

Supporting sustainability through data security

Netskope is proud to participate in Vision 2045: an initiative aimed to raise awareness on private industry’s role in sustainability.

Find out more
Supporting Sustainability Through Data Security
Thinkers, builders, dreamers, innovators. Together, we deliver cutting-edge cloud security solutions to help our customers protect their data and people.

Meet our team
Group of hikers scaling a snowy mountain
Netskope’s talented and experienced Professional Services team provides a prescriptive approach to your successful implementation.

Learn about Professional Services
Netskope Professional Services
Secure your digital transformation journey and make the most of your cloud, web, and private applications with Netskope training.

Learn about Training and Certifications
Group of young professionals working

GCP OAuth Token Hijacking in Google Cloud—Part 2

Aug 25 2020

Imagine you’ve protected your production Google Cloud environment from compromised credentials, using MFA and a hardware security key. However, you find that your GCP environment has been breached through the hijacking of OAuth session tokens cached by gcloud access. Tokens were exfiltrated and used to invoke API calls from another host. The tokens were refreshed by the attacker and did not require MFA. Detecting the breach via Stackdriver was confusing, slowing incident response. And there are multiple confusing options to revoking the active OAuth sessions and most do not work, causing further delays in remediation.

In OAuth Token Hijacking in Google Cloud (GCP)—Part 1, we demonstrated the ease of several attack scenarios using hijacked OAuth tokens. In Part 2 of this blog, we will focus on concrete steps that you can take to reduce risk around detection, remediation, and prevention of OAuth token abuse. Let’s continue the discussion of the attack in Part 1 from the viewpoint of the defender.

Prevention

Two of the most effective measures at preventing or mitigating the abuse of compromised credentials are:

  • Setting the expiration time for Google Cloud SDK sessions, which shrinks the window for compromise.
  • Enforcing IP allow lists using access policies and VPC service controls, which mitigates the usage if session tokens have been compromised. This also helps improve detection.

Session Expiration

In the attack scenario in Part 1, the production environment did not have an expiration period set at all. This effectively set an infinite window for potential endpoint compromise. Let’s look at what can be configured in G Suite Admin to expire sessions

  • Session duration: The Google cloud session duration in G Suite Admin > Security > Google Cloud session control should be set. It defaults to Session never expires and should instead be set to a time period between 1 and 24 hours. It is recommended that production environments be set to 1 hour and non-production environments between 1 and 8 hours.
Screenshot showing Google Cloud session control
  • Re-authentication method: On the same screen, one specifies the re-authentication method after the session expires. You specify one of two choices: password or security key. This should not be set to a lower authentication strength than your primary authentication. Specifically, if you have a security key in use, this should be set to match (not Password!). Otherwise, you will weaken security for re-authentication flows.

IP Allow Lists

VPC Service Controls

IP allow lists should be implemented with VPC Service Controls found in Google Cloud Console > Security > VPC Service Controls, using an access level based on IP address defined in Google Cloud Console > Security > Access Context Manager. An IP allow list does not prevent token hijacking, but is a mitigating control if tokens are hijacked. It also allows us to improve our detection as we’ll see later when we discuss Detection.

Screenshot of Access Context Manager
Screenshot of VPC Service Controls

When implemented, users attempting to call the specified APIs from a non-authorized source IP will get an access denied error:

Example of access denied error

Although the attacker can still utilize the OAuth tokens on the compromised host, this is still an effective way to mitigate the compromised tokens being exfiltrated and used on a different host. 

Compute Instances/Metadata Proxy

A custom solution for IP allow lists Compute Instance tokens would be to automatically update allow lists as Compute VMs are launched. The Netflix security team describes several techniques including a proxy for the AWS metadata service, and something similar could be implemented on GCP. This will reduce risk from Compute instance compromise of OAuth tokens.

Auditing/Enforcement

It is a good idea to regularly audit and check that IP allow lists are implemented. Checking an IP allow list policy manually with the CLI might involve:

Example of checking an IP allow list policy manually

In Netskope’s Continuous Security Assessment product, a custom rule could be written similar to this (assuming naming is like the examples in this blog):

Example of a custom rule written for Netskope's Continuous Security Assessment

MFA

  • MFA should be set and enforced in G Suite Admin > Security > 2-Step Verification for any Google Cloud admins. As we saw in the attack scenarios, it will not prevent reuse of hijacked OAuth tokens, and in fact, is not required unless the session expires. But when configured along with session expiration and re-authentication, it is an effective means to mitigating compromised passwords.
Screenshot showing 2-factor verification
  • Note: if your MFA is a software authenticator app, the re-authentication method after expiration will have to be Password, which is less secure, but is better than nothing.
  • When using security hardware keys, make sure the re-authentication method matches and is set to security key.

Detection

Assuming we have OAuth token hijacking, as in Part 1, would we have detected anything suspicious? Unlikely. Detection of compromised credential use is difficult since the logged activity may appear to be normal user activity. 

However, if we can implement IP allow lists, this will help improve detection when compromised credentials are used because we can detect attempts to use credentials from unapproved IP addresses.

To make this all clear, let’s take a closer look at Stackdriver logging, our centralized event logging service, as well as G Suite Audit logs for any recorded events based on API calls.

Improved Detection (IP Allow Lists)

If we use better prevention measures like IP allow lists, monitoring of logs can be done to check for authorization failures, which can help detect potential compromised token/credential attacks. False positives may occur if GCP administrators forget and attempt to use credentials from non-approved IPs, but this can still be an effective means to detecting that credentials (tokens) have been compromised.

Here is an example event from Stackdriver, showing a failed authorization attempt because the source IP did not match the allowed IPs specified in the VPC service control policy:

Example event from Stackdriver showing a failed authorization attempt because the source IP allow list did not match the access level in the policy

You should alert on any similar permission denied events for your VPC service control that implements your IP allow list as it may indicate a compromised credential has been exfiltrated and is being used on another host.

General logging

To complete our discussion, let’s review what is logged by Stackdriver for normal, successful access operations. Here’s a log entry related to the bucket access done in Part 1:

Example log entry related to the bucket access done in Part 1.

Note that we see a bucket access operation on bucket-foo-dev-mfa by user [email protected]. There is no indication of what is happening at the OAuth token level, and even if there were, it would be hard to determine if it was suspicious activity.

There are also Token Audit Logs in G Suite Admin, but they have the same challenge–any activity there does not necessarily identify suspicious activity.

Remediation

Multiple remediation options exist but are confusing because of the number of options and the differences between browser vs. API sessions and Google Cloud vs G Suite apps.

If you suspect an account has been compromised and that session tokens are being used, remediation needs to include two parts:

  1. We need to prevent future logins i.e. account access by the attacker
  2. We need to revoke/disable any current access by the attacker i.e. revoke any current OAuth session access and refresh tokens

Although there are many potential settings in the Google Cloud and G Suite Admin Consoles and APIs, here are the effective ones that work in all scenarios:

Table of effective remediation settings for User and Service accounts

User accounts

Prevent future account access via password reset

User accounts can be disabled by resetting the password in G Suite Admin > Users:

Screenshot of password reset in G Suite Admin.

Resetting the password will not revoke currently active sessions, which is a second, separate step described below.

How about deleting the user account? That would also disable future access by the attack, as well as revoke currently active sessions all in one action, but is not recommended as it’s a drastic measure, requiring a high-level of reprovisioning and reconfiguration.

Revoke current sessions via delete OAuth connected applications

The best (only) way to revoke current Google Cloud OAuth sessions is to:

  • Delete the connected Google Cloud SDK OAuth application in G Suite Admin Console or via the corresponding G Suite SDK API call.
  • The Console action and API call will immediately revoke all access and refresh tokens for that user.

In G Suite Admin > Users > user > Security > Connected applications:

Screenshot of connected applications in G Suite Admin

Or via the G Suite Directory API:

Screenshot of password rest in G Suite directory API

Since these actions are on the G Suite Console/APIs, ensure access is granted to G Suite for Google Cloud admins.

Service accounts

Prevent future account access via deleting/recreating service accounts

Service accounts can be deleted in Google Cloud Console > IAM & Admin > Service Accounts

Screenshot showing how to delete service accounts in Google Cloud Console

You might be wondering whether the drastic action of deleting the service account is required vs. disabling or deleting the API key itself. It is needed, because it is the only way to revoke the current session tokens. Deleting or disabling the API key under the service account will not revoke current session tokens.

Revoke current sessions via deleting/recreating service accounts

Deleting a service account will revoke current OAuth sessions, which is why it is the recommended action.

Note: Since the recommended action involves deleting and recreating a service account, this has large implications for recovery/downtime during an incident. All uses of the service account must be reconfigured, including compute instance VMs. This requires stopping the instance at a minimum, which would be recommended anyway in order to ensure current shell access to the VM by the attacker is revoked. This does account provisioning and compute instance configuration should be reviewed and automated with CLI/API scripts as much as possible to minimize downtime and reduce errors.

Other Remediation Actions

These remediation options do not work well in all cases and should be avoided. They are mentioned to ensure that their drawbacks are clearly understood since the number of options available can be confusing.

User accounts

Table showing remediation options for user accounts

Service accounts

Table showing remediation options for service accounts

Conclusion

Because the implementation of OAuth includes caching of session tokens, we’ve seen how easy it is for attackers to exploit this as another attack vector, once access to the GCP administrator’s machine is attained. Credential caches can be easily copied and used elsewhere, and will not require MFA even if configured.

Furthermore, we’ve seen how confusing it can be to prevent, detect, and remediate compromised credential situations due to the numerous options available in both G Suite and Google Cloud.

Despite these considerations, there are concrete measures that can help prevent abuse in the case of compromise:

author image
Jenko Hwong
Jenko has 15+ years of experience in research, product management, and engineering in cloud security, AV/AS, routers/appliances, threat intel, Windows security, vulnerability scanning and compliance. At Netskope, he researches new cloud attacks.

Stay informed!

Subscribe for the latest from the Netskope Blog