Voltar 
Imagine you’ve protected your production Google Cloud environment from compromised credentials, using MFA and a hardware security key. However, you find that your GCP environment has been breached through the hijacking of OAuth session tokens cached by gcloud access. Tokens were exfiltrated and used to invoke API calls from another host. The tokens were refreshed by the attacker and did not require MFA. Detecting the breach via Stackdriver was confusing, slowing incident response. And there are multiple confusing options to revoking the active OAuth sessions and most do not work, causing further delays in remediation.
In OAuth Token Hijacking in Google Cloud (GCP)—Part 1, we demonstrated the ease of several attack scenarios using hijacked OAuth tokens. In Part 2 of this blog, we will focus on concrete steps that you can take to reduce risk around detection, remediation, and prevention of OAuth token abuse. Let’s continue the discussion of the attack in Part 1 from the viewpoint of the defender.
Prevention
Two of the most effective measures at preventing or mitigating the abuse of compromised credentials are:
- Setting the expiration time for Google Cloud SDK sessions, which shrinks the window for compromise.
- Enforcing IP allow lists using access policies and VPC service controls, which mitigates the usage if session tokens have been compromised. This also helps improve detection.
Session Expiration
In the attack scenario in Part 1, the production environment did not have an expiration period set at all. This effectively set an infinite window for potential endpoint compromise. Let’s look at what can be configured in G Suite Admin to expire sessions
- Session duration: The Google cloud session duration in G Suite Admin > Security > Google Cloud session control should be set. It defaults to Session never expires and should instead be set to a time period between 1 and 24 hours. It is recommended that production environments be set to 1 hour and non-production environments between 1 and 8 hours.
- Re-authentication method: On the same screen, one specifies the re-authentication method after the session expires. You specify one of two choices: password or security key. This should not be set to a lower authentication strength than your primary authentication. Specifically, if you have a security key in use, this should be set to match (not Password!). Otherwise, you will weaken security for re-authentication flows.
IP Allow Lists
VPC Service Controls
IP allow lists should be implemented with VPC Service Controls found in Google Cloud Console > Security > VPC Service Controls, using an access level based on IP address defined in Google Cloud Console > Security > Access Context Manager. An IP allow list does not prevent token hijacking, but is a mitigating control if tokens are hijacked. It also allows us to improve our detection as we’ll see later when we discuss Detection.
When implemented, users attempting to call the specified APIs from a non-authorized source IP will get an access denied error:
Although the attacker can still utilize the OAuth tokens on the compromised host, this is still an effective way to mitigate the compromised tokens being exfiltrated and used on a different host.
Compute Instances/Metadata Proxy
A custom solution for IP allow lists Compute Instance tokens would be to automatically update allow lists as Compute VMs are launched. The Netflix security team describes several techniques including a proxy for the AWS metadata service, and something similar could be implemented on GCP. This will reduce risk from Compute instance compromise of OAuth tokens.
Auditing/Enforcement
It is a good idea to regularly audit and check that IP allow lists are implemented. Checking an IP allow list policy manually with the CLI might involve:
In Netskope’s Continuous Security Assessment product, a custom rule could be written similar to this (assuming naming is like the examples in this blog):
MFA
- MFA should be set and enforced in G Suite Admin > Security > 2-Step Verification for any Google Cloud admins. As we saw in the attack scenarios, it will not prevent reuse of hijacked OAuth tokens, and in fact, is not required unless the session expires. But when configured along with session expiration and re-authentication, it is an effective means to mitigating compromised passwords.
- Note: if your MFA is a software authenticator app, the re-authentication method after expiration will have to be Password, which is less secure, but is better than nothing.
- When using security hardware keys, make sure the re-authentication method matches and is set to security key.
Detection
Assuming we have OAuth token hijacking, as in Part 1, would we have detected anything suspicious? Unlikely. Detection of compromised credential use is difficult since the logged activity may appear to be normal user activity.
However, if we can implement IP allow lists, this will help improve detection when compromised credentials are used because we can detect attempts to use credentials from unapproved IP addresses.
To make this all clear, let’s take a closer look at Stackdriver logging, our centralized event logging service, as well as G Suite Audit logs for any recorded events based on API calls.
Improved Detection (IP Allow Lists)
If we use better prevention measures like IP allow lists, monitoring of logs can be done to check for authorization failures, which can help detect potential compromised token/credential attacks. False positives may occur if GCP administrators forget and attempt to use credentials from non-approved IPs, but this can still be an effective means to detecting that credentials (tokens) have been compromised.
Here is an example event from Stackdriver, showing a failed authorization attempt because the source IP did not match the allowed IPs specified in the VPC service control policy:
You should alert on any similar permission denied events for your VPC service control that implements your IP allow list as it may indicate a compromised credential has been exfiltrated and is being used on another host.
General logging
To complete our discussion, let’s review what is logged by Stackdriver for normal, successful access operations. Here’s a log entry related to the bucket access done in Part 1: