Palo Alto Networks SSL Decryption Health with Indeni

With SSL encryption being such a crucial part of securing your network traffic, you can imagine it also is very important to your company that it works securely and optimally. The goal of this post is to cover the importance of SSL, why it should be decrypted, how it affects firewalls and lastly how Indeni can help you breathe easier.

SSL Encryption and Importance of Decryption

SSL encryption is a method of protecting the integrity of data between endpoints in a conversation. Just as SSL can be used to protect data from malicious interception it can also be used to hide malicious transfer of data. This can be unintentional egress transfer of data, insider threat of data leakage, or command and control traffic. It also could bypass vulnerability protections of a firewall to a hosted web server or application.

Let’s think of it like this. Many businesses use VPN’s as an encryption method to keep traffic safe, but it is decrypted on your LAN. This allows for health checks of your network traffic by things like IPS, DLP, vulnerability protection, and application visibility. SSL essentially is doing just the same when encrypting. However, by not decrypting your traffic before the client, you are essentially saying you trust any data going into and out of that computer over that session. Do you really? These sessions are bypassing all of your network-based security allowing that client to server communication to remain uninspected thus transferring any type of data it wants without your knowledge.

According to multiple data sources over 50% of overall global internet traffic is SSL encrypted now. Many businesses are seeing easily more than 70 to 80% of traffic as being SSL encrypted. Here are several more reasons SSL is becoming so important for Internet traffic:

  • Cloud applications on the rise
  • Adoption of advanced SSL such as TLS 1.3 SSL encryption are on a significant rise.
  • Browsers and plug-ins are starting to warn you when visiting a site without SSL encryption and poor encryption methods.
  • SSL isn’t just for securing legitimate traffic. Malicious traffic is increasingly being encrypted

Without SSL Decryption:

  • Vulnerability to security threats hidden in web transactions like the javascript or HTTP requests within a session.
  • Malicious advertisement sites are not analyzed and/or blocked
  • Circumventing internal compliance policies (personal VPN’s or just unsanctioned/unknown cloud applications)
  • Inability to detect data breaches. Do you know for sure you haven’t already been breached if you can’t see the traffic you are allowing?
  • Unauthorized attacker Eavesdropping/man-in-the-middle attacks
  • Malware can bypass firewalls, email and web gateways, and any other security methods along the network path.

With SSL Decryption:

  • We at least have some hope for protection against all of the above concerns.
  • SSL Decryption makes it less difficult to troubleshoot problems and performance issues or even to identify application traffic.
  • Allows your file traffic to be sent to sandbox security systems for analysis.
  • Know your IoT! Allows your IoT device traffic to be decrypted as much is tolerated by their applications becoming familiar with the payload of these devices.

How to eliminate requirements to decrypt SSL traffic. Wait, what? Yes, it can be just as important to make sure SSL traffic is NOT decrypted. Not everything will support decryption with reasons including both technical and regulatory reasons.

  • Only permit applications and site categories required by your business.
    • Block Bittorrent?
    • DropBox personal
    • iCloud?
  • Block VPN’s that aren’t your own
  • Don’t decrypt personal information
  • Make it optional but suggest other personal traffic be decrypting SSL and opt-out if the user isn’t comfortable with decryption.

References:
Add it to the Decryption Exclusion list
Create a policy-based exclusion

How Indeni Can Help:

Just like going to the doctor only knowing one or a few of the symptoms, trained and experienced experts and crowdsourced knowledge can take these symptoms, determine your ailment and the prescription to relieve your pain and suffering. Indeni is like a team of doctors helping you deal with SSL Decryption Health.

It is important for systems and security operation teams to know before an issue with SSL Decryption occurs if possible. If you cannot know before, it is still important that Indeni notifies you of the root cause and allows for quick remediation. You might even have the solution in place by the time the service desk escalates any support tickets.

So, just what can Indeni do to help? Due to SSL decryption requiring so many different processes there are many commands and metrics that must be analyzed to determine the health of a system with SSL Decryption enabled.

  • Here is a suggested article discussing how to manually determine if SSL Decryption is causing the performance issues, maxing out and dropping sessions, or otherwise causing problems that you are seeing on the firewall.

1. Certificate cache limits: show system setting ssl-decrypt memory

  • Indeni will trigger a notification when the current cache allocation is nearing the limit. You can see both manually by running ‘show system setting ssl-decrypt memory’.

DP dp0:

proxy uses shared allocator
SSL certificate cache:

Current Entries: 94
Allocated 1928, Freed 1834

DP dp1:

proxy uses shared allocator
SSL certificate cache:

Current Entries: 202
Allocated 4162, Freed 3960

DP dp2:

proxy uses shared allocator
SSL certificate cache:

Current Entries: 234
Allocated 4563, Freed 4329

2. SSL Decryption

  • Each model has a limit of concurrent SSL decryption sessions. It is important to know when the model is reaching capacity and causing SSL decryption drops.
  • Indeni knows the session limits for every model of firewall. Indeni reaches out to the firewall and gathers the current session count every minute using ‘show session all filter ssl-decrypt yes count yes’ and compares it to the session limit by also checking the model of firewall. If it is nearing capacity by hitting 80% of available sessions it will notify you.

3. Now, one of the final things we must do to troubleshoot SSL decryption limits and performance is determining the root cause for packet drops.

  • Indeni needs to determine if SSL decryption is being dropped because of a misconfiguration or possibly unsupported ciphers.
    • show counter global name proxy ssl no resource
    • show counter global name proxy ssl unsupported cipher
    • show counter global name proxy ssl unsupported
    • show counter global name proxy url request pkt drop
    • show counter global name proxy low alloc failure
    • show counter global name ssl server cipher not supported
    • show counter global name ssl sess id resume drop
  • If error counters relating to SSL increment Indeni will send a notification. After that notification Indeni will mark it as resolved if the issue has gone away after the required period of time.

Rest of the notes section:

4. Remediation Options:

  • Cipher Support Remediation:
    • See #5 below as well
  • Capacity Remediation:
    • Reduce URL categories or inbound (hosted) websites being decrypted. One way to do this is to turn off decryption for trusted sites that don’t require decryption for DLP purposes, etc. For instance, you can create a custom URL category called “Trusted_Do_Not_Decrypt”; place URL’s in there that you trust such as Office 365 URLs etc.; then set that category to no decryption. Just remember when you do this security benefits provided by the firewall will be bypassed and you will be relying on the client or server’s security protection instead.
    • Upgrade your firewall to a model that can handle your capacity requirements. If you don’t know what that requirement is yet because of limitations on your current firewall, request a demo unit from your Sales Engineer. Another option would be to disable decryption and determine SSL requirements based on this article.
    • Best practice questions to ask: What are you protecting? Can you reduce key size, decryption algorithms and/or TLS protocol version for lower security level traffic?

5. Helpful List of SSL Decryption Limitations to be aware of and additional tips:

  • Content cannot be decrypted when unsupported protocols or ciphers are used
    • TLS 1.2 cannot be decrypted before PAN-OS 6.0
    • TLS 1.3 cannot be decrypted on PAN-OS older than 8.0.14 or 8.0.4. PAN-OS 6.x and 7.x will not decrypt TLS 1.3 sessions. Sessions in browsers requiring strict compliance will instead display an error such as “TLS_13_DOWNGRADE_DETECTED” while other sessions may downgrade to TLS 1.2 as long as the firewall is set to allow lower levels of TLS in the decryption profile.
    • Before PAN-OS 6.0 the TLS 1.2 sessions were downgraded to TLS 1.1
    • Firewalls do not decrypt sessions where Diffie-Hellman key exchange is used for Forward Proxy Decryption in PAN-OS 7.1 and earlier releases
    • The following cipher suites are not supported for decryption
      • AES128-GCM-SHA256 prior to PAN-OS 7.1
Error
Device PA-5060-FW-01
Severity Error
Headline SSL Decryption Issues Have Occurred
Created Thursday, November 29, 2018 4:00:28 PM CST
Description
Indeni has noticed that the PAN device is experiencing issues decrypting traffic either inbound or outbound. The counters below indicate why decryption failed and at what rate have packets either dropped or have been left encrypted.
Reasons for Decryption Failures
proxy_url_request_pkt_drop: The number of packets get dropped because of waiting for url category request in ssl proxy
2 packets per second.
Remediation Steps
The counters should provide root cause analysis for the end user. However if additional documentation is necessary, please reference this document for additional assistance.
The issue's content may change over time. For up to date information go to:
SSL Decryption Issues Have Occurred

It is very important for systems operations teams and security operations teams to know before an issue with SSL Decryption occurs. If you cannot know before it occurs, it is still important that Indeni notify you of the root cause and allow for quick remediation. You might even have the solution in place by the time the service desk escalates any support tickets.

Indeni can ensure your Palo Alto Networks devices are all running smoothly, you can try out Indeni now or see all of our automation scripts in our knowledge explorer.

Brad Spilde
About the author
Brad Spilde