Tag Archives: HLB

ADFS 3.0 and Hardware Load Balancing – the Lost Documentation

Article updated October 2014 to include http based health check probe as a workaround, made available with August 2014 Update Rollup for Windows Server 2012 R2.

BACKGROUND AND PURPOSE

ADFS and Federated Identities are common components for Enterprise Office 365 customers necessary to allow Single Sign-on, use of Claim Rules and immediate account lock-outs etc. As the availability the ADFS service decides the availability of Office 365 (if you can’t authenticate you can’t use the service), load balancing is a must-have. In most cases, Hardware Load Balancing (HLB) is used, and for ADFS 2.0 (Windows Server 2008/2008R2) and ADFS 2.1 (Windows Server 2012) this have never been a problem. You deployed your ADFS STS and/or ADFS Proxy, and published them with your Hardware Load Balancer like “any other SSL site” and it just worked.

Today, ADFS have with Windows Server 2012R2 reached version ADFS 3.0 – and the full “out-of-the-box-support” for using “any Hardware Load Balancer” is not as obvious any more. Making it  work is a bit more tricky, depending on what HLB you are using. The purpose of this article is to present my lessons learned from deploying ADFS 3.0 with HLB – and to save you from hours of trouble shooting when getting your own ADFS 3.0 HLB up and running with Office 365.

ADFS 3.0 AND HARDWARE LOAD BALANCING – WHAT COULD POSSIBLY GO WRONG?

ADFS 3.0 and Web Application Proxy (WAP) in Windows Server 2012R2 uses an extension to the TLS SSL protocol called Server Name Indication – SNI (RFC 4366). This extension allows web servers to present host names when handshaking SSL, so that multiple SSL sites can be hosted on a shared IP-address and port (443) – just like the concept of host headers in good old http have always allowed us to do. This is great because it allows preservation of IP-addresses which also can reduce a total cost of ownership for a service (public IP-addresses does not come for free these days).

What is not so great is that the SNI extension is not yet supported by “everything else in the world” – such as very old web browsers – and Hardware Load Balancers. Using a HLB that does not have a clue about SNI will not work, and that is the topic of this blog post.

If you are interested to read more about SNI in Windows Server 2012R2 – and how to solve issues that can occur, I can recommend Ian Parramore’s two excellent blog post on TechNet, which also have been a great inspiration and source of content to this post, found here, and here.

HOW DO I KNOW THAT MY HLB DOES NOT SUPPORT SNI?

If your HLB does not support SNI, you will notice it eventually when attempting to configure a load balancing and you don’t get any response on port 443 from any of the server IP-addresses. (that’s usually when you spend a day or two troubleshooting firewall ACL’s and routing without any luck). My best advice is to look for SNI-related settings in the HLB administrative interface and read through the release notes for the software version you are running on the appliance(s).

Examples of observed behaviour when attempting to configure HLB with a non-SNI compliant device is a “Http/1.1 Service Unavailable” message from a Citrix NetScaler device and a “Network Error (tcp_error) A communication error occured: Connection refused” message from a BlueCoat device.

Big-IP/F5 customers may turn to a technical article written by Greg Coward for assistance in configuring the Big-IP to work with ADFS 3.0/WAP, found here.

MY HLB DOES NOT SUPPORT SNI – WHAT CAN I DO?

CHECK FOR UPDATES: Before attempting to fix the issue on the Windows Server side – you should always see if there is an available software/firmware update for your HLB device that includes SNI support. There are two workarounds available (see below), but having native support is usually preferred as it reduces the overall complexity of your solution.

WORKAROUND #1 : ADD A HTTPS (PORT 80) BASED HEALTH CHECK PROBE

NOTE: This functionality was made available with the August 2014 Update Rollup for Windows Server 2012R2, found here. Running at least this update rollup level on your ADFS/WAP servers is required for the following workaround to work.

By adding a http-binding on the ADFS/WAP server, your HLB can do the health check probing using the http protocol and thus bypassing SSL and SNI completely. The http endpoint will respond with HTTP 200 OK if available if queried to http://servername-or-ip-address/adfs/probe, and will not automatically allow your ADFS traffic to use HTTP, they will function just as health check probes. Follow these steps to create such binding:

  1. Make sure you have installed all available Windows Updates on the ADFS/WAP machine – at least August 2014 Rollup.
  2. Launch the Windows Firewall with Advanced Security MMC on the first WAP server
  3. Go to Inbound Rules
  4. Create a new Inbound Rule (New Rule in Action Pane). Suggested name: “ADFS HTTP Health Check Probe”
  5. Configure the rule for TCP protocol, local port 80 (specific port) and Allow traffic (All ports as Remote port). See overview of expected result in this picture from Ian Parramore’s blog.
  6. Repeat steps on other ADFS/WAP machines.
  7. Configure HLB to do health check probe using the http protocol and this specific endpoint: http://servername-or-ip-address/adfs/probe

WORKAROUND #2 (RECOMMENDED): ADD A FALLBACK BINDING TO 0.0.0.0:443

It is possible to create a binding to make ADFS 3.0 and/or WAP to respond to SSL requests made out to ip-adresses. This is the recommended workaround as you can’t fully disable the SNI functionality completely. The process of adding this fallback binding consist of finding out the certificate thumbprint of your desired SSL certificate (added with private key to the Local Computer\Personal store) and then binding that thumbprint to all ip-adresses and for the ADFS or WAP application.

Follow these steps on all your ADFS 3.0 servers to add the fallback binding (and make your non-SNI compliant HLB be able to see your ADFS servers):

  1. Make sure that you have installed all available updates for Windows Server 2012R2 after adding and configured the ADFS STS or WAP Proxy role.
  2. Open a Command Prompt as administrator
  3. Run the following command:
    netsh http show sslcert
  4. You will see a list of SSL Certificate bindings.
  5. Mark and copy the ‘Certificate Hash’ value. Paste it to a temporary place (i.e Notepad)
  6. Mark and copy the ‘Application ID’ value. Paste it to the temporary place. The Application ID is what will associate the binding with ADFS 3.0 (for the internal STS servers) and WAP (for the ADFS Proxy).
  7. Now run the following commmand, where you insert the noted ‘Certificate Hash’ and ‘Application ID’ values from above (keep the { } characters):
    netsh http add sslcert ipport=0.0.0.0:443 certhash=Insert_Certificate_Hash_Here appid={Insert_Application_ID_here}
  8. Restart machine and repeat steps for remaining ADFS 3.0 machines.

After adding the fallback binding, your HLB should be able to reach your ADFS servers normally, and you can continue to configure and fine tune your load balancing. Be aware that if you replace your certificate in the future (when it expires), you must perform these steps again (a new certificate means a new certificate hash) – worth mentioning in your system documentation.

SUMMARY AND DISCLAIMER

Adding a http health check probe or a fallback binding can help us fix our load balancing, but my hope is that SNI support in HLB devices will become more of a standard than it is today. Like stated in the article, looking for native support with a software/firmware update and proper documentation from your HLB vendor is always recommended before applying the fallback binding workaround. If you are aware of SNI-related knowledge base articles and software updates from HLB vendors, please share them with me in the comments field below or on Twitter (@JesperStahle) and I will add them to this article.