Kerberos, and therefore LDAP with GSSAPI, has issues with servers behind NAT, or anywhere the forward DNS lookup does not match the reverse DNS lookup.
For instance, in our lab we have an OpenLDAP LDAP server:
$ dig +noall +answer ldap.itlab.stanford.edu ldap.itlab.stanford.edu. 207 IN CNAME idp.itlab.stanford.edu. idp.itlab.stanford.edu. 200 IN A 220.127.116.11
However, since it's running on an EC2 instance, the reverse lookup returns a record that matches neither the alias (ldap.itlab.stanford.edu), nor the "real" name (idp.itlab.stanford.edu).
$ dig +noall +answer -x 18.104.22.168 22.214.171.124.in-addr.arpa. 300 IN PTR ec2-54-189-121-117.us-west-2.compute.amazonaws.com.
To further complicate matters, DNS queries return the public IP associated with the EC2 instance; internally, the instance has a different IP address on eth0:
$ ip addr show dev eth0 2: eth0:
mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 22:00:0a:51:04:32 brd ff:ff:ff:ff:ff:ff inet 10.81.4.50/26 brd 10.81.4.63 scope global eth0 ...
Anything running on the instance will see the system's IP address as 10.81.4.50; anything outside AWS will see the IP address as 126.96.36.199.
Fixing the backend
The first thing we need to fix is how processes on the EC2 instance identify the system. This can be handled with entries in
127.0.0.1 localhost ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 188.8.131.52 idp.itlab.stanford.edu ldap.itlab.stanford.edu 10.81.4.50 ldap.itlab.stanford.edu idp.itlab.stanford.edu
These entires can be added or updated when the system boots, using the EC2 metadata:
$ echo $(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) 10.81.4.50 $ echo $(curl -s http://169.254.169.254/latest/meta-data/public-ipv4) 184.108.40.206
SSH works if the client acquires address-less tickets (Stanford's default), and if the client is also configured to not use reverse DNS lookups to determine the service name. For both Heimdal and MIT libaries, these settings are in /etc/krb5.conf in the libdefaults section, as noaddresses and rdns, respectively:
[libdefaults] default_realm = itlab.stanford.edu noaddresses = true rdns = false
If you're using MIT libraries and OpenLDAP clients, you also need this:
dns_canonicalize_hostname = false
These options are all describe in the MIT Kerberos Docs
SSH with GSSAPI now works as expected (using idp.itlab.stanford.edu, since that's the "real" name of the system):
$ssh -v firstname.lastname@example.org OpenSSH_6.7p1 Debian-5+deb8u3, OpenSSL 1.0.1t 3 May 2016 debug1: Connecting to idp.itlab.stanford.edu [220.127.116.11] port 22. debug1: Connection established. ... debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,keyboard-interactive ... debug1: Next authentication method: gssapi-with-mic debug1: Authentication succeeded (gssapi-with-mic). Authenticated to idp.itlab.stanford.edu ([18.104.22.168]:22). ... Linux idp.itlab.stanford.edu 3.2.0-4-amd64 #1 SMP Debian 3.2.78-1 x86_64 admin@idp:~$
Without modifications, attempts to query the ldap server result in errors like this:
$ ldapsearch -Q -LLL uid=swl cn SASL/GSSAPI authentication started ldap_sasl_interactive_bind_s: Local error (-2) additional info: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)
With the Heimdal libraries, the krb5.conf settings do not appear to affect the OpenLDAP
ldapsearch client. However, with the MIT libraries, dns_canonicalize_hostname must be set to false.
ldapsearch uses a SASL library to use GSSAPI, and that library handles service/server name canonicalization internally. The SASL library's canonicalization can be disabled in ldap.conf with this line:
Now GSSAPI LDAP binding works:
$ ldapsearch -Q -LLL uid=swl cn dn: uid=swl,cn=people,dc=itlab,dc=stanford,dc=edu cn: Scotty Logan
What about Macs?
MacOS X comes with the Heimdal clients and libraries, and appears to have canonicalization disabled.
MacOS X also includes the OpenLDAP clients and libraries, but does not appear to support SASL_NOCANON in /etc/openldap/ldap.conf. Instead the
-N option must be used with
ldapsearch (this option also works on Linux).