November 2007 Archives

I now have a v6-capable tcpwrappers built on AIX. Well, I think it's functional. According to tcpdmatch(8), it's handling hostnames with AAAA records. I have not been able to test it with a real connection, since sshd is statically linked against the old v4-only libwrap.

Static linking is bad, mmkay. Please, stop.

I'll rebuild OpenSSH on Monday morning.

Btw, I have confirmed that tcpwrappers on OS X 10.4, CentOS 4.x and Solaris 10 are IPv6-capable.

Readers of this blog will know that I claim that IPv6 adoption isn't as painful as some make it out to be. Well, after using IPv6 for several years, I've encountered my first real issue.

Yesterday, I was testing the builtin Kerberos library for v6 support on an AIX 5.3 machine. Since I was using the machine anyway, I decided to assign it a static IPv6 address, rather than using auto-configuration. A colleague in ET is the primary admin for the machine. He installed TCPwrappers on the box, but didn't use the IPv6-capable version.

So, now, I can't ssh into it over v6. But since there's an AAAA record for it, my ssh clients try to connect over IPv6 first. TCPWrappers immediately denies the connection, so there's about 3.7 microseconds of additional latency to establish the session (I happen to be only one hop from the box).

For the moment, I've fixed the problem by putting the following into my ~/.ssh/config:

Host foo.et-test.psu.edu
  AddressFamily inet

This forces OpenSSH only to use IPv4 for host foo.et-test.psu.edu.

I'm off to compile the IPv6-capable version of TCP Wrappers. Since I've never built anything on AIX, this should be fun.

IPv6 Topology

| | Comments (0) | TrackBacks (0)

I just submitted my data for CAIDA's, 2007 IPv6 Topology Collection.

In 2005, CAIDA did a comparision of AS topology for IPv4 -vs- IPv6. The results, not surprisingly, found that the IPv6 internet had significantly poorer peering than IPv4. They also found much better IPv6 peering in Europe and Japan than in the US (while the US had better IPv4 peering). CAIDA is rerunning the study two years later. I'm very interested to see the results. I'm curious how the number of IPv6-connected sites has changed, as well as how well interconnected they are.


UMich code and IPv6

| | Comments (0) | TrackBacks (0)

The University of Michigan has developed a lot of open-source software. For example, PSU makes heavy use of their CoSign and radmind products. Their graphical SFTP client, Fugu is also very popular. Sadly, none of these are IPv6-clean.

As I mentioned previously, I've got patches to make CoSign IPv6-clean. This afternoon I looked at the radmind and Fugu sources, and it looks like they share some low-level networking libraries with CoSign. I suspect I'll be able to easily adapt my CoSign patches for them.

I setup the VMs for my CoSign testing this afternoon. I hope to actually get to testing things later this week.

Perhaps I'll make a blog entry about converting legacy C code to use the new IPv6-capable APIs. It's generally a straightforward conversion.

PSU had an IPv6 outage on our border router from yesterday until this morning. I wanted to confirm that we couldn't, in fact, connect to PSU's border router from an external IPv6 host. Two useful sites for doing that are sixxs's traceroute and the IPv6 Portal's Looking Glass. The latter is especially useful as it lets you do a ping, ping6, traceroute or traceroute6 to a host from an external, IPv6-connected site.

SMTP & IPv6

| | Comments (0) | TrackBacks (0)

I just noticed that two colleagues of mine have IPv6-accesssible SMTP systems. One is at PSC, the other at SWITCH (who also has v6-accessible authoritative DNS :).

I sent both of them test emails and confirmed that ET's SMTP server does use IPv6 preferentially over IPv4. It's nice to see a (small) part of the Internet's mail infrastructure switching over to IPv6.


$ dig -t MX psc.edu

; <<>> DiG 9.3.4 <<>> -t MX psc.edu
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56239
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 3, ADDITIONAL: 5

;; QUESTION SECTION:
;psc.edu. IN MX

;; ANSWER SECTION:
psc.edu. 172226 IN MX 20 mailer2.psc.edu.
psc.edu. 172226 IN MX 20 mailer1.psc.edu.

;; AUTHORITY SECTION:
psc.edu. 104552 IN NS dns2.itd.umich.edu.
psc.edu. 104552 IN NS charon.psc.edu.
psc.edu. 104552 IN NS dns1.psc.edu.

;; ADDITIONAL SECTION:
mailer1.psc.edu. 172153 IN A 128.182.58.100
mailer1.psc.edu. 172153 IN AAAA 2001:5e8:1:3a::64
mailer2.psc.edu. 172153 IN A 128.182.66.106
dns1.psc.edu. 104552 IN A 128.182.58.105
charon.psc.edu. 104552 IN A 128.182.65.6

;; Query time: 2 msec
;; SERVER: 2610:8:6800:1::4#53(2610:8:6800:1::4)
;; WHEN: Tue Nov 27 10:01:05 2007
;; MSG SIZE rcvd: 234

$ dig -t MX switch.ch

; <<>> DiG 9.3.4 <<>> -t MX switch.ch
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21949
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 13

;; QUESTION SECTION:
;switch.ch. IN MX

;; ANSWER SECTION:
switch.ch. 86400 IN MX 10 medel.switch.ch.
switch.ch. 86400 IN MX 10 aletsch.switch.ch.

;; AUTHORITY SECTION:
switch.ch. 143509 IN NS merapi.switch.ch.
switch.ch. 143509 IN NS scsnms.switch.ch.

;; ADDITIONAL SECTION:
medel.switch.ch. 300 IN A 130.59.108.38
medel.switch.ch. 300 IN A 130.59.108.39
medel.switch.ch. 300 IN AAAA 2001:620::14:214:4fff:fe75:4774
medel.switch.ch. 300 IN AAAA 2001:620::14:214:4fff:fe75:4775
aletsch.switch.ch. 300 IN A 130.59.138.26
aletsch.switch.ch. 300 IN A 130.59.138.27
aletsch.switch.ch. 300 IN AAAA 2001:620::1b:203:baff:febe:9059
aletsch.switch.ch. 300 IN AAAA 2001:620::1b:203:baff:febe:905a
merapi.switch.ch. 68848 IN A 130.59.211.10
merapi.switch.ch. 68848 IN AAAA 2001:620::5
scsnms.switch.ch. 81739 IN A 130.59.10.30
scsnms.switch.ch. 81739 IN A 130.59.1.30
scsnms.switch.ch. 81739 IN AAAA 2001:620::1

;; Query time: 116 msec
;; SERVER: 2610:8:6800:1::4#53(2610:8:6800:1::4)
;; WHEN: Tue Nov 27 10:01:16 2007
;; MSG SIZE rcvd: 395

WebAccess and IPv6

| | Comments (0) | TrackBacks (0)

If you are IPv6-enabling a website protected by WebAccess, you might need to make a change to your web server's configuration. Specifically, you might need to set CosignCheckIP never in your Apache config. Otherwise, connections over IPv6 won't be authenticated. You can read more about why this happens in the ITS wiki.

Penn State's WebAccess product is based on CoSign, an open-source web single-sign-on system. The CoSign source code is not IPv6-clean, so even if ASET were to assign IPv6 addresses to the WebAccess servers, we'd still have problems. I'm working on a patch to CoSign to make it IPv6-clean. After the break, I'll do some more testing and submit it to the CoSign project for integration.

I had some freetime this afternoon, so I whipped up a script to check IPv6 deployment at the Big Ten. I checked for AAAA records for authoritative nameservers and SMTP servers.

Only University of Illinois, Urbana-Champagne and Penn State had any IPv6 presence. We both have one IPv6-accessible authoritative nameserver.

Yes, otc2.psu.edu is IPv6-accessible. It's answering DNS and NTP queries over IPv6. At the moment, it's not answering recursive queries over IPv6, but TNS is aware of this issue and hopes to have it corrected soon.

So, hats off to TNS for this. It's good to be ahead of the pack.

IPv6 Weekend Update

| | Comments (2) | TrackBacks (0)

Over the weekend, I had two minor issues with IPv6.

Issue 1: Misconfigured Routing

Per my previous entry, the ITS Wiki is now IPv6-accessible. However, we didn't have networking configured quite right on the box.

When I got home, I noticed extremely high latency when loading the wiki over IPv6. It turns out that I wasn't actually loading it over IPv6. Since v6 routing was broken on the server, my browser had to wait for v6 connections to time out and then fail over to IPv4 for each request to the server.

The server is running CentOS 5. We had forgotten that on Linux, you have to explicitly configure the IPv6 default router. Technically, you shouldn't have to, since IPv6 routers multicast their address, and hosts are supposed use these router advertisements to auto-configure their routing table. The good news was that this was a one-line fix.

In /etc/sysconfig/network, I had to add:

IPV6_DEFAULTGW=2610:8:6800:1::1

and restart networking. After that, IPv6 worked fine.

Issue 2: Non-nameserved 6to4 hosts

At home, I use 6to4 tunneling. This lets me connect to IPv6 hosts even though my cable modem provider doesn't offer IPv6. There is a special range of IPv6 addresses reserved for 6to4 tunneling (the 2002/16 range). Since these addresses are automatically configured, they're not normally nameserved.

As I've mentioned previously, many of ET's hosts have IPv6 enabled. I tried to ssh into one of these hosts, but my connection kept getting dropped. Since I have IPv6 enabled on my laptop at home, it was trying to use IPv6 to connect to the server (which also has IPv6). However, the server uses TCP wrappers to restrict access to ssh. The ACL looks like this:

$ cat /etc/hosts.allow
sshd: .psu.edu .hsd1.pa.comcast.net

Since my 6to4 address isn't nameservered, I can't connect. As a quick fix, I forced ssh to use IPv4 (using the -4 flag). Technically, this isn't an IPv6 issue. The issue is that I have a non-nameserved IP address.

We're still investigating how best to address this in ET. We could modify the TCP wrapper ACLs to allow 6to4 addresses from Comcast's State College range, but that coudl get messy if Comcast renumbers their network.

The ITS Wiki is now IPv6-accessible!

Both the web server and its authoritative nameservers are IPv6-accessible:

$ dig wikispaces.psu.edu any

; <<>> DiG 9.3.4 <<>> wikispaces.psu.edu any
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53202
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 4

;; QUESTION SECTION:
;wikispaces.psu.edu. IN ANY

;; ANSWER SECTION:
wikispaces.psu.edu. 21600 IN SOA et1.et-test.psu.edu. hostmaster.et-test.psu.edu. 2007111600 3600 1800 1209600 3600
wikispaces.psu.edu. 21600 IN NS et1.et-test.psu.edu.
wikispaces.psu.edu. 21600 IN NS et2.et-test.psu.edu.
wikispaces.psu.edu. 21600 IN A 128.118.27.121
wikispaces.psu.edu. 21600 IN AAAA 2610:8:6800:1::a

;; ADDITIONAL SECTION:
et1.et-test.psu.edu. 21600 IN A 128.118.27.4
et1.et-test.psu.edu. 21600 IN AAAA 2610:8:6800:1::4
et2.et-test.psu.edu. 21600 IN A 128.118.27.132
et2.et-test.psu.edu. 21600 IN AAAA 2610:8:7800:9::4

;; Query time: 0 msec
;; SERVER: 2610:8:6800:1::4#53(2610:8:6800:1::4)
;; WHEN: Fri Nov 16 18:58:52 2007
;; MSG SIZE rcvd: 259

I thought it might be useful to document how I use IPv6 on my workstation. I'm running Mac OS X 10.4.11 on a Mac Pro. I use IPv6 for DNS, NTP, SSH, HTTP (including RSS), IMAP and SMTP daily.

As I mentioned previously, ET's DNS servers support queries over IPv6. I use IPv6 exclusively for DNS queries. So far, I haven't noticed any apps that break. My machine is issuing many extra DNS queries, since it checks for AAAA records for every host I connect to. This causes some extra latency, but I haven't found it to be noticable. I do reguarly capture my DNS queries with WireShark. In a later entry I'll look at DNS latency with IPv6.

Our NTP servers support IPv6. OS X's ntpd is IPv6-capable, so my Mac syncs over IPv6. From an end-user perspective, there's nothing different than using IPv4: I type in the NTP server's hostname, and my clock gets synced.

For web browsing, I use FireFox and Opera. Both support IPv6; but FireFox required a minor configuration change to enable IPv6. Occassionally, I'll experience some extra latency when loading an IPv6-connected web site. There is a useful FireFox extension, ShowIP, which displays the IP address of the web sites. It's a convenient way to check if your favorite sites support IPv6. Several of ET's internal web sites, such as our internal wiki, are IPv6-accessible.

My RSS reader, NewsFire, supports IPv6, and I have several IPv6-accessible RSS feeds in my feed list.

I use Thunderbird for email. ET's IMAP and SMTP servers are IPv6-enabled, and Thunderbird connects to them over IPv6. Like FireFox, it required a small config change to enable IPv6. OS X's mail client also supports IPv6 for IMAP, but SMTP-over-IPv6 is broken in 10.4 on Intel Macs. This has been fixed Leopard.

Several of our ssh servers are IPv6-enabled. Again, there's nothing different from an end-user perspective. If you look in my ~/.ssh/known_hosts file, you'll see IPv6 addresses for some entries. My workstation runs sshd, and I can connect to it over IPv6 as well.

I use Kerberos heavily on my machine: for logins, for unlocking the screensaver, for email, etc. The ITS Kerberos servers aren't reachable over IPv6, but they do support embeddeing IPv6 addresses in Kerberos tickets. I use addressless tickets, though. There can be issues with addressed tickets when connecting from an IPv4-only host to a dual-stacked host. For this and other reasons, I suggest using addressless tickets. In fact, addressless tickets have been the default in MIT Kerberos 1.3.

Occassionally, I need to bring up a test KDC. I tend to use Sun's SEAM version of Kerberos, since I have plenty of Solaris 10 machines handy. When I do so, OS X will obtain tickets over IPv6 without issue.

IPv6 certification

| | Comments (0) | TrackBacks (0)

I primarily run OS X, Solaris 10 and Vista. Two of those OSes are now IPv6 Ready Phase 2 certified.

Vista received certification last month.

Solaris 10 Update 4 is also Phase 2 certified. U4 added a DHCPv6 client, which is required for Phase 2. The code was developed as an OpenSolaris project.

Sadly, Mac OS X is not IPv6 Ready certified, either at Phase 1 or 2. This is particularly sad because Apple has been an early proponent of IPv6 (it's been supported since Mac OS X 10.2 and enabled by default since 10.3). Mac OS X also lacks a DHCPv6 client.

We don't have DHCPv6 enabled in ET, but I'm anxious to try it, as it should let v6-enabled clients get DNS server addresses automatically. ET's authoritative DNS servers are v6-enabled, and we've noticed no problems so far.

Welcome

| | Comments (0) | TrackBacks (0)

The Emerging Technologies group in ITS has been running IPv6 on our network for several years. In the past two years, we've made an earnest attempt to use it actively. Recently I've been making a bit of a pest of myself advocating for IPv6 adoption on campus.

This blog will document my experiences living in an IPv6-enabled network, both at work and at home. Frankly, unless you're handy with WireShark, you won't notice much of a difference, which is exactly how IPv6 is supposed to work.

For more information, see my IPv6 working notes in the ITS wiki. You might also find my IPv6 del.icio.us feed useful.