3rd IPSec Interoperability Testing Results

(May 12 - Jun 4, 1999)

NTT (Nippon Telegraph and Telephone Corporation) performed three weeks of IPSec interoperability testing amongst various different vendors products at one of their testing facilities in Tokyo, Japan.


This testing was the 3rd interoperability test perform by them. The 1st testing was held earlier and only tested manual key exchange between vendors. The 2nd testing included automatic key exchange between vendors. This 3rd interoperability test was planned for 1Q 1999, and was to be performed over the internet using CAs, but due to the final IPSec specs being delayed as well as only limited (and proprietary) CA support by a majority of manufacturers, a continuation of the 2nd testing, including SA lifetime re-keying was performed instead. Thus a 4th testing probably including the "over the internet" and "CA support" will likely be performed in the future (time and final specifications are yet to be fixed).


After looking at the results, it is apparent that IPSec still has a long way to go, and those vendors who do claim IPSec compatibility only do so partially...

Before I go any further, I wish to stress one MAJOR important issue which you MUST NOT become confused with. Being 100% IPSec compliant and being 100% other vendor compatible are two (2) totally different issues and should not be mixed or confused under any circumstance. Thus, regardless of whether 20 vendors all have compatibility with each other, unless they also have 100% IPSec compatibility, they are not necessarily IPSec compliant albiet they can communicate with each other.

I was present at both the 2nd & 3rd Interoperability testing performing the setup and testing of IRE's SafeNet/Soft-PK^TM and was also able to see how compatible others were with the IPSec specifications.

IRE's SafeNet-PK^TM has already received the ICSA IPSec compatible certificate, but even so, there were still numerous vendors with whom keys couldn't be established or keys could be established but no traffic was sent after that in both the 2nd and 3rd Interoperability testing because many gateway vendors only support other gateways, but don't support the IPSec Client functionality.

This is due to the fact that a good many vendors tested were still not fully IPSec compatible. Of all the other vendors products I tested IRE's SafeNet-PK^TM against, many were unable to talk directly with an IPSec client. The majority of these devices were Gateways and were able to communicate with other vendor's Gateway products, but were unable to set up the SPI properly to handle Gateway-to-Client communications. At the 2nd IPSec testing, Ravlin also used their client in the testing, but this 3rd time around, they only used their gateway product. Thus ANY/ALL IPSec compliant Client software on the market today will experience similar results until these gateway vendors support IPSec 100% (including Client functions) within their gateway. So regardless of whether you are fully IPSec compliant or not, until ALL vendors become fully IPSec compliant (and many of them still have a long way to go), multi-vendor interoperability will definately remain an issue at hand for at least the rest of 1999 and maybe even into the first quarter of 2000.

The original Japanese can be found by clicking on the results of the 3rd Interoperability test, or the results of the 2nd Interoperability testing or on the original results of the 1st Interoperability testing which I was not present for as well as various other information concerning the testing (and which I have not translated).

NTT has finally released the Japanese 3rd Test results, and there are a lot of them. I've only translated part of them and will post the rest when ever I get a chance to translate them (probably one or two at a time).