ServersExploring Windows 2003 Security: IP Security Page 2

Exploring Windows 2003 Security: IP Security Page 2

ServerWatch content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

With the conclusion of this brief overview of IPSec technology, let’s investigate the most relevant improvements in its Windows 2003 implementation:

  • New and improved management tools
  • Increased security of the master key (during the IKE negotiation stage)
  • More-flexible authentication methods
  • The ability to operate in NAT-ed environments
  • Interaction with Network Load Balancing

New and Improved Management Tools

Windows 2003 provides four primary tools for managing IPSec:

  1. IP Security Monitor: Unlike its fairly limited Windows 2000 counterpart IPSECMON.EXE, this new version is implemented as an MMC snap-in and displays much more extensive information about IPSec-related settings. Its primary purpose is monitoring IPSec policies and security associations for local and remote computers. For each computer, the tree pane lists Active Policy, Main Mode, and Quick Mode top-level folders, displaying currently assigned IPSec policy and parameters of main and quick mode IKE negotiations, respectively.
  2. IPSec Context: NETSH is a rather cumbersome (due to lack of a friendly GUI interface) command line utility, introduced in Windows 2000, that can be used to modify and display a number of different network configuration settings. Its main benefits are speed, its capability to be executed in non-GUI scenarios (e.g., via Telnet), and scriptability. In Windows Server 2003, IPSec context enables administrators to take advantage of these features when managing IPSec settings. IPSec context also offers a number of additional configuration options not available from the IP Security Monitor, such as:
    • Configure bootmode property, which determines the type of traffic allowed during boot time prior to the initialization of IPSec Services. Choices include permit, block, or stateful (which limits inbound traffic to communication initiated locally) values. This can be set directly from the command prompt, by running
      netsh ipsec dynamic set config property=bootmode value=permit
    • Configure bootexemptions property, should you opt to block traffic. This is done by specifying exemptions in the form Protocol:SourcePort:DestinationPort:Direction, e.g. TCP:80:80:outbound, which requires executing the command of:
      netsh ipsec dynamic set config property=bootexemptions value="TCP:80:80:outbound"
    • Set IPSec and IKE diagnostics levels and logging intervals, by using ipsecdiagnostics (possible values range from 0 to 7), ikelogging (0 or 1), and ipsecloginterval (between 60 and 86400 seconds) properties.
    • Set persistent policy for enhanced security using the set store command with the persistent option. This way, selected IPSec policy gets assigned to the local system at startup, before the application of those assigned through local or Active Directory group policy setttings (and it remains assigned afterward).

  3. IP Security Policies: Although the interface remained relatively unchanged, there is no longer a limitation of two choices (My IP Address and Any IP address) when specifying the source and destination IP addresses of IP Filters for IPSec policies. In particular, it is possible to set base filters on a specific DNS name, an IP subnet, or even apply them to communication with source or destination being evaluated dynamically, by selecting such options as default gateway, DNS, WINS, or DHCP server.
  4. Group Policy Modeling and Group Policy Results: These include IPSec extensions, which simplifies both designing and troubleshooting of IPSec policies. For more information about Group Policy related Windows 2003 improvements, refer to the previous article in this series.

More Secure Master Key (During the IKE Negotiation Stage)

As described previously, one of the tasks performed during the first phase of IKE negotiation is the generation of the IKE master key, which, in turn, is used to secure communication during the second phase (and frequently is re-used to generate a new session key, when its expiration interval passes). Starting with Windows Server 2003, you can increase security by specifying a longer key (2048 bit) to be used by the Diffie-Hellman algorithm, based on which a master key is created.

Diffie-Hellman parameters are configurable from the Key Exchange Security Methods dialog box. (They are specified using a separate setting for each of the security methods listed there.) This dialog box is reached by first clicking on the Settings … button on the General tab of the IPSec policy Properties dialog box and then clicking on the Methods… button on the Key Exchange Settings dialog box.

More-Flexible Authentication Methods

As already explained, three authentication methods are available when establishing an IPSec session. Starting with Windows 2003, for certificate-based authentication, you can enable the certificate-to-account mapping option, as long as you operate in Active Directory environment. To turn this option on, launch the IP Security Policies MMC snap-in, double-click on the policy to be configured, choose a rule from the IPSec Policy properties dialog box to be modified (listed on the Rules tab), select the Authentication Methods tab of Edit Rule Properties dialog box, and, finally, edit the certificate authentication method (by checking on the checkbox labeled “Enable certificate to account mapping”). Once this setting is enabled, you can take advantage of user rights (such as Access this computer from the network or Deny access to this computer from the network) assigned via Group Policies to target computers to control which ones will be able to establish IPSec session.

Network Address Translation Awareness

One of the most significant problems with IPSec implementation in Windows 2000 (typically used for VPN L2TP/IPSec connections) was its incompatibility with Network Address Translation (NAT). This meant that two computers could not communicate using IPSec if their traffic was passing a NAT router. Although this could be circumvented by applying PPTP tunneling (with its inherent encryption methods) rather than IPSec/L2TP, or by setting a tunnel between an IPSec client and a firewall (before a NAT device was reached), these workarounds frequently compromised overall security.

Microsoft resolved this problem in Windows 2003. Its implementation of L2TP/IPSec is no longer based on proprietary IPSec encapsulation mechanism but rather on IETF RFP standards, incorporating NAT Transparency (NAT-T). Changes have also been made to RRAS NAT implementation. In addition, recently published updates to Windows XP and Windows 2000 L2TP/IPSec NAT-T VPN client software, as well as VPN client software for older versions of Windows (downloadable from the Microsoft Web site) allow connecting to Windows 2003 VPN server using L2TP/IPSec residing behind a NAT device from practically any version of Windows. However, the NAT device will likely need to be configured to allow the passing of L2TP (UDP ports 500 and 1701), NAT-T (UDP port 4500), and ESP (IP protocol 50) traffic (specifics would depend on whether the NAT device resides on the client side, the server side, or both). Obviously, there must also be some way of authenticating the IPSec session (typically via certificate or preshared key).

For more information about the update, refer to Microsoft Knowledge Base article 818043.

Interaction With Network Load Balancing

Windows 2003 Network Load Balancing provides support for IPSec connections. The steps necessary for such configuration are outlined in Microsoft Knowledge Base article 820752, so we will provide only a short summary here. First, you must ensure that appropriate IPSec policy apply to the shared IP address of the cluster that will be used for secure communication. Next, using NLB Manager, in the Cluster Properties dialog box, on the Port Rules tab, allow UDP traffic on ports 500 and 4500 and set Affinity to Single or Class C.

Get the Free Newsletter!

Subscribe to Daily Tech Insider for top news, trends & analysis

Latest Posts

Related Stories