What should I do if my IPsec VPN connection fails?

Contents

Objective

Requirements

Introduction

Troubleshooting steps

Site to Site Mode

Client to Site Mode

Conclusion

Objective

This article provides detailed troubleshooting steps for IPsec VPN connection issues.

Follow the troubleshooting steps based on your IPsec VPN mode.

Requirements

• Omada/Omada pro/Festa Gateway

Introduction

Internet Protocol Security (IPsec) is a suite of protocols and services that provide security for IP networks. It is a widely used virtual private network (VPN) technology.

IPsec VPN requires remote users to install a dedicated VPN client or deploy a VPN gateway at the site. User access is checked by the client or gateway in terms of user authentication rules, security policy rules, or content security filtering.

Troubleshooting steps

Site to Site Mode

Step 1. Make sure the WAN IP addresses of both Site Gateways can ping each other.

Step 2. Log in to Controller, go to Settings > Network Security > Attack Defense, disable Block ping from WAN.

Step 3. On the PC connected to Gateway 1, ping the WAN IP of Gateway 2.

Step 4. Verify if Gateway 1 has a public IP, and Gateway 2 is behind a NAT device.

Fill in the Remote Gateway of Gateway 1's IPsec settings with either 0.0.0.0 or the public IP of the NAT device in front of Gateway 2. Set the Negotiation Mode of Gateway 1 and Gateway 2 to responder and initiator modes respectively, and use NAME as the identity.

Note: The NAME mode in Local ID Type and Remote ID Type may have different names in different vendor devices, such as FQDN.

Step 5. Verify if both Gateway 1 and Gateway 2 are behind NAT devices.

Configure NAT forwarding rules (UDP 500, 4500) for the NAT device in front of Gateway 1. Other configurations are the same as in last step.

Step 6. Check if the basic configurations of the two Site Gateways are matched: Remote Gateway, Local Subnet, Remote Subnet, Pre-shared Key, and WAN interface.

Step 7. Check if the Phase-1 configurations of the two Site Gateways are matched: IKE Version, Proposal, Exchange Mode, Local ID, and Remote ID. If there is a NAT device between the two Gateways, use NAME mode as the identity.

Step 8. Check if the Phase-2 configurations of the two Site Gateways are matched: Encapsulation Mode, Proposal, and Perfect Forward Secrecy (PFS). By default, ESP protocol is used because AH cannot pass through NAT.

Step 9. Check if Auto IPsec is being used. Auto IPsec may not establish a connection in Controller mode. It is recommended to use Manual IPsec.

Step 10. Confirm if the ISP allows IPsec-related traffic (UDP 500, 4500) to pass through.

Step 11. Verify if both Gateways have ACL rules that block IPsec-related traffic.

Client to Site Mode

Step 1. Make sure the client device can ping the Gateway’s WAN IP.

In Controller web, go to Settings > Network Security > Attack Defense, disable Block ping from WAN, then ping the Gateway’s WAN IP on the client device.

Step 2. Log in to Controller, go to Settings > Network Security > Attack Defense, disable Block ping from WAN.

Step 3. Confirm the client device model.

• If the client device is using the iOS operating system, there can be NAT devices in front of the Gateway. Both Local ID Type and Remote ID Type should be set to NAME mode.
• If the client device is a Samsung device, there can be NAT devices in front of the Gateway. Both Local ID Type and Remote ID Type should remain in the default IP Address mode.
• If the client device is an Android device (except Samsung devices), there should be no NAT devices in front of the Gateway. Set Local ID Type to IP Address mode and Remote ID Type to NAME mode.

Step 4. Confirm your Gateway configuration.

• Basic configuration: Fill in the Remote Host with either 0.0.0.0 or the public IP of the client device's front-end.
• Phase-1 configuration: Ensure IKE Version is consistent with the client. Proposal can be set to sha256-aes256-dh14. Select Responder Mode for Negotiation Mode. Configure Local ID Type and Remote ID Type according to step 2.
• Phase-2 configuration: Proposal can be set to sha256-aes256-dh14.

Step 5. Verify if the proposal matches.

Enable port mirroring for packet capture and capture the traffic packets of the WAN interface associated with the IPsec entry.

Use Wireshark to filter the ISAKMP packets. If the first ISAKMP packet replied by the Gateway contains the payload: Notify (41) - NOPROPOSALCHOSEN, it means the proposals do not match, as shown in the figure below.

The first ISAKMP packet initiated by the client contains all security proposals. You can set the Gateway's proposal to include the options specified in the packet.

Conclusion

If the issue of IPsec VPN is still not resolved with the above steps, please contact TP-Link via hotline or email for support.

Get to know more details of each function and configuration please go to Download Center to download the manual of your product.