IPSec issues - one packet at a time
Hey Guys,
Looking for anywhere to look that I may have missed with this one.
I have an IPSec Mesh VPN setup using multiple PAN devices.
SIte A as an example is a single virtual sys, single virtual router,
Site B is the problem site, which has 2 virtual routers on a single virtual sys.
OSF-PHQ works, running on one virtual router 1.
OSF-DMZ works for 1 packet, and then stops working.
Every time I run a "test vpn ike-sa gateway DMZ-OSF", ping packets work for a single packet.
You can see from the sequence number how it wasnt working between 11:10:02 (seq 78) , and 11:45:23 (seq 2150)
There are additional PA firewalls between the DMZ->OSF and PHQ->OSF, however there are no blocks shown on these firewalls. Whenever I run the "test vpn ike-sa gateway DMZ-OSF", I can see the ESP traffic on the firewall in between, which is all allowed.
This Mesh VPN was running previously using a Checkpoint Community VPN and no firewall rules/routing was changed. No blocks being seen, etc.
DH groups, keys have been checked. We have tried different authentication types, DH groups, IPSec crypto suites, etc..
There is no Maximum Lifesize set on the tunnels
We have static routes configured.
Running a test shows the correct tunnel interface on both ends.
I am at a complete loss in terms of what else we can check. I am waiting for a response from PA support, but just in case anyone here has seen similar behaviour, I thought I'd ask.
Cheers
Tried ikev1 and ikev2 - same behaviour on both albeit ikev1 requires me to clear vpn ike-sa gateway / clearvpn ipsec-sa tunnel in order to see that first packet come through. with Ikev2 I can use the test vpn and the first packet comes through
This is the counters on the source and destination while the ping is failing for the same ~9 second period
Source:
Global counters:
Elapsed time since last sampling: 9.406 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_outstanding 9 0 info packet pktproc Outstanding packet to be transmitted
pkt_alloc 20 2 info packet resource Packets allocated
session_allocated 9 0 info session resource Sessions allocated
session_installed 9 0 info session resource Sessions installed
session_servobj_timeout_override 9 0 info session pktproc session timeout overridden by service object
flow_tunnel_ipsec_esp_encap 9 0 info flow tunnel Packet encapped: IPSec ESP
flow_tunnel_encap_resolve 9 0 info flow tunnel tunnel structure lookup resolve
flow_fpp_sess_bind_notify 9 0 info flow offload Sess bind notification to FPP
appid_ident_by_icmp 9 0 info appid pktproc Application identified by icmp type
dfa_sw 9 0 info dfa pktproc The total number of dfa match using software
aho_sw_offload 9 0 info aho pktproc The total number of software aho offload
ctd_pscan_sw 9 0 info ctd pktproc The total usage of software for pscan
ctd_process 9 0 info ctd pktproc session processed by ctd
ctd_pkt_slowpath 9 0 info ctd pktproc Packets processed by slowpath
--------------------------------------------------------------------------------
Total counters shown: 14
--------------------------------------------------------------------------------
Destination
Global counters:
Elapsed time since last sampling: 9.240 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_outstanding 18 1 info packet pktproc Outstanding packet to be transmitted
pkt_alloc 20 2 info packet resource Packets allocated
session_allocated 9 0 info session resource Sessions allocated
session_installed 9 0 info session resource Sessions installed
session_servobj_timeout_override 9 0 info session pktproc session timeout overridden by service object
flow_np_pkt_xmt 9 0 info flow offload Packets transmitted to offload processor
flow_host_pkt_xmt 9 0 info flow mgmt Packets transmitted to control plane
flow_host_service_allow 9 0 info flow mgmt Device management session allowed
flow_tunnel_ipsec_esp_encap 9 0 info flow tunnel Packet encapped: IPSec ESP
flow_tunnel_encap_resolve 9 0 info flow tunnel tunnel structure lookup resolve
flow_fpp_sess_bind_notify 9 0 info flow offload Sess bind notification to FPP
appid_ident_by_icmp 9 0 info appid pktproc Application identified by icmp type
dfa_sw 18 1 info dfa pktproc The total number of dfa match using software
dfa_sw_offload 9 0 info dfa pktproc The total number of software post dfa offload
aho_sw_offload 9 0 info aho pktproc The total number of software aho offload
ctd_pscan_sw 9 0 info ctd pktproc The total usage of software for pscan
ctd_process 9 0 info ctd pktproc session processed by ctd
ctd_pkt_slowpath 18 1 info ctd pktproc Packets processed by slowpath
--------------------------------------------------------------------------------
Total counters shown: 18
--------------------------------------------------------------------------------