Forum Discussion

quattroginger's avatar
quattroginger
Icon for Nimbostratus rankNimbostratus
Nov 26, 2019

npath troubleshooting

i have npath configured where the VIP, both pool members and self IP are within the same subnet. i believe i am seeing what i should through a tcpdump. i see the SYN from the client, and the ACK from the client. unfortunately i dont have access to the servers on the back end to complete a packet capture but i am seeing resets in my dump from my VIP to the client. is this normal?

 

i am being told the application functions for a short period then no longer works until a reboot is completed.

  • Is your virtual a fastl4 virtual with no SNAT with a profile that allows Loose Initiation and Loose Close?

    • quattroginger's avatar
      quattroginger
      Icon for Nimbostratus rankNimbostratus

      i used the iapp to build it out but it looks like Loose Close is enabled but Loose Initiation is not.

      • Simon_Blakely's avatar
        Simon_Blakely
        Icon for Employee rankEmployee

        If it is truly an npath configuration, you will need Loose Initiation, as the BigIP will not see the [SYN,ACK] from the pool member to complete the connection initiation.

  • had my customer test connectivity again from the client side and he is still receiving the same error of "the server has closed the connection unexpectedly". i did notice the below in the logs during connection attempt. Additionally, I am going to try and get someone with access to server to run a capture as well.

     

    Syncookie HW mode activated, server = 192.168.100.42:XXX, HSB modId = 2

    Syncookie HW mode exited, server = 192.168.100.42:XXX, HSB modId = 2 from HSB

  • Questions to ask: Did this ever work successfully? When did it stop? Were any changes (such as patches or hardware changes) made to the server devices? Was a new server recently added to the pool?

    The configuration of the servers for npath routing is equally important to the F5 config. Your comment about everything working fine for a while and then requiring a reboot to fix it sounds a lot more like a server issue than the F5.

     

    For reference: https://techdocs.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm-implementations-13-0-0/4.html

    https://techdocs.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm-implementations-13-0-0/5.html

    • quattroginger's avatar
      quattroginger
      Icon for Nimbostratus rankNimbostratus

      this is a new configuration, so its never been in production functioning without issues. a few times he reported it worked within few minutes of reboot but then had error again. thanks for links. those are the ones I've been referencing. the resets i was seeing is what made me second guess the f5 config. waiting on capture now to see traffic from server side.

  • Here is some info to explain my current setup and what im expierencing

     

    vlan 100 - 192.168.3.0/24

    GW - 192.138.3.1

     

    192.168.3.140/24 port 3241 (V)IP

    192.168.3.5/24 f5 self ip

    192.168.3.200/24 backend server

    192.168.3.140/32 backend server loopback

     

    client (printer) 192.168.1.25

     

    i am seeing handshakes between the printer and f5, as well as the printer and backend server. i am seeing resets from the f5 to the printer.

    The printer requires login to access the application im configuring for npath/dsr. normally, it fails with either it cannot reach server, or 

    the server terminated the connection during the first attempt, followed by a second login which then normally grants access to the application.

     

    This is part of a dump on the f5

     

    07:05:20.145616 IP 192.168.1.25.39974 > 192.168.3.140.3241: S 526145886:526145886(0) win 5840 <mss 1460,sackOK,timestamp 561049 0,nop,wscale 4>

    07:05:20.145652 IP 192.168.3.140.3241 > 192.168.1.25.39974: S 2706486537:2706486537(0) ack 526145887 win 4380 <mss 1460>

    07:05:20.146429 IP 192.168.1.25.39974 > 192.168.3.140.3241: . ack 1 win 5840

    07:05:20.146446 IP 192.168.3.140.3241 > 192.168.1.25.39974: R 1:1(0) ack 1 win 0

    07:05:20.147014 IP 192.168.1.25.39974 > 192.168.3.140.3241: P 1:1021(1020) ack 1 win 5840

    07:05:20.147025 IP 192.168.3.140.3241 > 192.168.1.25.39974: R 1:1(0) ack 1021 win 0

    07:05:20.148475 IP 192.168.1.25.39975 > 192.168.3.140.3241: . ack 2715765297 win 5840

    07:05:20.148507 IP 192.168.3.140.3241 > 192.168.1.25.39975: R 1:1(0) ack 0 win 0

    07:05:20.148957 IP 192.168.1.25.39975 > 192.168.3.140.3241: P 0:1020(1020) ack 1 win 5840

    07:05:20.148970 IP 192.168.3.140.3241 > 192.168.1.25.39975: R 1:1(0) ack 1020 win 0

    07:05:24.501013 IP 192.168.1.25.39977 > 192.168.3.140.3241: . ack 2779584638 win 5840

    07:05:24.501024 IP 192.168.3.140.3241 > 192.168.1.25.39977: R 1:1(0) ack 0 win 0

    07:05:24.501037 IP 192.168.1.25.39977 > 192.168.3.140.3241: P 0:1020(1020) ack 1 win 5840

    07:05:24.501041 IP 192.168.3.140.3241 > 1921:04 PM 1/7/2020.168.1.25.39977: R 1:1(0) ack 1020 win 0

     

    i also did another dump on the f5 and exported it so i could view within wireshark. Both the SYN/ACK and RST/ACK i confirmed was the f5 mac address for its source. i thought it might of somehow been the server traffic traveling back through the f5.

     

    the capture on the server looks correct. i see the SYN, source IP of client, source mac address of f5, dest IP of loopback w/ mac address of server. then the SYN/ACK source IP of loopback, source mac of server, dest IP of client, mac address of gateway.

     

     

    i cant figure out why i see the handshake at all between the client and f5 or why the resets are being sent from the f5 to the client. it does not appear to be coming from the server.

     

     

     

     

     

     

  • VIP Config

    Type: Performance Layer 4

    Source: 0.0.0.0/0

    Destination: 192.168.3.140

    Service Port: 0 (All)

    Notify Status to Virtual Address: Checked

    State: Enabled

    Protocol: TCP

    Protocol Profile: custom see below

    HTTP Profile: None

    VLAN and tunnel traffic: All

    Source Address Tanslation: None

    iSession Profile: None

    Rate Class: None

     

    Protocol Profile / FastL4

     

    Reset on Timeout: Enabled

    Idle Timeout: 5 seconds

    TCP Handshake Timeout: 5 seconds

    Maximum Segment Size Override: Disabled

    PVA Acceleration: Full

    Offload State: SYN

    PVA Flow Aging: Enabled

    PVA Flow Evict: Enabled

    PVA Offload Dynamic: Enabled

    PVA Dynamic Client Packets: 2 Packets

    PVA Dynamic Server Packets: 2 Packets

    IP ToS to Client: Pass Through

    IP ToS to Server: Pass Through

    Link QoS to Client: Pass Through

    Link QoS to Server: Pass Through

    TCP Timestamp Mode: Preserve

    TCP Window Scale Mode: Preserve

    Loose Initiation: Enabled

    Loose Close: Enabled

    TCP Close Timeout: 60 seconds

    TCP Keep Alive Interval: Disabled

    Hardware SYN Cookie Protection: Enabled

  • windows server 2016

    i have auto metric set on loopback interface

    removed dns register from loopback interface

    also issued below commands weak send/receive

     

    netsh interface ipv4 set interface "net" weakhostreceive=enabled

    netsh interface ipv4 set interface "loopback" weakhostreceive=enabled

    netsh interface ipv4 set interface "loopback" weakhostsend=enabled