1. Computer problem? Tech Support Guy is completely free -- paid for by advertisers and donations. Click here to join today! If you're new to Tech Support Guy, we highly recommend that you visit our Guide for New Members.

Cisco Router question concerning NAT tcp-timeout

Discussion in 'Networking' started by Mithrilhall, Sep 7, 2005.

Thread Status:
Not open for further replies.
Advertisement
  1. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    I have a router that I'm monitoring/managing and the tcp-timeout was:

    ip nat translation tcp-timeout 460


    I had a complaint from a user that was being disconnected last night about every 7-8 minutes. This happed when they were playing WoW or using AIM. So I adjusted the timeout to:

    ip nat translation tcp-timeout 2400

    The CPU/Memory utilization on the router is still low and I'm keeping and eye on it.

    I thought the NAT tcp-timeout setting only affected translations after a period of inactivity. I wouldn't think playing WoW would equate to an inactive connection and I'm pretty sure AIM sends a keep alive packet every so often.
     
  2. StumpedTechy

    StumpedTechy

    Joined:
    Jul 7, 2004
    Messages:
    7,232
    No its not the period of inactivity that it resets -

    It is in fact - to change the amount of time after which Network Address Translation (NAT) translations time out.

    So if the translation is taking a long time you may in fact have to bump up the amount of time that it is set to retry. The question now becomes why is it taking so long to do the translation if the CPU usage is so low?
     
  3. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    Someone recommended that I remove the line from the config but I'm wondering why the translation is timing out.

    AIM (AOL Instant Messenger) sends out a keep alive packet from what I've read. So I don't know what's is causing it.
     
  4. O111111O

    O111111O

    Joined:
    Aug 26, 2005
    Messages:
    894
    What version of IOS are you running?

    Do sh ver, give me major/minor/featureset.
     
  5. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    cisco 2691 (R7000) processor (revision 0.1) with 105472K/25600K bytes of memory.
    Processor board ID JMX0823L4TW
    R7000 CPU at 160MHz, Implementation 39, Rev 3.3, 256KB L2 Cache
    Bridging software.
    X.25 software, Version 3.0.0.
    2 FastEthernet/IEEE 802.3 interface(s)
    3 Serial network interface(s)
    DRAM configuration is 64 bits wide with parity disabled.
    55K bytes of non-volatile configuration memory.
    31168K bytes of ATA System CompactFlash (Read/Write)
     
  6. O111111O

    O111111O

    Joined:
    Aug 26, 2005
    Messages:
    894
    That's well and good, it shows the hardware. Like I said, what feature set/major/minor rev? (The very first part of the sh ver that you cut out)

    Example is below. This is a 3745 running IPBASE feature set 12.3(7)T


    Cisco IOS Software, 3700 Software (C3745-IPBASE-M), Version 12.3(7)T, RELEASE SOFTWARE (fc1)
    Technical Support: http://www.cisco.com/techsupport
    Copyright (c) 1986-2004 by Cisco Systems, Inc.
    Compiled Sat 21-Feb-04 05:53 by eaarmas

    ROM: System Bootstrap, Version 12.2(8r)T2, RELEASE SOFTWARE (fc1)

    routerblah uptime is 44 weeks, 6 days, 19 hours, 22 minutes
    System returned to ROM by power-on
    System restarted at 15:32:54 EDT Thu Apr 14 2005
    System image file is "flash:c3745-ipbase-mz.123-7.T.bin"
     
  7. O111111O

    O111111O

    Joined:
    Aug 26, 2005
    Messages:
    894
    By the way, the reason I'm asking is TCP FIN bug. Sum rev's of IOS would clear xlates immediately on TCP FIN.

    Normally in IOS, TCP translations time out after 24 hours by default, unless a TCP Reset (RST) or TCP Finish (FIN) is seen in the TCP stream, in which case the translation times out after 1 minute.

    Also, how many hosts do you have connected behind this router? I don't understand why your TCP timeout is so low. The default is typically fine unless you're running IP Inspect/have alot of hosts/are worried about memory. If you're worried about virii breakout running the router out of memory you could consider upgrading to 12.3T and using NAT rate limiting.

    There's also a distinct possibility that you ARE simply seeing TCP FIN from destination host (HTTP does this), and your router is simply clearing the xlate like it should.

    Try:

    "ip nat translation finrst-timeout 300" (Bump up fin/rst timeout to 5 minutes)
     
  8. Rockn

    Rockn

    Joined:
    Jul 29, 2001
    Messages:
    21,189
    And you are allowing them to play these games on the network? It would be a better use of your time to tell them to do actual work and stop complaining about their games not working. It might be worth looking into now however if there are future legit apps that might not work.
     
  9. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    Here is the show ver:


    Code:
    Cisco Internetwork Operating System Software 
    IOS (tm) 2600 Software (C2691-IPBASE-M), Version 12.3(5c), RELEASE SOFTWARE (fc2)
    Copyright (c) 1986-2004 by cisco Systems, Inc.
    Compiled Tue 13-Apr-04 23:41 by cmong
    Image text-base: 0x60008AF4, data-base: 0x60FFA000
    
    ROM: System Bootstrap, Version 12.2(8r)T2, RELEASE SOFTWARE (fc1)
    
    Router uptime is 12 weeks, 6 days, 4 hours, 41 minutes
    System returned to ROM by reload at 09:12:01 EDT Fri Jun 10 2005
    System restarted at 09:13:00 EDT Fri Jun 10 2005
    System image file is "flash:c2691-ipbase-mz.123-5c.bin"
    
    cisco 2691 (R7000) processor (revision 0.1) with 105472K/25600K bytes of memory.
    Processor board ID JMX0823L4TW
    R7000 CPU at 160MHz, Implementation 39, Rev 3.3, 256KB L2 Cache
    Bridging software.
    X.25 software, Version 3.0.0.
    2 FastEthernet/IEEE 802.3 interface(s)
    3 Serial network interface(s)
    DRAM configuration is 64 bits wide with parity disabled.
    55K bytes of non-volatile configuration memory.
    31168K bytes of ATA System CompactFlash (Read/Write)

    BTW...this is for an apartment building...not work; so they can play games as they see fit.

    Also, I would expect no more than 300 concurrent users.
     
  10. O111111O

    O111111O

    Joined:
    Aug 26, 2005
    Messages:
    894
    Well, to start with 12.3(5)c is very deferred. Upgrade to a minimum of 12.3(5)e. There's a CSC bug ID for 12.3(5)c that pertains to turning of ip nat tran timeout, which is why your router may have this.

    Also, if you have anybody trying to use Vonage, that'll blow it up. Many many SIP issues with that release. (SIP fixup)


    I would suggest the following, considering that you will have a fair user load. (What's your WAN? DS1, a couple of DS1's? Are you running IMA or MLPPP?)

    1] Increase NAT TCP timeout to a day.**
    ** I would suggest upgrading to ED 12.3T release and using NAT rate limiting. This will help against a single host "DOS" if they're busy.

    2] Do "sho ip nat tran detail" Will give you state table with source/destination NAT. See what's connected when this happens.

    3] Create "Internet friendly" input ACL on your Ethernet interface facing your users. (Block RPC and other MS-BS from touching your WAN port outbound. Will help with virii outbreaks.)

    4] Low latency/class based weighted fair queueing. Prioritize traffic based on port/protocol. NBAR can help with this. (I.E. Make sure there's always enough WAN pipe for jitter/latency sensitive traffic.)

    5] Depending on your WAN load, there's also a possibility that TCP congestion control isn't working through NAT. You can turn this on in global as ip nat option.

    -----

    Example "Internet friendly" ACL, this will knock down alot of Microsoft poomp.

    ip access-list extended eth-input-priv
    deny tcp any any range 135 139
    deny udp any any range 135 139
    deny tcp any any eq 445
    deny udp any any eq 445
    permit ip any any
    !
    interface fastethernet 0/0
    access-group eth-input-priv in
     
  11. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    Currently running 3 T1s.

    This is what I have for an ACL:

    Code:
    access-list 110 remark Block Dangerous Traffic from Internet
    access-list 110 deny   tcp any any eq 135
    access-list 110 deny   tcp any any eq 139
    access-list 110 deny   tcp any any eq 445
    access-list 110 deny   tcp any any eq 593
    access-list 110 deny   tcp any any range 6881 6889
    access-list 110 deny   tcp any any range 6346 6350
    access-list 110 deny   tcp any any range 16881 16882
    access-list 110 deny   tcp any any eq 1214
    access-list 110 deny   udp any any eq 135
    access-list 110 deny   udp any any eq netbios-ns
    access-list 110 deny   udp any any eq netbios-dgm
    access-list 110 deny   udp any any eq 445
    access-list 110 deny   udp any any range 6881 6889
    access-list 110 deny   udp any any range 16881 16882
    access-list 110 permit tcp any any
    access-list 110 permit udp any any
    access-list 110 deny   53 any any
    access-list 110 deny   55 any any
    access-list 110 deny   77 any any
    access-list 110 deny   pim any any
    access-list 110 permit ip any any
    
     
  12. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    Also, do you have any info or link on the CSC bug ID for 12.3(5)c as well as the SIP fixup pertaining to Vonage? We do have some Vonage users on the network that have complained about the service.
     
  13. O111111O

    O111111O

    Joined:
    Aug 26, 2005
    Messages:
    894
    So are your 3 T1's running in IMA or ML PPP? Or do you just have 3 /30's? With that many users jitter sensitive apps probably suck. If you're not currently, you'll want to bond those DS1's and use CBWFQ with RSVP/WRED on der Ethernet.

    --------

    Your ACL is input "from Internet". What I'm trying to help you with is NAT state table filling up because of outbound poop. Inbound ACL is ok, but you really need to think about "outbound". This protects the router (and you.)

    ----------

    I'm curious, do you have a large CIDR for NAT pool that you're doing overload? SIP has issues accross PAT, so stateful inspection/fixup is usually needed to make that work.

    Some that are applicable. Subsets of code are deferred for this very reason. (Supersceded by new ED rev.)

    CSCdm05667 - Voice qual.
    CSCsa47225 - MLPPP/PPP negotiation loop. (If you're running MLPPP.)
    CSCed57814 - SIP/NAT setup. (You doing SIP inspect?)
    CSCeh47763 - NAT causes router to generate ACKs for RST's
    CSCef13633 - NAT causes malformed SIP contact header.

    There are a bunch. Again, why the code is deferred. This happens a fair amount, ED code (especially "T" train) is "bleeding edge" stuff. Especially VOIP support.

    To put it in perspective, allot of the core stuff I deal with runs "S" code, (Service Provider). 12.0(24)S is what I'm running in many places. As you can see, XM and other releases are up to 12.4 to support the myriad of things that customers want. Due to this, and the pile of different CPU/ASIC you see bugs.

    In any case, before you beat your head against the wall - get out of deferred code.
     
  14. Mithrilhall

    Mithrilhall Thread Starter

    Joined:
    Mar 28, 2001
    Messages:
    781
    This is what I have so far, and my Ts are bonded.

    Code:
    interface Serial0/0
       57 :  ip address xxx.xxx.xxx.xxx 255.255.255.252
       58 :  ip access-group 110 in
       59 :  ip nat outside
       60 :  ip load-sharing per-packet
       61 :  encapsulation ppp
       62 :  no ip route-cache
       63 :  no fair-queue
       64 :  service-module t1 timeslots 1-24
       65 :  service-module t1 remote-alarm-enable
       66 : !
       67 : interface FastEthernet0/1
       68 :  no ip address
       69 :  duplex auto
       70 :  speed auto
       71 : !
       72 : interface Serial0/1
       73 :  ip address xxx.xxx.xxx.xxx 255.255.255.252
       74 :  ip access-group 110 in
       75 :  ip nat outside
       76 :  ip load-sharing per-packet
       77 :  encapsulation ppp
       78 :  no ip route-cache
       79 :  no fair-queue
       80 :  service-module t1 timeslots 1-24
       81 :  service-module t1 remote-alarm-enable
       82 : !
       83 : interface Serial0/2
       84 :  ip address xxx.xxx.xxx.xxx 255.255.255.252
       85 :  ip access-group 110 in
       86 :  ip nat outside
       87 :  ip load-sharing per-packet
       88 :  encapsulation ppp
       89 :  no ip route-cache
       90 :  no fair-queue
       91 :  service-module t1 timeslots 1-24
       92 :  service-module t1 remote-alarm-enable
       93 : !
    
     153 : access-list 1 permit xxx.xxx.xxx.xxx 0.0.1.255
      154 : access-list 88 permit xxx.xxx.xxx.xxx
      155 : access-list 88 permit xxx.xxx.xxx.xxx
      156 : access-list 90 permit xxx.xxx.xxx.xxx 0.0.0.255
      157 : access-list 99 permit xxx.xxx.xxx.xxx
      158 : access-list 99 permit xxx.xxx.xxx.xxx
      159 : access-list 99 permit xxx.xxx.xxx.xxx
      160 : access-list 99 permit xxx.xxx.xxx.xxx
      161 : access-list 99 permit xxx.xxx.xxx.xxx
      162 : access-list 99 permit xxx.xxx.xxx.xxx 0.0.0.255
      163 : access-list 99 deny   any log
      164 : access-list 110 remark Block Dangerous Traffic from Internet
      165 : access-list 110 deny   tcp any any eq 135
      166 : access-list 110 deny   tcp any any eq 139
      167 : access-list 110 deny   tcp any any eq 445
      168 : access-list 110 deny   tcp any any eq 593
      169 : access-list 110 deny   tcp any any range 6881 6889
      170 : access-list 110 deny   tcp any any range 6346 6350
      171 : access-list 110 deny   tcp any any range 16881 16882
      172 : access-list 110 deny   tcp any any eq 1214
      173 : access-list 110 deny   udp any any eq 135
      174 : access-list 110 deny   udp any any eq netbios-ns
      175 : access-list 110 deny   udp any any eq netbios-dgm
      176 : access-list 110 deny   udp any any eq 445
      177 : access-list 110 deny   udp any any range 6881 6889
      178 : access-list 110 deny   udp any any range 16881 16882
      179 : access-list 110 permit tcp any any
      180 : access-list 110 permit udp any any
      181 : access-list 110 deny   53 any any
      182 : access-list 110 deny   55 any any
      183 : access-list 110 deny   77 any any
      184 : access-list 110 deny   pim any any
      185 : access-list 110 deny   ip xxx.xxx.xxx.xxx 0.0.1.255 any
      186 : access-list 110 permit ip any any
      187 : access-list 111 remark Block Dangerous Traffic from TenantNet
      188 : access-list 111 deny   tcp any any eq 135
      189 : access-list 111 deny   tcp any any eq 139
      190 : access-list 111 deny   tcp any any eq 445
      191 : access-list 111 deny   tcp any any eq 593
      192 : access-list 111 deny   udp any any eq 135
      193 : access-list 111 deny   udp any any eq netbios-ns
      194 : access-list 111 deny   udp any any eq netbios-dgm
      195 : access-list 111 deny   udp any any eq 445
      196 : access-list 111 permit tcp any any
      197 : access-list 111 permit udp any any
      198 : access-list 111 deny   53 any any
      199 : access-list 111 deny   55 any any
      200 : access-list 111 deny   77 any any
      201 : access-list 111 deny   pim any any
      202 : access-list 111 permit ip any any
      203 : !
    
     
  15. O111111O

    O111111O

    Joined:
    Aug 26, 2005
    Messages:
    894
    Hi,

    Your DS1's aren't bonded. You're using per-packet cef load sharing. Very much not to be confused with inverse multiplexing.

    What you have are NxDS1 interfaces, CEF table on your router will forward outbound packets per-packet with the CEF adjacency table.

    To put it bluntly, per-packet load sharing is horrible. This is crap that Cisco puts as requirments in IE labs to try and make things like MPLS TE labels map accross it. (It's a *****) With per-packet load sharing it fires each packet out blindly and forces the end host to re-assemble the packet. In many instances packets arrive at the destination out of order and you'll see huge variances in packet delivery time (Jitter). Also, with out of order packets the host may simply drop them. (I.E. your AIM issue.)

    On top of all that, is your ISP also using per-packet load sharing? Probably not. Default is per-destination. What this means is that your loadsharing is simplex. Outbound from you may "balanced" across three DS1's, but inbound from your ISP is destination balanced. The end result is that something like an FTP will max out a single DS1.

    It's probably reason #1 that any users passing SIP/H.323 traffic are complaining.

    If your ISP is flexible (hopefully they are) I would suggest you have them move you to something that's truly inverse multiplexing. In your case; Multilink PPP.

    With any inverse-multiplexing on Cisco devices, you'll have a virtual interface (in this case Multilink 1) with an IP address, and your physical interfaces will have no IP but will be inserted into a channel-group.

    The router essentially has a heuristic queuing engine for each packet. In essence it truly round-robins each packet across "n" given interfaces in the multilink group. In your case, since you have 3 DS1's you will receive true 4.5Mbps bi-directional throughput.

    In summary;

    NAT w/ your code is an issue. I would suggest using default timeouts.
    Access-list 110 is ok, but it's not stateful. You'll drop certain connections (whole conversation there)
    per-packet load sharing with CEF is bad! Make it go away! Your users will be so much happier! :)

    I would tackle the multiplexing of your DS1's first.

    PPP multilink example below: (Your ISP HAS TO DO THIS AS WELL.)

    interface multilink1
    ip address xxx.xxx.xxx.xxx 255.255.255.252
    ppp encapsulation
    ppp multilink
    multilink-group 1
    ip access-group 110 in
    multilink min-links 3 mandatory **not really needed, but I like it.
    ip nat outside

    interface S0/n (where "n" is the interface number.)
    no ip address
    encapsulation ppp
    no fair-queue
    no cdp enable
    ppp multilink
    multilink-group 1
     
  16. Sponsor

As Seen On
As Seen On...

Welcome to Tech Support Guy!

Are you looking for the solution to your computer problem? Join our site today to ask your question. This site is completely free -- paid for by advertisers and donations.

If you're not already familiar with forums, watch our Welcome Guide to get started.

Join over 733,556 other people just like you!

Loading...
Thread Status:
Not open for further replies.

Short URL to this thread: https://techguy.org/397126