Question : High processor utilisation on Cisco 3750 stack

Hi Experts,

I have an issue that I could do with some help on. I am working on a stack of 2 3750 switches, connected via their Stackwise interfaces. They in turn connect to 5 2950 switches via dual gigabit Ethernet links, configured as port channels. There are a number of VLANs which are configured such that half of the ports on the 2950s are in each VLAN, and the servers are configured on another VLAN to which the gigabit ports of the 3750s belong. Inter VLAN routing is configured to get the traffic between the VLANs.

I am experiencing high processor utilisation on the 3750s, mostly against the IP Input process. I have done some research and it seems that the switching algorithm may not be the most efficient. Attached is some show ip interface output to show how each VLAN is configured, and some output showing the process usage. I would appreciate any comments on improving the routing performance between VLANs - I am making the sweeping assumption in all this that the 3750s are capable of routing this amount of traffic without breaking a sweat when configured correctly.

CORESW#sh ip int
Vlan1 is up, line protocol is up
  Internet protocol processing disabled
Vlan2 is up, line protocol is up
  Internet address is x.x.x.x/24
  Broadcast address is 255.255.255.255
  Address determined by setup command
  MTU is 1500 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Multicast reserved groups joined: 224.0.0.10
  Outgoing access list is not set
  Inbound  access list is not set
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is enabled
  ICMP redirects are always sent
  ICMP unreachables are always sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP CEF switching is enabled
  IP CEF switching turbo vector
  IP multicast fast switching is disabled
  IP multicast distributed fast switching is disabled
  IP route-cache flags are Fast, CEF
  Router Discovery is disabled
  IP output packet accounting is disabled
  IP access violation accounting is disabled
  TCP/IP header compression is disabled
  RTP/IP header compression is disabled
  Probe proxy name replies are disabled
  Policy routing is disabled
  Network address translation is enabled, interface in domain outside
  WCCP Redirect outbound is disabled
  WCCP Redirect inbound is disabled
  WCCP Redirect exclude is disabled
  BGP Policy Mapping is disabled
  Input features: NAT Outside
  Output features: Post-routing NAT Outside
Vlan3 is up, line protocol is up
  Internet address is y.y.y.y/24
  Broadcast address is 255.255.255.255
  Address determined by setup command
  MTU is 1500 bytes
  Helper address is x.x.x.1
  Directed broadcast forwarding is disabled
  Multicast reserved groups joined: 224.0.0.10
  Outgoing access list is not set
  Inbound  access list is not set
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is enabled
  ICMP redirects are always sent
  ICMP unreachables are always sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP CEF switching is enabled
  IP CEF switching turbo vector
  IP multicast fast switching is disabled
  IP multicast distributed fast switching is disabled
  IP route-cache flags are Fast, CEF
  Router Discovery is disabled
  IP output packet accounting is disabled
  IP access violation accounting is disabled
  TCP/IP header compression is disabled
  RTP/IP header compression is disabled
  Probe proxy name replies are disabled
  Policy routing is disabled
  Network address translation is enabled, interface in domain outside
  WCCP Redirect outbound is disabled
  WCCP Redirect inbound is disabled
  WCCP Redirect exclude is disabled
  BGP Policy Mapping is disabled
  Input features: NAT Outside
  Output features: Post-routing NAT Outside

Process output:

PID      Runtime(ms)      Invoked      uSecs      5Sec      1Min      5Min      TTY      Process
-----------------------------------------------------------------------------------------------------
159      805309277      1180331872      682      44.65%      55.05%      49.99%      0      IP Input


Thanks in advance.

George

Answer : High processor utilisation on Cisco 3750 stack

Forgot to get back with a result on this one.

The process decided by ourselves and Cisco support was to restart the switches first, then try a bunch of hardware diagnostics and looking at the CPU buffers etc. to see whether switched packets were still being sent via the CPU.

First step, restart the switches. Lo and behold, problem ceased! This was a month ago adn the problem hasn't reoccured. Teh cisco tech found this surprising and suspected a hw fault, but in the absence of a recurrence, it may just have to be accepted as one of those inexplicable faults. Frustrating, but seems to be the case.