Get webhook notifications whenever Network & Infrastructure creates an incident, updates an incident, resolves an incident or changes a component status.
Following the update of Nexus 5000 switches with the latest version 5.2.1 we encountred a lot of random problems that caused several failures.
We worked with Cisco teams and we found the origin of the problem. In the version 5.2.X Cisco added new tools to the vlan. Thus a vlan can have 2 different types.
When we update to 5.2.1 we should do the upgrade through a special version that fixes the type of vlan.
Otherwise, vlan type is not fixed and it is unknown.
To fix it:we should make a
no vlan XX, XX, XX
then reapply the vlan configuration
vlan XX, XX, XX
Update(s):
Date: 2012-10-16 00:21:03 UTC 64 bytes from 172.16.XX.1: icmp_seq=59. time=0.334 ms
64 bytes from 172.16.XX.1: icmp_seq=60. time=0.433 ms
64 bytes from 172.16.XX.1: icmp_seq=66. time=0.371 ms
64 bytes from 172.16.XX.1: icmp_seq=67. time=0.344 ms
64 bytes from 172.16.XX.1: icmp_seq=68. time=0.484 ms
64 bytes from 172.16.XX.1: icmp_seq=69. time=0.337 ms
64 bytes from 172.16.XX.1: icmp_seq=70. time=0.721 ms
^C
----172.16.XX.1 PING Statistics----
95 packets transmitted, 90 packets received, 5% packet loss
round-trip (ms) min/avg/max/stddev = 0.228/0.6535/6.17/0.866
pcc-19-n5
done.
We didn't notice any impact on the performance of PCC.
The bug of all performing vlans of pcc storage switches has been fixed.
We will conduct some corrections on the maintenance vlans tomorrow during the day.
Date: 2012-10-16 00:20:33 UTC 64 bytes from 172.16.XX.1: icmp_seq=11. time=0.307 ms
64 bytes from 172.16.XX.1: icmp_seq=12. time=0.361 ms
64 bytes from 172.16.XX.1: icmp_seq=18. time=0.299 ms
64 bytes from 172.16.XX.1: icmp_seq=19. time=0.345 ms
64 bytes from 172.16.XX.1: icmp_seq=20. time=0.960 ms
^C
----172.16.XX.1 PING Statistics----
21 packets transmitted, 16 packets received, 23% packet loss
round-trip (ms) min/avg/max/stddev = 0.299/0.4337/0.960/0.182
pcc-118-n5 fait.
The last switch pcc: pcc-119-n5
Date: 2012-10-16 00:20:17 UTC 64 bytes from 172.20.XX.1: icmp_seq=13. time=0.341 ms
64 bytes from 172.20.XX.1: icmp_seq=14. time=0.616 ms
64 bytes from 172.20.XX.1: icmp_seq=31. time=0.312 ms
64 bytes from 172.20.XX.1: icmp_seq=32. time=0.299 ms
^C
----172.20.XX.1 PING Statistics----
33 packets transmitted, 17 packets received, 48% packet loss
round-trip (ms) min/avg/max/stddev = 0.299/0.4072/0.616/0.109
pcc-113-n5 fait.
Next: pcc-118-n5
Date: 2012-10-16 00:19:53 UTC 64 bytes from 172.19.XXX.1: icmp_seq=54. time=0.312 ms
64 bytes from 172.19.XXX.1: icmp_seq=66. time=0.414 ms
64 bytes from 172.19.XXX.1: icmp_seq=67. time=0.304 ms
64 bytes from 172.19.XXX.1: icmp_seq=68. time=0.273 ms
64 bytes from 172.19.XXX.1: icmp_seq=69. time=0.310 ms
64 bytes from 172.19.XXX.1: icmp_seq=70. time=0.337 ms
64 bytes from 172.19.XXX.1: icmp_seq=71. time=0.321 ms
^C
----172.19.XXX.1 PING Statistics----
72 packets transmitted, 61 packets received, 15% packet loss
round-trip (ms) min/avg/max/stddev = 0.273/0.3487/0.879/0.0768
pcc-112-n5 fait.
Next : pcc-113-n5
Date: 2012-10-16 00:19:38 UTC 64 bytes from 172.19.XXX.1: icmp_seq=97. time=0.320 ms
64 bytes from 172.19.XXX.1: icmp_seq=98. time=0.339 ms
64 bytes from 172.19.XXX.1: icmp_seq=115. time=0.321 ms
64 bytes from 172.19.XXX.1: icmp_seq=116. time=0.310 ms
^C
----172.19.XXX.1 PING Statistics----
117 packets transmitted, 101 packets received, 13% packet loss
round-trip (ms) min/avg/max/stddev = 0.278/0.33181/0.538/0.04253
pcc-107-n5 fait.
Next: pcc-112-n5
Date: 2012-10-16 00:19:20 UTC 64 bytes from 172.19.XX.1: icmp_seq=215. time=0.330 ms
64 bytes from 172.19.XX.1: icmp_seq=216. time=0.341 ms
64 bytes from 172.19.XX.1: icmp_seq=217. time=0.320 ms
64 bytes from 172.19.XX.1: icmp_seq=226. time=0.709 ms
64 bytes from 172.19.XX.1: icmp_seq=227. time=0.347 ms
64 bytes from 172.19.XX.1: icmp_seq=228. time=0.335 ms
64 bytes from 172.19.XX.1: icmp_seq=229. time=0.365 ms
^C
----172.19.XX.1 PING Statistics----
230 packets transmitted, 222 packets received, 3% packet loss
round-trip (ms) min/avg/max/stddev = 0.256/0.37774/1.37/0.1168
pcc-105-n5 fait.
Next: pcc-107-n5
Date: 2012-10-16 00:19:00 UTC 64 bytes from 172.18.XX.1: icmp_seq=277. time=0.515 ms
64 bytes from 172.18.XX.1: icmp_seq=278. time=0.416 ms
64 bytes from 172.18.XX.1: icmp_seq=279. time=0.377 ms
64 bytes from 172.18.XX.1: icmp_seq=280. time=0.404 ms
64 bytes from 172.18.XX.1: icmp_seq=292. time=0.360 ms
64 bytes from 172.18.XX.1: icmp_seq=293. time=0.356 ms
64 bytes from 172.18.XX.1: icmp_seq=294. time=0.291 ms
64 bytes from 172.18.XX.1: icmp_seq=295. time=0.326 ms
^C
----172.18.XX.1 PING Statistics----
296 packets transmitted, 285 packets received, 3% packet loss
round-trip (ms) min/avg/max/stddev = 0.272/0.45126/2.74/0.2082
pcc-104-n5 fait.
Next: pcc-105-n5
Date: 2012-10-16 00:18:42 UTC We are starting with pcc-104-n5.
Date: 2012-10-16 00:18:23 UTC It remains the N5 pCC storage, we will do it tonight.
then: rps, backup,3 pairs of N5-15, 27.31 storage to be done tomorrow morning with all the storage team.
Date: 2012-10-15 23:34:42 UTC 178.33.122.248/24 done
Date: 2012-10-15 23:34:25 UTC 188.165.14.0/24 done
Date: 2012-10-15 23:34:12 UTC 188.165.15.0/24 done