OVHCloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#9274 — vac2-10-tms
Incident Report for Network & Infrastructure
Resolved
We had a problem on the Arbor
TMS 4000 box of VAC2 last week.
One of the mitigation cards
is dead and we are waiting for
the replacement:
http://status.ovh.co.uk/?do=details&id=5335

The 2 remaining cards are visibly not in the
best condition. We have just shut down VAC2.

TMS Fault
Appliance: vac2-10-tms
Subhost 'apm-0-0' is 'Inactive'
We are looking into the problem with Arbor but
card 0 seems dead as well as the 2nd...

Update(s):

Date: 2013-09-10 22:27:48 UTC
We have reset the box. It is stable.



Date: 2013-09-10 15:11:34 UTC
The chassis seems stable.

We will add the 3rd card. We are shutting down VAC2.


Date: 2013-09-10 08:16:08 UTC
The tracking of card n° written on the chassis and
in the system is not the same. When the card in
slot 3 (CLI) must be changed, the card is slot 2
(phsyical) must be withdrawn.
Totally illogical. We will put labels on the
chassis so that tracking is the same between the
CLI and the physical.

==================

The mapping of Slot Number (physical labeling) and that shown by CLI commands for TMS-4000 series is as below:

Slot Number Physical Labeling
Shelf Manager
0 MCM
1 PSM
2 APM
3 APM
4 APM
5 APM

Slot Number CLI (/ system hardware)
0 Shelf Manager
1 MCM
2 PSM
3 APM
4 APM
5 APM
6 APM

Date: 2013-09-10 08:10:31 UTC
We have found a bug in the in the naming of cards
between the chassis, the Arbor documentation and the
SHELL commands output.

We've just replaced the cards again.

We have restored the traffic.

Date: 2013-09-10 08:03:15 UTC
TMS Fault
Appliance: vac2-10-tms
Subhost 'apm-0-1' is 'Inactive'

The new APM card crashed. It will be the chassis ..

Date: 2013-09-09 13:42:30 UTC
We rebooted the chassis with an APM card
that we received as an RMA.

Date: 2013-09-09 08:30:27 UTC
The APM-E blade is failing to boot correctly.
We will request an RMA of the APM-E blade.

Date: 2013-09-09 08:29:50 UTC
The box crashed again.

TMS Fault
Appliance: vac2-10-tms
Subhost 'apm-0-1' is 'Inactive'

Date: 2013-09-09 08:28:31 UTC
We have put VAC2 back in production.

Date: 2013-09-09 08:27:47 UTC
We are seeing how to reinstall the profiles on the TMS
that rebooted without doing a STOP START on them.


Date: 2013-09-09 08:26:33 UTC
apm-0-0- has refused to recover with gentler measures.
..
The latest attempt to restart apm-0-0 appears to have succeeded.
..
apm-0-0 crashed again after restart.
..
Looks like the appliance reboot was able to recover this TMS 4000.
All appears to be functioning correctly at this time.
Posted Sep 09, 2013 - 08:24 UTC