Server administrators often use scom notifications to receive an e-mail for critical alerts. A common problem with this is that the e-mail is sent once and if no action is taken, the alert will stay in the scom console untill there is a system failure. After the failure everyone is going to search for a scapegoat.Why didn’t the scom operators take action? To whom was the e-mail sent? Why didn’t this person take action? Blahblahblah… too late…
With this runbook we can change the ResolutionState of an alert that reaches a certain age and for which no action was taken. This can be a trigger to send out mail to a somebody that’s higher up the ladder. This person can then take action to see who’s not doing his/her job correctly.
System Center Orchestrator with SC 2012 Operations manager Integration Pack
- a valid Microsoft System Center Operations Manager Connection (options > SC 2012 Operations Manager)
- The following runbook activities
– Monitoring Date/Time Activity
– Run .NET Script
– Get Alert
– Update Alert
- powershell cmd
– $AlertDateBefore = (Get-Date).AddDays(-2) | get-date -Format ‘yyyy-MM-ddTHH:mm:ss’
- A new Alert resolution State (i.e. “AlertAge Passed 1 day”)
- An active alert to test the runbook
- The MonitoringRuleId of the active alert if you want to update only certain alerts (not really necessary)
In your Orchestrator Runbook designer add the four runbook activities and connect them.
1) In the Monitor “Date/Time” activity you can set when and how frequent you want to run the runbook.
2) In the “Run .Net Script” activity paste the powershell command
3) In the “Get Alert” activity specify the filters to granularly select which alerts you want to update.
– MonitoringRuleId: you can get this through using the “get-scomalert” cmdlet + some parameters
– TimeRaised: For this one you have to subscribe to the published data from our previous step. Just right click in the blank field 😉
– ResolutionsState: We are only going to update New alerts.
– Severity: We are only going to update Critical alerts.
4) In the “Update Alert” activity, right click the Alert ID field. Choose subscribe > Published Data and select “Id” from the Get Alert activity. Use the “Select field…” to add the Resolution State item. You can select the new resolution state you’ve create in scom (see ingrediënts)
That’s it!! Now test your runbook.