Scheduled Troubleshooting
Tip 01 | Tip 02 | Tip 03 | Tip 04 | Tip 05 | Tip 06 | Tip 07 | Tip 08 | Tip 09
Introduction
Scheduled Troubleshooting in Discovery Admin allows you to automatically run the Troubleshooting on a predetermined dynamic list of Discovery Status records, leveraging the out-of-the-box ServiceNow Scheduling UI.
NOTE:
Make sure you have perused the Scheduled Troubleshooting Design article before following the steps to configure the Scheduled Troubleshooting.
The filters in the Video below are for demonstration only, aligning with our demo data. Please use valid filters for your use case aligned with the recommendations after perusing the Scheduled Troubleshooting Design article.
Include ALL the recommended attributes (referenced in the Scheduled Troubleshooting Design article) to build your Discovery Status List View Filter, as a precursor to creating a new Scheduled Troubleshooting record.
Video: Configure Scheduled Troubleshooting (4 mins :: 13 secs)
Time Stamps:
0:19 - Introduction to the Scheduling of Troubleshooting
With Discovery Admin, we can also schedule the Troubleshooting Process.
0:30 - Navigating to the Previous Scheduled Troubleshooting
Navigate to the Discovery Status module and then scroll down to the bottom of the page. Click the { Scheduled Troubleshooting } UI Action to be redirected to the Scheduled Troubleshooting List View where all the previously configured Scheduled Troubleshooting Records are shown.
0:48 - Creation of a new Scheduled Troubleshooting
Navigate to the Discovery Status Module. Create the filter on the Discovery Status List View. Once the filter is created, scroll down and click on the { Scheduled Troubleshooting } UI Action, which then pops up a new dialog box to name the new Scheduled Troubleshooting Record.
2:25 - Information on the Scheduled Troubleshooting Form
Once we name the new Scheduled Troubleshooting, it redirects us to the new Scheduled Troubleshooting Form. Any attribute on the Troubleshooting Form can be updated. Update the Run and the Repeat Interval, to update the frequency with which the particular Scheduled Troubleshooting Record runs.
3:25 - Tweaking of the Query Filter
For advanced users, tweak the Query Filter by changing the query which updates the URL to the filter.
Tip 01: Run Next, Priority, Incident Generation (v9.3)
Taking inputs from the Scheduled Troubleshooting Design, OPTIONALLY populate the Run Next (Reference Attribute), Priority (Boolean Attribute), or Incident Generation (Boolean Attribute) to align the Scheduled Troubleshooting with your requirements.
NOTE:
The Scheduled Troubleshooting configured in the 'Run Next' of another Scheduled Troubleshooting should be configured to Run 'On Demand'. This prevents accidental triggering of the 'Run Next' Scheduled Troubleshooting outside of the configured Daisy Chain.
Tip 02: Manual Execution of Scheduled Troubleshooting (v9.3)
Scheduled Troubleshooting can be triggered manually using the
{ Execute Now } UI Action.
Initiating a Scheduled Troubleshooting manually via the { Execute Now } UI Action simply executes the Discovery Status Records returned by the Discovery Status List Filter and doesn't take into account the following attributes on the Scheduled Troubleshooting Form:
Active: A Troubleshooting will be created even if this is false.
Run Next: This value will not be passed over to the Troubleshooting Record. To trigger a Daisy Chain, use 'Run Once' instead of the { Execute Now } UI Action.
Priority: This will run with a separate background queue to not interfere with the scheduled ones.
Generate Incidents: This will not Trigger Incident Generation. To generate an incident, manually click the { Generate Incidents } UI Action on the Troubleshooting Form.
Tip 03: Unable to create a new Scheduled Troubleshooting Record
Click "Run" after creating the Filter on the List View: Discovery > Status.
Not only does this provide a visual validation of the Discovery Status Records, but it also provides the URL needed by the { Scheduled Troubleshooting } UI Action to create the Scheduled Troubleshooting Record.
Occasionally, we may go ahead and create the filter on the List View: Discovery > Status, BUT forget to Run it, before we click the { Scheduled Troubleshooting } UI Action.
Tip 04: Scheduled Troubleshooting does not Trigger
If the 'Discovery Status Query Filter' results (on the List View: Discovery > Status) are empty AT THE TIME the Scheduled Troubleshooting is configured to run, no Discovery Status records are passed to Discovery Admin for Troubleshooting.
As a result, no corresponding Troubleshooting Record is created.
We also log this exception as an Info Message in the System Logs.
This log can be found by doing a contains search for the following text 'ERROR: Troubleshooting Record is NOT created because' around the time stamp when the Scheduled Troubleshooting was expected to run.
To fix this, check the following, in the order of the recommendations below:
Typos in the Filter:
Ensure we have no typos corresponding to the text fields in the filter.
This can be easily validated by clicking the 'Discovery Status List Filter' Link on the Scheduled Troubleshooting Form and visually validating the list of Discovery Status Records corresponding to the Discovery Schedules scoped for the Scheduled Troubleshooting.
Case Sensitive Filter:
Ensure we match the case in the filter. This is a common error and often easy to overlook.
Even though the ServiceNow UI is not case-sensitive, the filters that run in the backend are case-sensitive.
This especially applies to scenarios where Schedule Names (or any free text fields) are used in the filters.
ServiceNow Discovery did not run at the expected time:
Ensure ServiceNow Discovery is running as expected.
It may be possible that NONE of the Discovery Schedules corresponding to the configured filter on the Scheduled Troubleshooting Form ran at the original intended time.
As a result, no corresponding Discovery Status records were generated and nothing was passed to Discovery Admin for Troubleshooting.
Invalid Filter due to change in Design of ServiceNow Discovery:
Ensure there are no recent design changes to ServiceNow Discovery.
This scenario warrants a redesign of the Scheduled Troubleshooting to update the filter and/or update when the Scheduled Troubleshooting runs to align with the new ServiceNow Discovery Design.
Tip 05: Scheduled Troubleshooting does not Trigger at the Scheduled Time
We use the out-of-the-box Scheduling UI for configuring the time for Scheduled Troubleshooting.
So, if the Scheduled Troubleshooting is not triggering at the scheduled time, adjust the timezone settings on the Scheduled Troubleshooting Record and align those with what you have configured at the timezone for your User Profile.
Tip 06: Scheduled Troubleshooting has missing Discovery Schedules
There are two scenarios where a Troubleshooting generated by a Scheduled Troubleshooting does not have the Discovery Status Records (corresponding to the Discovery Schedules) included as a part of the analysis.
Discovery Schedule did not Run at the expected time:
ServiceNow Discovery may not be running as expected due to reasons like Administrative Oversight, MID Server issues, Long Running Schedules, etc, resulting in the Discovery Status Records (corresponding to the Discovery Schedules) not being generated at the expected time.
As a result, Discovery Admin doesn't see the Logs corresponding to the missing Discovery Schedules.
To fix this, validate the timing of the historical runs of the missing Discovery Schedules to understand why the Discovery Schedules didn't run at the expected time.
Inaccurate Configuration while creating the Scheduled Troubleshooting:
As a part of configuring a new Scheduled Troubleshooting Record, we get a visual confirmation for which Discovery Status records will be included via the Discovery Status Query Filter on Scheduled Troubleshooting Form.
Additionally, we recommended time-boxing the Scheduled Troubleshooting with the 'Created' attribute, so that we can run the analysis regularly.
Examples of time-boxing:
Created ON Yesterday
Created ON Last week
However, since the Scheduled Troubleshooting is configured to run at a different time (than when we create the Schedule Troubleshooting), there may not be a way to accurately visually validate the selection.
To help fix this, temporarily remove the 'Created' timestamp to better visualize which Discovery Status Records (corresponding to the Discovery Schedules) are being included.
Once the results are validated, re-introduce the 'Created' filter and follow the steps to update the Discovery Status Query Filter on the Scheduled Troubleshooting Form.
Tip 07: Unexpected Scheduled Troubleshooting running
NOTE: admin role is needed to troubleshoot this
If you observe an unexpected Scheduled Troubleshooting being triggered, the best way to validate this is to search for the name of the Scheduled Troubleshooting under the Name column on the sys_trigger Table.
The sys_trigger Table holds records for what is scheduled to be run by ServiceNow. If you see any records in the sys_trigger Table that represent run times that are not valid, you can delete those records and verify that no new unexpected records get created.
Tip 08: Run Next Scheduled Troubleshooting doesn't get triggered (v9.3)
NOTE:
The 'Run Next' information is copied:
FROM the Scheduled Troubleshooting Record
TO the Troubleshooting Record (generated by the Scheduled Troubleshooting)
at the time of the CREATION of the corresponding Troubleshooting Record
So, if 'Run Next' IS NOT populated on the Scheduled Troubleshooting Record, when the corresponding Troubleshooting Record is created, the 'Run Next' field on that Troubleshooting Record will be empty.
As a result, there will not be anything to 'Run Next' after the Troubleshooting Record completes the analysis.
The following conditions should be met for the Scheduled Troubleshooting Daisy Chain works as expected:
The 'Run Next' attribute on the Troubleshooting Form should NOT empty (see NOTE above)
The next Scheduled Troubleshooting only gets triggered when the current Troubleshooting changes its Status to 'Analysis Executed'. Any other Status change including canceling the current Troubleshooting will break the Daisy Chain.
The Scheduled Troubleshooting referenced in 'Run Next' should have Active = true.
If the Scheduled Troubleshooting IS NOT triggered via the { Execute Now } UI Action. Instead, it should be triggered via:
Run = Daily | Weekly | Monthly | Periodically | Once
Tip 09: Update the Discovery Status Query Filter for the Scheduled Troubleshooting
To change the Discovery Status Query Filter:
Open the Scheduled Troubleshooting Record.
Click the 'Discovery Status List Filter' link. This opens the Filtered List of Discovery Status Records in a new Tab.
Update the Filter on the Discovery Status List View and click 'Run' to validate the new Filter.
Once the new Filter gives you the expected result, right-click the bread-crumb and click: 'Copy query'.
Navigate to the Previous Tab where Scheduled Troubleshooting is still open. Replace the 'Discovery Status Query Filter' by selecting it and pasting the current value (copied to your clipboard via the previous step).
Save the Scheduled Troubleshooting Record.
Reverify the new Filter by Clicking the 'Discovery Status List Filter' link.
Next time the Scheduled Troubleshooting runs, it will use the updated Query Filter.