ServiceNow Discovery Design

Introduction | ServiceNow Discovery | ServiceNow Discovery :: Schedules

Schedules | Schedules :: MID Server Clusters | MID Server Clusters

MID Server Clusters :: MID Servers | MID Servers

MID Server Hosts | MID Server Hosts :: Credentials | Credentials

Introduction

This article provides guidelines for the design of ServiceNow Discovery.

While more complicated implementations deviate from the recommendations below, this provides a solid foundation to underpin every ServiceNow Discovery Design to simplify its ongoing maintenance.

These recommendations are ONLY for IP Based Discovery

i.e. Discover = Configuration Item

ServiceNow Discovery

The elements of a ServiceNow Discovery Design include Schedules, which reference MID Server Clusters, which contain one or more MID Servers, which run on MID Server Hosts.

Independently, correctly configured Credentials on the target host and in ServiceNow, along with numerous additional nuanced configurations ensures ServiceNow Discovery is set up successfully.

ServiceNow Discovery :: Schedules

Schedules control when ServiceNow Discovery is triggered and which parts of the network get discovered at what time.

Since the network and the devices on the network constantly change, these Schedules are configured to run on a recurring basis.

When a Discovery Schedule runs, it generates a corresponding Discovery Status, which provides information on what was successfully discovered and more importantly, what didn't get discovered.

The Discovery Admin App operationalizes the maintenance of ServiceNow Discovery by analyzing the logs generated by ServiceNow Discovery and enforcing best practice recommendations for the successful operationalization of the implemented ServiceNow Discovery Design.

Schedules

The steps below, provide an iterative checklist of the items for designing/creating Schedules.

Collect IP Subnets including their Description and their Location
Group the IP Subnets by Location. THIS is the granularity of your Discovery Schedules. As an added benefit, we can, out-of-the-box, populate the value of the ‘Location’ attribute on the Discovery Schedule Form into the ‘Location’ attribute of the CI, discovered via this Schedule!
If Location is not available, Group the IP Subnets by Class B.
Avoid grouping Schedules by Protocol or in any other random order.
Do not have the same subnets be a part of multiple schedules.
Daisy Chain the schedules (i.e. have them run one-after-another)
(HINT: You can always use the ‘Discover Now’ UI Action to run the Schedule without triggering the Daisy Chain)
Align the first schedule in the Daisy Chain to trigger at the beginning of the approved run time. Typically design this so that the complete Subset is scanned once a week.
Avoid Periodic Schedules or Bi-weekly Schedules. Weekly cadence is the most preferred option.
Set the ‘Max run time’ to be 1 hour more than the average time taken for that Discovery Schedule to run.
Select the ‘Run if Cancelled’ checkbox, so an unforeseen error doesn’t disrupt the Daisy Chain.

With the steps captured above, any subsequent IP Subnets received easily go into the correct Discovery Schedule driven by Location or Class B.

This approach extends well beyond the initial configuration, allowing for simple and intuitive addition and removal of Subnets.

Schedules :: MID Server Clusters

There are several ways to associate a MID Server with a Discovery Schedule.

Auto-Select a MID Server
Specify a single MID Server
Configure Behaviors
MID Server Clusters

While it may be intriguing to try a different set of these combinations to discover ‘Configuration Items’, the most preferred option is MID Server Clusters.

With this approach, we see the following benefits:

No errors during migration, since the Schedules are referencing a MID Server Cluster Record, vs a MID Server Record. And by definition, each MID Server has a different sys_id in different environments.
Easy to scale your horsepower by adding MID Servers to the Cluster, without making any changes to Discovery Schedules.
Prevents the use of Behaviors, Functionalities, and Capabilities, which complicates the ServiceNow Discovery Design.

MID Server Clusters

It may seem more intuitive to use MID Server Clusters only when you have more than one MID Server. However, it is recommended to configure MID Server Clusters, even when you currently have only one MID Server.

In short, ALWAYS use MID Server Clusters.

Cluster the non-Prod MID Servers in the same way as the Prod MID Servers. This is for Design Validation as well as Performance Validation of Discovery in non-Prod, so you know exactly what to expect in Prod, after migration.

Design Recommendation:

One MID Server Cluster per Country / Geography / Data Center / Location

The location of the MID Servers should be strategically selected to maximize the network visibility for the MID Server.

Consider moving the MID Servers closer to their targets to minimize latency and accommodate for scheduling aligned with local time zones.

It also minimizes firewall exceptions and proves to be extremely agile for large, complex, and ever-changing landscapes.

The simplest interpretation of the above design is to have only one MID Server Cluster hosted at an optimal location on the Network.

If applicable, add additional MID Server Clusters for segregated network segments like the DMZ to accommodate any exceptions.

MID Server Clusters :: MID Servers

The number of MID Servers in a MID Server Cluster should be determined based on:

the number of devices being scanned by the MID Server Cluster
the time available to scan the devices
the budgets aligned with the number of Servers / Virtual Machines available for ServiceNow Discovery

Leverage ServiceNow's MID Server Calculator here: Excel Download

Guidelines for MID Servers in the same MID Server Cluster:

They should be configured on similar MID Server Hosts (Ensure the MID Server Hosts in the Cluster have the SAME configurations.)
They should be co-located i.e. be on the same Subnet to prevent issues with Network / Firewall
They should have the same MID Server Configurations / Properties / Parameters

Other Considerations:

Do not assign a MID Server to more than one Cluster.
Always use a Load-Balanced Cluster vs a Fail-Over Cluster, as Load-Balancing does Fail-Over.
Note that MID Server Clusters are not captured in Update Sets and need to be manually migrated (via XML) across the ServiceNow Instances. Once the MID Server Clusters are created (and synced across all the environments), any future migrations for Discovery Schedules are seamless and error-free.
(Reason: Unlike the MID Server, the sys_id of the MID Server Cluster remains the same across your instances)

MID Servers

What is a MID Server (Xanadu): https://www.servicenow.com/docs/bundle/xanadu-servicenow-platform/page/product/mid-server/concept/mid-server-landing.html

Installing a Windows MID Server (Xanadu): https://www.servicenow.com/docs/bundle/xanadu-servicenow-platform/page/product/mid-server/concept/mid-server-landing.html

Important MID Server Properties:

thread.max: Set it to 100
mid.powershell.local_mid_service_credential_fallback: Set it to false
mid.ssh.use_snc: Set it to true
mid.eccq.max_payload_size: Set it to 30,000,000
mid.snmp.request.timeout: Set it to 15,000
mid.snmp.session.timeout: Set it to 15,000

All MID Server Properties (Xanadu): https://www.servicenow.com/docs/bundle/xanadu-servicenow-platform/page/product/mid-server/reference/r_MIDServerProperties.html

All MID Server Parameters (Xanadu): https://www.servicenow.com/docs/bundle/xanadu-servicenow-platform/page/product/mid-server/reference/mid-server-parameters.html

MID Servers :: MID Server Hosts

Design Recommendation: One MID Server per MID Server Host per ServiceNow Instance

This results in predictable availability of the MID Server Host resulting in predictable performance of the MID Server.

The number of MID Server Hosts (for ServiceNow Discovery) should be equal to the number of MID Servers in Production.

NOTE: Non-Prod MID Server(s) should be on the same host as Prod MID Server

While it may be natural to request different hosts for Prod vs non-Prod MID Server(s), the non-intuitive recommendation is to have both the Prod and non-Prod MID Server(s) run on the same host!

This design provides a solid grounding to ensure that discovery validated in non-Prod works the same way in Prod.

Rationalization:

The MID Servers are completely independent of each other from an App footprint and execution standpoint.
Prod and non-Prod Discovery do not run at the same time (non-Prod Discovery is typically made inactive). This eliminates resource contention at the onset. Independently, if there are issues with Prod Discovery, having another MID Server running on the same MID Server Host allows for the best scenario for targeted discovery and Troubleshooting in non-Prod, enabling quicker resolution for Prod issues.
Network Firewall Rules and ACLs on SNMP Devices are important configurations that need to be done outside of ServiceNow for Discovery to work consistently. Since the Prod and non-Prod MID Servers are on the same MID Server Host, when these configurations are tested in non-Prod, we can be sure they will work in Prod. More so, there is no other way to ensure this parity, apart from having the Prod and non-Prod MID Servers running on the same Host.

MID Server Hosts

MID Server Hosts are Servers / Virtual Machines that run the MID Server App.

While running MID Servers on a Linux Host is supported, it is recommended to use Windows Hosts for MID Servers as they can discover all types of Devices on the Network (unlike Linux Hosts which can not discover WMI Devices).

See the requirements for the Windows Hosts Here (Xanadu): https://www.servicenow.com/docs/bundle/xanadu-servicenow-platform/page/product/mid-server/reference/r_MIDServerSystemRequirements.html

MID Server Hosts :: Credentials

There is an option to allow Windows Credentials on the MID Server Host to be used for WMI ServiceNow Discovery.

However, do not use Credentials configured on the MID Server Host for this purpose.

It does not scale well and is hard to maintain.

Always configure credentials for ServiceNow Discovery in the Credentials Table in ServiceNow.

Credentials

Depending on the types of CIs in the scope of ServiceNow Discovery, we need to collect and configure the appropriate credentials.

See the requirements for the Credentials here (Xanadu): https://www.servicenow.com/docs/bundle/xanadu-platform-security/page/product/credentials/task/t_CreateCredential.html