NetApp Monitoring

New

• check_netapp_time: Check for a valid ntp-server configuration on the filer and alarm if the drift between the filers system-time and the monitoring-server is getting too high.
• The Usage check gets a new parameter --aggr_over=<percentage> to alarm only if aggregates are both overcommitted and begin to fill up.
• Overall checks like Usage have a new parameter --top=<n>. Setting --top to a number will reduce the number of instances, e.g. volumes shown in the output to that number. This has been requested to reduce the sometimes very long lists of volumes, disks and the like.

Improved

• PerfDisk got an additional performance object disk_constituent. Using disk_constituent (instead of disk) avoids getting Multiple_Values returned instead of valid instance names. If this new object is used for the check the getters object must be changed as well!

New

• AutosizeMode checks if the auto-size mode of all volumes matches a given preset (grow, grow_shrink).
• Head has a new object version. This both shows and monitors the DataONTAP version of the nodes. Plus it prints the box-model onto the first line so that it can be quickly searched in the GUI.
• SisStatus flags volumes, whose compression and deduplication is not enabled.
• Service Acknowledgements can be removed by a custom-script using the new parameter --rm_ack_handler=<handler-script>.
• check_netapp_unused_lun searches and flags LUNs which are mapped but whose initiators are not logged in.
• DiskPathQuality checks disk path qualities, reports i/o-error percentages and raises a CRITICAL error whenever an error percentage is above zero.

Improved

• Completely rewritten example configurations. The etc/*.cfg-files are taken as they are from a running monitoring-system in our lab. So typos or outdated left-overs are almost impossible from now on.

Fixed

• The storefiles written by the getters from the 4.0.x releases were several times bigger than the ones from older versions. We have fixed that now.
• The input-values of --object is validated in check_netapp_snapshots.
• VolumeAutosize failed on MetroClusters because of undefined values. These volumes are skipped now.

Changed

• Remove --node from check_netapp_cluster: The check_netapp_cluster is divided into check_netapp7_cluster for legacy 7-Mode systems and the cdot-logic for checking the cf-status has been incorporated into check_netapp_takeover.
• Checks using system-cli have got an improved documentation.

Fixed

• Updated the cpanfile and the chapter on Required Perl Modules in the installation document.
• The parameter --max_records is evaluated (this speeds up getters which pull a large number of instances.)

Fixed

• The getters --stm has been reenabled (in 4.0.0 it was silently ignored which made UsageTrend failing).

Fixed

• ShelfEnvironment --what=voltage failed (exit 255) in case of a PSU error. Failed PSUs are reported as CRITICAL now.

Changed

• Completely rewrote the cluster-mode getter to make it more efficient and better maintainable. Several objects have now their own dedicated getter (disk, snapshot, ...).
• Introduce the --support switch as a replacement for --mammamia. (This applies to all getters with the exception of 7m and perfdata.)
• Removed --vserver|--node|-s parameter from mosts checks. Now these checks have the node- or vserver-name prefixed to every instance-name. So instead of the old --node=my-node one can filter now with --include=^my-node. This also obsoletes the --svm_in_name switch, we had implemented in various checks. Also this switch has been removed since that behavior is the default now.
• The call-files are grouped into subfolders per host now. This change will not be visible on sites which start with an empty storedir. (Background: This change reduces the number of files per directory to avoid negative implications on file-system performance.) Sites which want to keep their existing (long-term)-stores must run call2dir.pl once.

New and improved

• New PerfQtree which can monitor some ops-counters on the q-tree level.
• Enhanced check_netapp_takeover: New option --ic_check checks the interconnect-links and reports how many of them are up or down.
• ShelfEnvironment can have thresholds for the temperature-object in addition to the sensor-state check.
• Snapshots-check: Add additional filter --check_only=with(out)_lun

Fixed

The following checks are backwards compatible with 7m again:

• check_netapp_cluster
• check_netapp_health
• check_netapp_license
• check_netapp_scrub
• check_netapp_spare

Plus improved the help-text in many other checks.

New and improved

• Remove --svm_in_name switch.
• Getters are getting smarter with the new update mode (e.g. --update=3min )
• New --perf_format=grafana: This allows perf-data also for status-checks which is important to visualize them in Grafana.
• New check_netapp_unused_lun: Checks for luns which are online but do not have an initiator connected.
• PerfSysNode has got a new parameter: --math=average|total.
• UsageTrend: Additional parameter --reduced_history=OK|WARNING|CRITICAL|UNKNOWN allows to specify at which level a reduced-history condition is communicated. Defaults to WARNING which is the behavior we had hardcoded so far.
• New check_netapp_process: Checks for runaway processes on a filer.

Fixed

• IMPORTANT: Fixes a bug in ServiceProcessor which made the check exiting with OK even if errors were found (false negative). This makes this update an important one for all customers relying on the check ServiceProcessor! See also Blog: ServiceProcessor Check does not alarm as expected.
• Fixes in VolumeAutosize: Under some circumstances non-autosized 7-Mode volumes were not skipped. Also the base for calculation can be changed to size instead of size-total now (More details are in the help for --base=size|size-total).
• Filter --check_only=without_lun_7m in UsageTrend can now be written as --check_only=without_lun. (The 7m-variant is deprecated now and will disappear from the UI in the next major release.)
• check_netapp_license: Exits with helpful message if not a single license is found and has got a new option --no_licenses=OK|WARNING|CRICTIAL|UNKNOWN.
• SnapshotLessVolume: Fixed a bug (Perl warning) if the number of skipped instances equals to 1.

New and improved

• The checks and getters are compatible with the upcoming DataONTAP 9.3
• Check LunAlignment has a new switch: --show_misaligned_luns.
• Check UsageTrend supports two new filter- and display switches already known from other checks: --check_only=without_lun|with_lun|... and --svm_in_name.
• The ASUP-log-check will now consider retransmitted messages and has got a --lookbehind filter to ignore messages which are too old.
• UsageTrend handles missing data in a store more flexible (tries to calculate with reduced look-behind).
• DiskPaths2 is a planed replacement for the deprecated DiskPaths check. DiskPaths2 has a different UI and is in an early alpha-state. Do not rely blindly on the config-sets delivered with this check!

Fixed

• Check Job correctly shows the number of queued jobs in its perf-data. Also enhanced the --help and examples.

New and improved

• New check PerfSysNode (amends PerfSys for clusters with DataONTAP 8.3+)
• New check LunAlignment
• New check PerfAggregate
• New check check_netapp_asup.pl monitors the autosupport-log for failed transmissions.
• Much better documentation for the growth metric of OvercommitAggr
• Add option --check_only=with_lun|without_lun to SnapshotLessVolume
• Enhance UsageTrend: --what=inodes interpolates the usage of inodes and warns if the number of available inodes will be reached within a given time.
• Enhance check_netapp_takeover: checks the metro-cluster configuration
• Snapshots --older_than and --younger_than accepts also seconds, minutes and hours (plus days and weeks as in previous versions).

Fixed

• Better error handling if --storedir points to a non-writable directory
• Sis: Handling of volumes where an error occurs while fetching the status values
• Head: Fixed several more and less cosmetic issues when one node is down
• Disk: Fixed --what=non-zeroed-spare
• UsageTrend: Fixed handling of offline or otherwise not available volumes.

New and improved

• New check ServiceProcessor
• nfs counter in PerfVolume
• PerfDisk: additional counters (user_read, latency, total_transfers, ...)
• Systeminfo (model name, DataONTAP Version, ...) in Head

Fixed

• Usage: --check_only and --svm_in_name are compatible now
• check_netapp7_vfiler handles --check_network even if vfnet is empty
• check_netapp_quotas: Option --path filters by qtree-path now
• Distribution: takeover- and quota-check are now distributed as part of the ADVANCED Bundle
• Documentation: missing api aggr-scrub-list-info in list_apis.pl --mode=7m

Fixed

• Improved the error message in case of too less data in perf-stores.
• Fixed the error-message in case a filer does not return a counter-value.
• Removed example from Snapshots which shows how to search with Snapshots for volumes without Snapshots, as this did not work as expected. Using Snapshots to check for snapshot-less volumes will result in false negatives! Existing customers should check their configuration and replace these service-checks with the new SnapshotLessVolume check.

• Fixed a bug which stoped UsageTrend from working, if some instances have been created within the look-behind-period

New and improved

• New: check_netapp_takeover - checks if the take-over facility is enabled and takeovers are possible
• New check: SnapshotLessVolume - searches for volumes without a snapshot.
• New Check Job - checks for failed jobs.
• New switch --svm_in_name for the Usage-check prefixes the volumes name with the vserver. This avoids confusing duplicates in the output.

Important

Due to a change in the store-format existing store-files must be converted (or deleted) before upgrading! Please read the installation instructions.

Fixed

• --mammamia for 7m-getter did not write into storedir
• provide --check_only=without_lun_7m for 7m-filers
• check_netapp_anycli: Document switches --like_result and --unlike_result
• get_netapp_shelfenvironment: Do not stop if one node is down
• check_netapp7_vfiler.pl: Handle undefined vfiler-status
• SnapMirrorState: Avoid UNKNOWN caused by no longer existing snap-mirrors
• SnapMirrorState/Metric: Excluding all relations and setting --no_instances=OK is ok now

New and improved

• New check: LunSize
• Faster, leaner and more reliable store-format (existing store-files must be converted before upgrading! Please read the installation instructions.)
• New switch: --ignore_case (ignore case for --exclude and --include patterns)

New and improved

• New check: check_netapp_quotas.pl - monitors quotas on a cdot filer.
• StorageUtilization output units can be controlled by --factor=ki|Mi|Gi|...
• DiskPaths: Bugfix and additional switch --port_pattern_ok=AAAA | BBBB | ...
• ShelfEnvironment False positives regarding the shelf-status solved with switch --workaround_for_279931
• ShelfEnvironment: False positive for voltage-sensors in DataONTAP 9.1P1 (solved with temporary switch --DataONTAP_91P1)

New and improved

• New check: FCPAdapter checks fcp-adapters status (online, ...)
• Usage-check: --check_only=500GiB..1TiB (ranges to specify different thresholds depending on the volumes total size)
• New switch in PerfVolume: --math=average|total
• New metric in Sis: changelog-used-percent
• New switch --ignore=^temp__ in perfdata-getter - this is the default!
• New switch --node|--vserver-switch for VolumeAutoSize check
• New switch read_ops- and write_ops-counter for PerfDisk check.
• New switch --exclude=<counter> in perfdata-getter. Required to exclude the newly added _ops-counters on older filers.
• Specific cdot-getter for the ShelfEnvironment -Check; this new getter does not need the --node parameter any more.

Fixed

• ClusterPeerHealth: Returns "UNKNOWN - No matching instances found" instead of "no data" if no peer has been found

New and improved

• Performance and History-Checks: Improved usability by setting a mostly higher --tolerance and printing detailed instructions in case of a misconfigured getter.

  • The default for --tolerance is now always 50% of the --delta
  • The checks analyze the existing store-data and print any applicable solution to stdout.

Fixed

• Snapshots: --show_problems works now as documented
• Sis: handles sis-disabled volumes (skips them if --check_disabled is not set)
• PerfDisk: tolerance was missing

New and improved

• All getters can write to more than one storedir. This allows to check from more than one peer/poller. ATTENTION 1: You need either to start with an empty storedir or migrate existing stores. (See also section Upgrading to Version 3.6.x and 3.7.x in the docs.) ATTENTION 2: Storedirs do not get created automatically any more. Add --autocreate_storedir to the getters to get back to the old behavior.
• Direct checks with overall-logic (e.g. check_netapp7_fcpstats, check_netapp_scrub) can write their instance_results into more than one storedir.
• New Multi-Tenancy Checks: ReportSpace, ReportIOPS
• New Check Sis - check for dedup-problems (stale-fingerprints, run-time)
• New check check_netapp_scrub to alarm if last scrubs timestamp is over a certain age.
• Disk check has thresholds now - get a WARNING for 1 failed disk and a CRITICAL alarm for 2 or more.
• --max_records can be configured to reduce the time to collect data (volume, vol_snapshot, disk) from cdot-filers
• Snapshots can check the size of single snapshots now (--single).

New and improved

• A new format and logic for some store-files dramatically reduces the memory-footprint of checks using performance- and history-stores.
• New check_netapp_license.pl
• New switch --vserver for LunState
• New switch --prefix for PerfLif
• New switch --uninitialized_relation_is_ok for SnapMirrorMetrics (Useful if you setup a far away vserver peer and activated protection with snapmirror - synchronization can take a long time than.)
• New switch --exclude_dependency=vclone for the Snapshot check to filter on application-dependency
• Hint in output if instances have been excluded
• Snapshots-check: --node|--vserver is optional now

Fixed

• ShelfEnvironment fixed to check 7m MetroCluster

New and improved

• Length of output can be limited by setting max_length_multi_line in check_netapp_pro.pl

New and improved

• --average implemented into PerfDisk

New and improved

• New check ClusterPeerHealth
• New check MetroClusterVserver
• New option for the collector-checks: --no_store=OK|... (most useful together with the collectors option --no_instances=OK|...)
• DiskPaths: unassigned disks are skipped
• VolumeState: Node-filter --exclude_nodes/--include_nodes introduced

Fixed

• Usage-Check: Handling of volumes with no state-attribute
• shelf-environment-object in cm-getter needs a node-name (sometimes)
• snap-mirror getter for 7-mode: Handle empty config (no snap-mirrors)
• Respect the options --timeout and --hide_uuid

New and improved

• new DiskCount check (counts spare-disks per type or storage-pool)
• new workload-object in the perf-getter (in preparation for a cdot total-ops check)
• extended lun-object in the 7m/cm-getters (in preparation for a LunSize check)

Fixed

• Usage check: unknown volumes are skipped now
• Snapshots: target type not properly shown in case of singular

New and improved

• Switch --vserver|--node is now optional for the Usage check
• Documentation and examples

Fixed

• Vserver checks operational-state (instead of admin-state) if DataONTAP >= 8.3

New and improved

• New: ShelfEnvironment checks a Shelfs temperature, power-supplies and sensors
• New: Check for non-zeroed spare disks with the Disk check
• New check BadlyPerformingDisks for detecting unbalanced disks usages.
• New check_netapp_mc_config.pl for checking a Metroclusters mode and configuration.
• New check_netapp_anycli.pl for building checks with simple CLI-commands.
• New option --alarm_limit (signals critical conditions as warning)
• check_netapp7_snapvault.pl can check the lag-time
• New option --display_factor for check_netapp7_snapvault.pl
• New option --skip_reason in Vserver check
• New option --avg in PerfCpu (for getting the averaged usage)

New and improved

• DiskPaths can check Fabric Metro Clusters with 8 disk-paths
• User-friendly time-values in addition to seconds: h, min, d(ay), (w)eek E.g. -w 3 -c 72 --factor=h instead of -w 10800 -c 259200
• New Option: --perfdata_uom_string to replace correct but not accepted units like '/s' with whatever you want. E.g. --perfdata_uom_string=s|persec|empty|°C|µs|us|...
• IfGrp lists ports which are down
• NetPort/NetPort7m: new switch --ignore_disabled
• Getter for shelf-bay objects can retrieve data from a --single_system

Documentation

• Handling of Error "Magic number checking on storable file failed"

Fixes

• NetPort handles undefined link-status (sends an alarm)
• Usage of check_netapp7_vfiler.pl mentioned wrong switch

New and improved

• SnapMirror has a getter for 7-mode
• New Check for 7-mode: check_netapp7_snapvault
• New Check for 7-mode: check_netapp7_vfiler
• New Check: check_netapp7_fcpstats: Checks for fcp crc-errors (7mode only)
• Disk shows inventory info (so actions can be taken immediately)
• New Switch to hide the lengthy UUID in output
• New: Snapshots can now filter on the snap-mirror-label
• Additional counters (evicts, disk_reads_replaced) in the FlashCash-Check
• Preparation for DiskPath: path_use_state gets collected, required for checking Fabric MetroClusters (FMCs)

Fixes

• interface-getter/IfGrp check: handle interfaces with undefined mediatype or empty link-states
• IfGrp check can handle vifs-in-vifs recursively
• The labels in the perf-data are always uniq now

Documentation

• New chapter: How to configure a cluster-mode read-only user for monitoring
• Several improvements, mainly in the Trouble-Shooting section.

New and improved

• new check: PerfTcpIp
• new check: NetInterface
• new check: PerfHostadapter
• new check: UsageTrend
• new check: SyncMirror for cdot Metro Cluster
• new switch: --no_instances=OK|WARNING|...
• new switch for volume Usage: --check_only=with_lun|without_lun
• alarm (UNKNOWN) in case of counter overflows
• microdeltas (experimental feature)
• check_netapp_spare.pl: Better (more human understandable) help and output

Fixes

• duplicate perf-instances in cmode if the number of instances was > 150

Fixes

• IMPORTANT: broken disks are no longer skipped in cluster mode
• disk getter handles unassigned disks

New and improved

• Legacy Checks ported from 2.x to 3.x: check_netapp7_head.pl, check_netapp_health.pl
• Filer can be checked for broken or unassigned disks

Fixes

• some small fixes in the cfg-files
• removed confusing default-thresholds in PerfVolume

New and improved

• Dependency Table (netapp-monitoring.info/en/help/dependencies.html)
• New switch --show_problems appends all non-ok instances to the outputs first line. Ideal for sending more comprehensive alarm messages to mobile-phones. (More details are in the blog)

Fixes

• StorageUtilization returns 25 out of bounds
• snapshot-getter can handle offline/restricted/busy volumes

New and improved

• --no_relations_is_ok switch for SnapMirror checks
• DiskPaths can check up to 4 paths
• Documentation improvements

New and improved

• VolumeAutosize for cm and DataONTAP 8.2.1
• PerfLif for DataONTAP 8.2.1 or higher
• New check Disk (replaces the deprecated DiskFailed)
• New option-argument to switch of performance-data: --perf_data=suppress

Fixes

• Lots of fixes and enhancements based on feedback from internal UA-tests
• Lots of fixes and enhancements based on customer-feedback (thanks to all of you!)
• Reduced ONTAPI requirement for all 7m checks (1.11 instead of 1.15)

Documentation

• Much more complete configuration files (Nagios cfg) for 7m and cm
• Getter tells user correct getter-objects if a unknown object is detected
• Getter: Pseudo-object 'explore' lists all available objects (-o explore).