Release Notes SevOne NMS 7.1.0

Below please find the Release Notes for SevOne NMS 7.1.0. Please contact your Technical Account Manager (if applicable) or Systems Engineering Team or Support Team to discuss and plan the installation. Thank you for being a customer.

Note: If you have a WiFi solution in your environment and want to upgrade to SevOne NMS 7.1.x, please upgrade from SevOne NMS <= 7.0.x to SevOne NMS 7.1.2 directly.

Browser Requirements

Minimum Resolution: 1200x768 Browsers:

  • Modern, standards-compliant browser
  • JavaScript enabled
  • Pop-up blocker disabled for hostname/IP

The following browsers are supported in the current versions of SevOne. SevOne recommends use of the latest version of your preferred (supported) browser.

Vendor Family SevOne NMS 7.1
Google Chrome (latest) Tested, Supported, & Recommended
Mozilla Firefox (latest) Tested & Supported
Microsoft Edge (latest) Supported (with limited testing performed)
Apple Safari (latest) Supported

Tested = Complete UI regression testing completed prior to release of updates.

Supported = Developer-led testing and resolution of any customer reported defects. No complete UI regression test is performed.

Facts & Requirements

  • SevOne NMS on Red Hat Enterprise Linux (RHEL) release 8.10 (Ootpa)
  • SevOne Data Insight 7.x is compatible with both SevOne NMS 6.8.x and 7.x versions
    Note: If you are using SevOne NMS versions earlier than 6.8.x or later than 7.x in combination with SevOne Data Insight 7.x, SevOne does not guarantee full feature compatibility or expected performance.
  • SOA must be running the same version as SevOne NMS, or a higher version
Note: For details on SevOne Data Insight, please refer to its guides and Release Notes.

Containers

Important: Containerization

As of SevOne NMS 7.0.0, SevOne is distributed using container technology, allowing a more confident deployment of the software. To run administrative commands on a SevOne appliance, the administrator must now execute commands in the context of the intended container.

By default, the container deployment of SevOne is set to be read-only.

CAUTION: stopIcon DO NOT proceed to SevOne NMS 7.0 or above if you have Expert Labs customizations. Please contact Expert Labs first. If you have deployed any of your own custom scripting, you may also require amendment(s) to these custom scripts if writing to directory structures that are now in the read-only pod.
  • The host and the container each has its own ssh config; both for the server and the client.
  • To ssh as root, you must use the sudo command.

For details on how to connect IBM SevOne NMS appliance to the Command Line Interface where SevOne NMS software runs in podman containers, please refer to SevOne NMS Connect Appliance using Command Line Interface.

For additional details, please refer to SevOne NMS System Administration Guide and / or SevOne NMS User Guide.

Other Notices

Attention: for new-installs only,

Please DO NOT install SevOne NMS 7.1.0 if you want SDN plugin (Cisco ACI Monitoring).

Note: Prior to SevOne NMS 7.0, if you had /etc/sysctl.d/99-sysctl.conf file and had modified net.ipv4.ip_forward, then after the upgrade to SevOne NMS 7.0 or above, you will need to instead create /etc/sysctl.d/90-custom.conf file and add net.ipv4.ip_forward = 0 to it.
Note: Starting SevOne NMS 6.1, SevOne stores backups of installed hot-patch bundles in /opt/patches.

/opt/patches is a reserved directory; please refrain from making any modifications.

Note: Applies to SevOne NMS 6.8 and prior releases only

During the initial deployment, when you execute SevOne-fix-ssh-keys, it produces /root/.ssh/authorized_keys file which contains your cluster's public keys.

If you have custom keys, the keys must be added to /root/.ssh/custom_keys.pub file.

  • if /root/.ssh/custom_keys.pub file does not exist, using a text editor of your choice, add the new custom key(s) to it.
  • if /root/.ssh/custom_keys.pub file already exists, concatenate the new custom key(s) after the existing custom keys in the file.

To persist the custom keys added in /root/.ssh/custom_keys.pub file, run SevOne-fix-ssh-keys script for the keys in /root/.ssh/custom_keys.pub file to be automatically added in /root/.ssh/authorized_keys file. The /root/.ssh/authorized_keys file will now contain your cluster's public keys along with a set of custom keys stored locally in /root/.ssh/custom_keys.pub file.

Retains 'all' keys - cluster's public keys & custom keys

SevOne-fix-ssh-keys

Third-Party Packages / Resources Required

The following are third-party packages updated to address security.

This is a list of third-party packages updated to address security.

Package Version

General

Kafka For SevOne Data Bus 7.1.0,
  • Internal Kafka Client (for SDP): sarama 1.41.0
  • NMS Kafka Server : 3.6.0-2.13.2.el8 (version must be < 4.x)

Upstream package used for Kafka, provided by RedHat: OpenJDK v21.0.7

Kernel

4.18.0-553.el8_10.x86_64

NOTE: The kernel will automatically get installed as part of the upgrade and will be loaded after the reboot of the appliance.

KVM

9.9.0-1.el9

(libvirt-libs)

MySQL 10.6.12-MariaDB
Nginx 1.24.0-1
PHP

8.3.11-1.el8

NOTE: To consume PHP 8, please contact Expert Labs if assistance is needed.

Signature Tools
Note: The latest files can be downloaded from IBM Passport Advantage (https://www.ibm.com/software/passportadvantage/pao_download_software.html) via Passport Advantage Online. However, if you are on a legacy / flexible SevOne contract and do not have access to IBM Passport Advantage but have an active Support contract, please contact SevOne Support Team for the file.

Once you have downloaded the package, unpack the fix / upgrade packs to obtain the required files.

  • signature-tools-2.0.3-build.1.tgz
  • signature-tools-2.0.3-build.1.tgz.sha256.txt
Artifacts
Note: For new installs / upgrades / downgrades,

the latest TAR and CHECKSUM files can be downloaded from IBM Passport Advantage (https://www.ibm.com/software/passportadvantage/pao_download_software.html) via Passport Advantage Online. However, if you are on a legacy / flexible SevOne contract and do not have access to IBM Passport Advantage but have an active Support contract, please contact SevOne Support Team for the file.

Once you have downloaded the package, unpack the fix / upgrade packs to obtain the required files.

  • TAR / CHECKSUM Upgrade:
    • v7.1.0-build20241015-153317-3595bbec.tar.gz
    • v7.1.0-build20241015-153317-3595bbec.tar.gz.sha256.txt
  • TAR / CHECKSUM Downgrade
    • v7.1.0-to-v7.0.1-build20241015-153317-3595bbec.tar.gz
    • v7.1.0-to-v7.0.1-build20241015-153317-3595bbec.tar.gz.sha256.txt
  • ISO / CHECKSUM
    • nms-7.1.0-20241014-140649-1aa56c99.iso
    • nms-7.1.0-20241014-140649-1aa56c99.iso.sha256.txt
  • AWS OVA - nms-7.1.0-20241015-153317-3595bbec-ami.ova
  • OVA - nms-7.1.0-20241015-153317-3595bbec.ova
  • QCOW2
    • nms-7.1.0-20241015-153317-3595bbec.qcow2
    • nms-7.1.0-20241015-153317-3595bbec.qcow2.sha256.txt
  • VHD
    • nms-7.1.0-20241015-153317-3595bbec.vhd.gz
    • nms-7.1.0-20241015-153317-3595bbec.vhd.gz.sha256.txt
Fabric Azure Windows 2008 (modified)
Hypervisor OpenStack >= 10.a
VMware
  • Intel-VT or AMD-V CPU extensions
  • ESXi 5.0 and later (VM version 8) - minimum requirement
  • Tested with vSphere Client version 7.0 (ESXi 7.0 and later; VM version 8)

Planning & Preparation

  • Prior to applying the patch, system creates a backup of the files and puts them into an archived file to be reverted.
  • Total Upgrade Time and Polling Outage: On a cluster consisting of 20 x (PAS 200Ks, DNCs, HSAs), upgrade takes approximately 2 hours 15 minutes. The polling outage on this cluster ranged from 2 minutes to 5 minutes. Polling outages can be slightly higher when a MySQL restart is required and it does not include the time it takes for the reboot of a new kernel. Depending on the cluster and load per appliance, times will vary. The total Netflow outage for this cluster ranged from 10 minutes to 15 minutes. Netflow outage can be up to 2 hours since the Netflow shortterm tables which hold 2 hours of data, do not get backed up when MySQL is restarted.

    Important: When a new flow interface is setup with a DNC at capacity, the system collects all existing allowed flows and denies any new flows.
  • The number of peers in a cluster must not exceed 200 peers; this includes the HSAs. This limit is due to MySQL replication maintainer.
  • On large deployments, Object Groups may take 15 minutes to update.

Forward / Reverse Migrations

Please refer to SevOne NMS Upgrade Process Guide published with this release for details on forward / reverse (upgrade / downgrade) migrations. The latest tarball files can be downloaded from IBM Passport Advantage via Passport Advantage Online. However, if you are on a legacy / flexible SevOne contract and do not have access to IBM Passport Advantage but have an active Support contract, please contact SevOne Support Team for the forward / reverse migration files.

Once you have downloaded the package, unpack the fix / upgrade packs to obtain the required files.

Deprecated / Removed Features & Functions

As of SevOne NMS 7.1, the following features/functions have been deprecated / removed. If you have any questions, please contact IBM SevOne Support, IBM Expert Labs, or your IBM Account Manager.
Feature / Function Reason Deprecated / Removed Next Steps
Cisco NAM plugin The older Cisco line cards compatibility with Cisco NAM plugin has reached end of life. This plugin is being deprecated and removed from the software. NAM data collection has been removed however, NAM reporting on historical data after the upgrade is still supported.

After upgrading to 7.1, NAM flow data cannot be collected. Historical NAM data and ability to report on it, is unaffected after the upgrade.

Cisco NBAR plugin NBAR data collection, a Cisco standard for network application recognition, has been deprecated in favor of automatic application classification available in IBM SevOne NMS 6.8+. NBAR data collection has been removed however, NBAR reporting on historical data after the upgrade is still supported.

After upgrading to SevOne NMS 7.1, i.e., after removal of NBAR functionality, user must enable NetFlow export on devices that have the NBAR functionality to continue to collect and report on the applications using NBAR and the Application ID field.

VMware plugin Use of a proxy to collect data from VMware does not meet IBM security standards. Customers should configure their vCenter to allow direct polling from all IBM SevOne peers involved in managing cluster resources. This will ensure end-to-end encryption and authentication is maintained. If you are relying on the VMWare proxy to poll a VMWare device, you will need to device move affected devices to a peer that communicates to the vCenter without the aid of a VMWare Proxy. Or, you can update your network so that the existing pollers can reach the vCenter directly.

New Features / Enhancements

  • Google Cloud Platform introduced in SevOne NMS to allow you to store, manage, and analyze data. Also used for developing, deploying, and scaling applications on Google's environment.
  • New Wi-Fi Plugin added to reduce install and setup time.
  • S1NPM-78087 - IPFIX can be consumed from SD-WAN VeloCloud edge devices.
    Cluster Manger: Cluster Settings tab > FlowFalcon subtab > by default, check boxes Create Egress Records When Not Available and Create Ingress Records When Not Available are unchecked.
    • Egress records are no longer simulated for
      • NetFlow v5
      • NetFlow v7
      • sFlow
    • In the absence of direction field 61 on the Flow Template, egress records are no longer simulated for
      • NetFlow v9
      • IPFIX
Important: IMPORTANT
If you want to upgrade SevOne NMS to SevOne NMS 7.1 and you are currently on SevOne NMS version below SevOne NMS 6.1, you must:
  • upgrade from SevOne NMS version below SevOne NMS 6.1 to SevOne NMS 6.7
  • then, upgrade from SevOne NMS 6.7 to SevOne NMS 6.8 or SevOne NMS 7.0.1+
  • now, from SevOne NMS 7.0.1+ you can upgrade to 7.1.
Warning: Metadata: As of SevOne NMS 7.1.0, Device and Location namespaces are shipped out-of-the-box (OOTB). If these namespaces already exist, they will now be renamed to Device.custom and Location.custom respectively.

Resolved Issues

Component/s Key Resolved Issues
Reporting Flow S1NPM-78019 Policy Editor / Threshold Editor: When upgrading from SevOne NMS 6.x / SevOne NMS 7.0 to SevOne NMS 7.1, you will need to recreate app alert policy based on an enabled aggregated view that includes the bandwidth metric field.

Enable App Enrichment field from Cluster Manager > Cluster Settings tab > FlowFalcon subtab must be enabled to create an app alert.

Enable the required aggregated views to make use of them for the app alert policy.

Reporting Flow S1NPM-78040 FlowFalcon Views: New raw views enabled OOTB are available for fresh installs and upgrades.
Reporting Flow S1NPM-78083 FlowFalcon Views: New aggregated views enabled OOTB are available for Fresh Installs.
Platform S1NPM-78092 Platform: You no longer need to manually clear the browser cache. The browsers will automatically flush their cache and fetch updated HTML/CSS/JavaScript when pages are reloaded after the SevOne NMS upgrade.
Platform Operations S1NPM-97402 Threshold Browser: Can successfully access Threshold Browser from NMS with Specific Device - Plugin filters.
Collection Other S1NPM-97598 Platform: NAM plugin support has been deprecated.

NOTE: After upgrading to SevOne NMS 7.1, i.e., after removal of NAM functionality, NAM flow data cannot be collected. Historical NAM data and the ability to report on it, is unaffected after the upgrade.

Collection Flow S1NPM-109207 FlowFalcon View Editor: SevOne flow is now enriched with two new standard fields.
  • Observation Domain ID
  • Template ID
These allow additional flexibility to the user to create reports that can be filtered by these values.
Metadata S1NPM-109684 Metadata: Device namespace is shipped out-of-the-box (OOTB). If this namespace already exists, it will now be renamed to Device.custom.
Platform Clustering S1NPM-109725 Platform: Allows removal of peer or HSA to be initiated from any peer.
Metadata S1NPM-109728 Metadata: Location namespace is shipped out-of-the-box (OOTB). If this namespace already exists, it will now be renamed to Location.custom.
Platform Clustering S1NPM-109893 Platform: The configuration of NMS hosts for self-monitoring occurs automatically as part of the peer/HSA add process. Further, except in single node clusters, peers do not monitor themselves.
Platform Clustering S1NPM-110145 Platform: The Self-monitoring scripts have now been replaced by peermon scripts.
Platform Clustering S1NPM-110147

Policy Browser: Default policies now support the following.

  • SevOne Process: mysqld Availability
  • SevOne Process: mysqld Instances
Platform S1NPM-110280 Platform: After successful upgrade to SevOne NMS 7.1, the NAT deployment scripts work as expected.
Alerts S1NPM-110318 Alert Emails: Improved algorithm to handle alertmailer efficiently on all alerts attempting to mail.
Platform S1NPM-110338 Platform: BGP / ASPath feature deprecated from flow system.
Platform S1NPM-110594 Platform: VMware Proxy deprecated. From Administration > Cluster Manager > select a peer > choose Peer Settings tab > General > VMware Proxy has been removed.
Platform S1NPM-110671 Platform: Active Directory feature deprecated from the Flow system.
Platform S1NPM-110779 Platform: nginx certificate and private key can be found in /secrets/nginx/nginx.crt and /secrets/nginx/nginx.key respectively.
Platform S1NPM-110931 Platform: NBAR plugin support has been deprecated.

NOTE: NBAR data collection has been removed however, NBAR reporting on historical data after the upgrade is still supported.

After upgrading to SevOne NMS 7.1, i.e., after removal of NBAR functionality, user must enable NetFlow export on devices that have the NBAR functionality to continue to collect and report on the applications using NBAR and the Application ID field.

Platform S1NPM-111075 Cluster Manager:
  • Cluster Overview tab > added field Unknown Flow Devices to display flow devices that are not mapped with a device in Device Manager.
  • Peers tab > added column Unknown Flow Device Count to display the number of flow devices in a peer that are not mapped with a device in Device Manager.
Metadata Schema S1NPM-111096 Metadata Schema: Add Attribute to a namespace includes new value Accepted Values for field Type. It allows you to enter comma-separated list of accepted values in field Accepted Value List.
Platform S1NPM-111103 Device Manager: When exporting CSV from Device Manager, ' is added at the beginning of the data to prevent remote code execution if it starts with the following.
  • = (equals to)
  • + (plus)
  • - (minus)
  • @ (at)
  • 0x09 (tab)
  • 0x0D (carriage return)
Platform S1NPM-111229 Platform: On a device with many indicator types (for example, 10k or more) associated to it, allows successful enabling / disabling of the indicator types.
Flow S1NPM-111278 Flow Configuration: Apps and Protocols > tab App Mapping > Flow apps for SaaS OOTB app profiles are not visible as they are proprietary to IBM. Please run flow reports including App Profile field to gain visibility into SaaS application usage and associated endpoints.
AWS / Azure S1NPM-111298 AWS / Azure: Can configure tag collection.
REST API S1NPM-111315 REST API:
  • can create mapping of entity with metadata attribute with type Accepted Values.
  • can delete mapping of entity with metadata attribute with type Accepted Values.
  • can create mapping with Accepted Values attribute for all entities.
Platform S1NPM-111396 Device Manager: Select a device and click the Metadata icon under column Actions. In Edit Metadata for <device name> pop-up, editable attributes with Type = Accepted Values, allows you to choose one or more Value from the list of acceptable values available in the drop-down list.
Platform S1NPM-111523 Platform: Kernel has been updated to 4.18.0-553.el8_10.x86_64.
REST API S1NPM-111530 REST API: Endpoint GET /api/v3/statistics/cluster/metering added to filter out only licensing metrics details from the cluster and peers. Allows user to fetch historical data for current week and current month.
Platform S1NPM-111835 REST API: Endpoint DELETE /api/v3/plugins/object_subtype/rule added for SNMP objects subtype rule.
Platform S1NPM-111969 Policy Browser: Several OOTB Self-monitoring policies have been updated with new names and descriptions to better reflect their functions.
AWS S1NPM-111971 AWS: Sizing chart for AWS instance types for SevOne appliances updated based on Amazon's recommendation to use the latest EC2 generations.
Alerts S1NPM-111977 Alert Mailer: The clear alert email shows the time the alert was cleared and not the time when the alert was last triggered.
AWS S1NPM-112100 AWS: The following services have been added.
  • Lambda
  • Route 53 Health Checks
  • Route 53 Resolvers
FlowFalcon Views S1NPM-112325 FlowFalcon Views: Top Types of App has been renamed to Top Types of Service.
Platform S1NPM-113453 Platform: If you are on SevOne NMS 7.0.1 and want to upgrade to SevOne NMS 7.0.2 or above, you may encounter an issue with the upgrade. In SevOne NMS 7.1.1, task to wait for requestd is now more robust; upgrade from SevOne NMS 7.0.1 to SevOne NMS 7.1.1 is successful.

37 issues

CVEs

Important: It is strongly recommended to run the external security scans such as Nessus, Snyk, or similar on the latest available patch for this release to verify whether the vulnerability has been addressed. If upgrading the production environment is not currently possible, these scans can still be performed in a lab or test environment.
CVEs CVEs (continued) CVEs (continued)
CVE-2018-17199 CVE-2021-26690 CVE-2022-23943
CVE-2019-0190 CVE-2021-26691 CVE-2022-26377
CVE-2019-0211 CVE-2021-33193 CVE-2022-28615
CVE-2019-0215 CVE-2021-34798 CVE-2022-29404
CVE-2019-10081 CVE-2021-36160 CVE-2022-30556
CVE-2019-10082 CVE-2021-39275 CVE-2022-31813
CVE-2019-10097 CVE-2021-40438 CVE-2022-36760
CVE-2019-9517 CVE-2021-44224 CVE-2023-25690
CVE-2020-11984 CVE-2021-44790 CVE-2023-27522
CVE-2020-35452 CVE-2022-22719 CVE-2023-31122
CVE-2020-9490 CVE-2022-22720 CVE-2023-44487
CVE-2021-23017 CVE-2022-22721 CVE-2024-27316

Known Issues

This section lists issues that SevOne is aware of in the 7.1.0 release. Most of these issues were discovered during quality assurance testing and are published here to provide you with information that may be relevant when you plan your update. This list does not include feature requests or low impact issues that do not affect functionality. If you have questions, comments, or concerns, please contact us.

  • If you have a scenario where adding HSA has failed during the masterslave console, format slave step, you may execute the following steps as a workaround.
    • Using a text editor of your choice, edit /config/cron.d/mode file.
    • Search for the line containing discover-netflow.
    • Comment this line by adding a # at the start of this line.
    • Save /config/cron.d/mode file.
    • Add the HSA.
    • Using a text editor of your choice, edit /config/cron.d/mode file again.
    • Search for the line containing discover-netflow.
    • Uncomment this line by removing the # that is at the start of this line.
    • Save /config/cron.d/mode file.
  • REST API docs are unavailable when the domain name has an underscore. For example, http://sevone_test1/api/docs/ or http://sevone_test1.sevone.com/api/docs/.
  • Wi-Fi Plugin: At present, Wi-Fi Audit Logs cannot be deleted.
Component/s Key Known Issues
Collection xStats + Deferred S1NPM-77884 Platform: Evaluation of Synthetic Indicator does not happen when the data point goes into backfill task of process SevOne-ingestion-resolved.
Platform Operations S1NPM-77927 Platform: SevOne import/export does not work with the AWS plugin device.
Platform Operations S1NPM-79915 SNMP: When SNMPv3 credentials are updated, polld requires a restart as the cache does not reflect the new credentials.
Platform S1NPM-110201 SevOne Data Publisher: When configuring the kerberos config krb5.conf file, SDP will not work if variable dns_canonicalize_hostname is set.
Platform S1NPM-110549 xStats: In SevOne NMS 7.0 and above, the configuration of the xStats adapters based on ADK is not migrated properly after the upgrade from a prior release.

Workaround: The following steps must be executed manually post-upgrade.

  • You must be in the NMS container. To enter the NMS container, execute the following command.
    
    sudo podman exec -it nms-nms-nms /bin/bash
    
  • Execute the following command to identify the log rotations and cron.d files for your adapter.
    
    echo "adapterName,cronDfile,logDir,logRotateFile";\
    find /opt/sevone-xstats/ -type f -name install.config.json | \
    grep -v backup | \
    while read INSTALLCONFIGFILE;do adapterName=$(cat ${INSTALLCONFIGFILE} | \
    jq -r '.adapterName');LOGDIR=$(cat ${INSTALLCONFIGFILE} | jq -r '.logDir');LOGROTATEDIR=$(cat ${INSTALLCONFIGFILE} | \
    jq -r '.components.Logrotate.logrotateDir');LOGROTATEFILE=$(cat ${INSTALLCONFIGFILE} | \
    jq -r '.components.Logrotate.logrotateFile');CRONDIR=$(cat ${INSTALLCONFIGFILE} | \
    jq -r '.components.Cron.cronDir');CRONDFILE=$(cat ${INSTALLCONFIGFILE} | \
    jq -r '.components.Cron.cronFile');echo "${adapterName},${CRONDIR}/$(eval echo \"$CRONDFILE\"),$(eval echo \"$LOGDIR\"),${LOGROTATEDIR}/$(eval echo \"$LOGROTATEFILE\")";done
    
    where the output is,
    
    adapterName,cronDfile,logDir,logRotate
    GenericCSVTransform,/etc/cron.d/xstats-GenericCSVTransform,/var/log/xstats/GenericCSVTransform,/etc/logrotate.sevone.d/adapter-xstats-GenericCSVTransform
    AccedianCSVTransform,/etc/cron.d/xstats-AccedianCSVTransform,/var/log/xstats/AccedianCSVTransform,/etc/logrotate.sevone.d/xstats-AccedianCSVTransform
    AccedianNidXmlTransform,/etc/cron.d/xstats-AccedianNidXmlTransform,/var/log/xstats/AccedianNidXmlTransform,/etc/logrotate.sevone.d/xstats-AccedianNidXmlTransform
    
  • Move the following files to the correct folder.
    
    mv /etc/cron.d/<your xStats adapter name> /config/cron.d/<your xStats adapter name>
    
    mv /etc/logrotate.sevone.d/adapter-xstats* /config/logrotate.sevone.d/
    
    mv /etc/logrotate.sevone.d/xstats* /config/logrotate.sevone.d/ 
    
Platform S1NPM-112463 Platform: Peering does not properly distribute ssh keys in Hub-and-Spoke setup.
NMS supports a Hub-and-Spoke setup, where some peers cannot reach each other across the network, as long as the cluster master / cluster leader is fully reachable by all peers. The objective is to make a best-effort attempt to support functions that do not strictly require connectivity to other peers. One case where this standard is not reached is during peering. If a new peer is being added to the cluster which lacks connectivity to just one other peer (not even the cluster master / cluster leader) then, while peering will succeed, the distribution of ssh keys will fail, and no keys will be distributed.

2024-09-23T21:38:49+00:00 SevOne soa: {"error":"rpc error: code = FailedPrecondition desc = problem encountered executing command",\
"fatal":false,"level":"warning","method":"AddNewNodeStage2","module":"clusterOrchestrator",\
"msg":"failed to fix ssh keys, user should manually fix them from the cluster leader","time":"2024-09-23T21:38:49Z"}
This prevents ssh communication even between peers that are connected across the network, including the cluster master / cluster leader. Running SevOne-fix-ssh-keys manually from the new peer also fails.

SevOne-fix-ssh-keys

time=2024-09-23T22:37:30.503Z level=ERROR msg="error getting public keys" error=<nil> failureIds="[PeerId: 2 | Peer State: Active]"

The reason is simply that the operation bails entirely if keys are failed to be obtained from even a single peer.

Workaround: Run SevOne-fix-ssh-keys manually on the Cluster Master / Cluster Leader after peering.

Platform S1NPM-112542 Platform: dex and samplicator do not restart on boot even if running previously.

systemctl enable dex

Failed to enable unit: Unit /run/systemd/generator/dex.service is transient or generated.
  • dex is needed if and only if you are using Single Sign-On (SSO). Otherwise, dex should not be running.
  • samplicator is an optional service if you want to multiplex flow.
Workaround:
  • If you are using Single Sign-On (SSO), you must start dex if it is not already running.
    
    systemctl restart dex
    
  • To ensure dex starts on reboot, using a text editor of your choice, edit /etc/containers/systemd/dex.container file.
    
    vi /etc/containers/systemd/dex.container
    
  • Uncomment the following line to enable dex. i.e., remove the # before WantedBy.
    Change from:
    
    [Install]
    #WantedBy=multi-user.target
    
    to:
    
    [Install]
    WantedBy=multi-user.target
    
  • Save file /etc/containers/systemd/dex.container.
Note: This will be have to be re-executed one time after a SevOne NMS upgrade or downgrade.
Platform S1NPM-113036 Platform: Downgrade from SevOne NMS 7.1.0 may fail on mysqldump on large clusters.

Workaround: Retry the downgrade process.

NMS S1NPM-113539 xStats: If you have upgraded your SevOne NMS from version 6.x to 7.x, you must set the following variables from Command Line Interface before running /opt/sevone-xstats/GenericCSVTransform/bin/console.php script.

PHPRC=/config/php.d/php-cli.ini

HOME=/
NMS / WiFi S1NPM-114363 If you have WiFi solution in your environment and on the cluster leader, NO_PREFIX flag is either set to true or the flag is not set in /config/collectors/wifi/configuration/wifi_global_settings.json file and you want to do an upgrade, it will result in duplicate devices and loss of data.

Workaround: Please refer to the note block titled WiFi solution in your environment? for details.

NMS S1NPM-114545 Platform: SevOne-trapd stops processing traps with high CPU.

11 issues