AIOps 2.6 - Test Summary Report

Contents

  1. Definitions and Acronyms Used
  2. Purpose
  3. Test Execution Summary
    1. 3.1 Jira Defects Tested
  4. Outstanding Defects in Release 2.6
  5. Areas not tested
  6. Known Issues / Limitations
  7. Test Result Reference

Definitions and Acronyms Used

List the definitions and acronyms used in this document in alphabetical order

Acronym / Item

Expansion / Definition

PWF

Packaged WorkFlow

Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of AIOps Release 2.6.0

Executive Summary

End to end functional and regression testing of AIOps is completed.

Period of Testing: 13th May 2022 to 8th June 2022

Test Execution Summary

Sl No.

Area

Execution Status

Comments

1

AIOps PWF

Completed

 

3.1 Jira Defects Tested

 

AIOps

Delivery -TFS ID

AIOps – TFS ID

 

Comments

7732

91810

Verified – QA Passed

 

8170

96750

Verified – QA Passed

 

8654

97962

Verified – QA Passed

 

 

Outstanding Defects in Release 2.6

AIOps 2.6 Open Defects

 

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations(Deferred Defects)

AIOps PWF

 

  • Top alerts considered for MTBF calculation in Dashboard and device details page will be different if more than 5 Top alerts has same count.

  • Resolve option in addition to Recommended Resolution in listing page will be available only on alert clusters created or updated after go-live

  • For email parsing, the first regex line in the regex Template is expected to match the field in start of the alert in the Email Message

  • Automation Stories last modified date is shown only to admin or automation story owner in AIOps

  • Custom field dates are not converted and shown as per preferred Time zone

  • Uploading Custom mapping files up to size 1MB is supported.

  • Dashboard creation conflicts when two projects dashboard is opened at the same time from two different tabs

  • No data displayed for top categories widgets in existing projects.

  • Configured columns in device details list is not maintained on re-login

  • Active Surge cluster will be shown on top of the page it is fetched (in sorted order) and not on top of the first page when custom query is applied.

  • Dashboard Alert Pattern – Range indicator of alert Pattern is not positioned centrally to widget or vertically to the right of widget

  • Sync with ITSM for ticket updates will be affected, when there are multiple projects’ API channels in use simultaneously.

  • Alert Processing report from Alert console will show error message to choose shorter time range when API scanner channel polls empty results continuously for long time.

  • Email Channels should be created and authenticated from service account in incognito window to have it working.

  • Multi realm projects’ dashboard should be accessed from different browsers if accessed simultaneously

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, More information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Additional details if added to solarwinds mail format, gets added to previous field data in ES

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering larger value does not work as expected.

  • AIOps currently support iHUB channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API.

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

  • Flavor based role access is not available.

Test Result Reference

AIOps PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 2.6.0

28

26

2

174

90%

0

0

7

0

Manual Regression - 2.6.0

 

 

 

113

99%