AIOps 2.5 - Test Summary Report

Contents

  1. Definitions and Acronyms Used
  2. Purpose
  3. Executive Summary
  4. Test Execution Summary
    1. 3.1 Jira Defects Tested
  5. Outstanding Defects in Release 2.5
  6. Areas not tested
  7. Known Issues / Limitations
  8. Test Result Reference

Definitions and Acronyms Used

List the definitions and acronyms used in this document in alphabetical order

Acronym / Item

Expansion / Definition

PWF

Packaged WorkFlow

 

 

Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of AIOps Release 2.5.0/2.5.1/2.5.2

Executive Summary

End to end functional and regression testing of AIOps is completed.

Period of Testing: 21st March 2022 to 3rd May 2022

Test Execution Summary

Sl No.

Area

Execution Status

Comments

1

AIOps PWF

Completed

 

3.1 Jira Defects Tested

 

AIOps

Delivery -TFS ID

AIOps – TFS ID

 

Comments

7181, 7239

79035

Verified – QA Passed

 

7488

84818

Verified – QA Passed

 

7986, 8070

91563

Verified – Dev Passed

 

7938

91158

Verified – Dev Passed

 

 

Outstanding Defects in Release 2.5

AIOps 2.5 Open Defects

 

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations(Deferred Defects)

AIOps PWF

  • Automation Stories last modified date is shown only to admin or automation story owner in AIOps

  • Custom field dates are not converted and shown as per preferred Time zone

  • Uploading Custom mapping files up to size 1MB is supported.

  • Dashboard creation conflicts when two projects dashboard is opened at the same time from two different tabs

  • No data displayed for top categories widgets in existing projects.

  • Configured columns in device details list is not maintained on re-login

  • Active Surge cluster will be shown on top of the page it is fetched (in sorted order) and not on top of the first page when custom query is applied.

  • Dashboard Alert Pattern – Range indicator of alert Pattern is not positioned centrally to widget or vertically to the right of widget

  • Sync with ITSM for ticket updates will be affected, when there are multiple projects’ API channels in use simultaneously.

  • Alert Processing report from Alert console will show error message to choose shorter time range when API scanner channel polls empty results continuously for long time.

  • Email Channels should be created and authenticated from service account in incognito window to have it working.

  • Multi realm projects’ dashboard should be accessed from different browsers if accessed simultaneously

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, More information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Additional details if added to solarwinds mail format, gets added to previous field data in ES

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering larger value does not work as expected.

  • AIOps currently support iHUB channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API. 

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

  • Flavor based role access is not available.

Test Result Reference

AIOps PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 2.5.0/2.5.1/2.5.2

25

25

0

125

98%

0

0

8

2

Manual Regression - 2.5

 

 

 

87

100%