AIOps 2.4 - Test Summary Report

Contents

  1. Definitions and Acronyms Used
  2. Purpose
  3. Executive Summary
  4. Test Execution Summary
    1. 3.1 Jira Defects Tested
  5. Outstanding Defects in Release 2.4
  6. Areas not tested
  7. Known Issues / Limitations
  8. Test Result Reference

Definitions and Acronyms Used

List the definitions and acronyms used in this document in alphabetical order

Acronym / Item

Expansion / Definition

PWF

Packaged WorkFlow

 

 

Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of AIOps Release 2.4.0/2.4.1/2.4.2/Patch release for 2.4

Executive Summary

End to end functional and regression testing of AIOps is completed.

Period of Testing: 4th January 2022 to 28th February 2021

Patch Release: Completed on 23rd March 2021

Test Execution Summary

Sl No.

Area

Execution Status

Comments

1

AIOps PWF

Completed

 

3.1 Jira Defects Tested

 

AIOps

Delivery -TFS ID

AIOps – TFS ID

 

Comments

4075, 4218

81871

Verified – QA Passed

 

7181

79035

Verified – Not reproducible

 

7435

83439

Verified – QA Passed

 

7379

82347

Verified – QA Passed

 

7488

84818

Verified – Not reproducible

 

7437

83429

Verified – Not reproducible

 

 

Outstanding Defects in Release 2.4

 

AIOps 2.4 Open Defects

 

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations(Deferred Defects)

AIOps PWF

  • Uploading Custom mapping files up to size 1MB is supported.

  • Templates created for custom mapping refined enrichment, must be manually deleted on deleting a custom mapping file

  • Active Surge cluster will be shown on top of the page it is fetched (in sorted order) and not on top of the first page when custom query is applied.

  • Dashboard Alert Pattern – Range indicator of alert Pattern is not positioned centrally to widget or vertically to the right of widget

  • Sync with ITSM for ticket updates will be affected, when there are multiple projects’ API channels in use simultaneously.

  • Alert Processing report from Alert console will show error message to choose shorter time range when API scanner channel polls empty results continuously for long time.

  • Email Channels should be created and authenticated from service account in incognito window to have it working.

  • Multi realm projects’ dashboard should be accessed from different browsers if accessed simultaneously

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, More information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Additional details if added to solarwinds mail format, gets added to previous field data in ES

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering larger value does not work as expected.

  • AIOps currently support iHUB channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API.

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

Test Result Reference

AIOps PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 2.4.0/2.4.1/2.4.2/Patch 2.4

40

40

0

276

98%

0

0

2

0

Manual Regression - 2.4

 

 

 

55

100%