AIOps 2.2 - Test Summary Report


Contents

  1. Definitions and Acronyms Used
  2. Purpose
  3. Executive Summary
  4. Test Execution Summary
    1. Jira Defects Tested
  5. Outstanding Defects in Release 2.2
  6. Areas not tested
  7. Known Issues / Limitations
  8. Test Result Reference

Definitions and Acronyms Used

List the definitions and acronyms used in this document in alphabetical order

Acronym / Item

Expansion / Definition

PWF

Packaged WorkFlow

 

 

Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of AIOps Release 2.2

Executive Summary

End to end functional and regression testing of AIOps is completed.

Period of Testing: 19th November 2021 to 10th December 2021

Test Execution Summary

Sl No.

Area

Execution Status

Comments

1

AIOps PWF

Completed

 

Jira Defects Tested

N/A

Outstanding Defects in Release 2.2

AIOps 2.2 Open Defects

 

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations(Deferred Defects)

AIOps PWF

  • In progress indicator is not shown to user when AutoResolution triggered, moves to HITL and details for same not captured in BE

  • Impact and Urgency in ITSM cannot be changed and tested in AIOps for High and Medium combinations.

  • Dashboard Alert Pattern – Range indicator of alert Pattern is not positioned centrally to widget or vertically to the right of widget

  • Sync with ITSM for ticket updates will be affected, when there are multiple projects’ API channels in use simultaneously.

  • Alert Processing report from Alert console will show error message to choose shorter time range when API scanner channel polls empty results continuously for long time.

  • Email Channels should be created and authenticated from service account in incognito window to have it working.

  • Multi realm projects’ dashboard should be accessed from different browsers if accessed simultaneously

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, More information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Additional details if added to solarwinds mail format, gets added to previous field data in ES

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering larger value does not work as expected.

  • AIOps currently support iHUB channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API.

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

  • Reassignment template is not working as it takes from ISTM default configuration(Bug 77306) – will be resolved in 2.3.

  • Alerts satisfying failure policy and rule is not getting added to ES as failureAlert – true (Bug 77013) – will be resolved in 2.3
     

Test Result Reference

AIOps PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Regression/E2E Testing for 2.2

47

47

0

151

100%

0

0

2

0