AIOps 2.3-Test Summary Report

Contents

  1. Definitions and Acronyms Used
  2. Purpose
  3. Executive Summary
  4. Test Execution Summary
    1. Jira Defects Tested
  5. Outstanding Defects in Release 2.3
  6. Areas not tested
  7. Known Issues / Limitations
  8. Test Result Reference

Definitions and Acronyms Used

List the definitions and acronyms used in this document in alphabetical order

Acronym / Item

Expansion / Definition

PWF

Packaged Workflow

 

 

Purpose

The purpose of this document is to summarize the end- to- end testing activities performed by QA team as part of AIOps Release 2.3

Executive Summary

End to end functional and regression testing of AIOps is completed.

Period of Testing: 16th December 2021 to 27th December 2021

Test Execution Summary

Sl No.

Area

Execution Status

Comments

1

AIOps PWF

Completed

 

Jira Defects Tested

N/A

Outstanding Defects in Release 2.3

Open Defects_2.3

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations (Deferred Defects)

AIOps PWF

  • Correlation has issues when custom field added as a 'Group By' attribute.

  • In progress indicator is not shown to user when Auto Resolution triggered, moves to HITL and details for same not captured in BE

  • Active alerts remain in open state with the next action empty.

  • Impact and Urgency in ITSM cannot be changed and tested in AIOPs for high and medium combinations.

  • 'Remove hold' is captured wrongly in the activity session in ITSM tickets.

  • Dashboard Alert Pattern – Range indicator of alert Pattern is not positioned centrally to widget or vertically to the right of widget

  • Sync with ITSM for ticket updates will be affected when there are multiple projects’ API channels in use simultaneously.

  • Alert Processing report from Alert console will show error message to choose shorter time range when API scanner channel polls empty results continuously for long time.

  • Email Channels should be created and authenticated from a service account in the incognito window to have it working.

  • Multi realm projects’ dashboard should be accessed from different browsers if accessed simultaneously

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, more information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Additional details if added to SolarWinds mail format, gets added to previous field data in ES

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering a larger value does not work as expected.

  • AIOps currently support iHub channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API.

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

Rollback

  • Full rollback option is not available. Rollback will take care of only items upgraded using upgrade container. Any upgrade done outside of upgrade container will not be rollbacked using the shipped rollback feature

Test Result Reference

AIOps PWF 

# Of User Stories

# Of Test Cases

# Of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

E2E Testing for 2.3

12

12

0

12

100%

0

0

0

5

Regression

 

 

 

309

100%