AIOps 2.0 - Test Summary Report

Contents

  1. Definitions and Acronyms Used
  2. ‎Purpose
  3. Executive Summary
  4. Test Execution Summary
    1. Jira Defects Tested
  5. Outstanding Defects in Release 2.0
  6. Areas not tested
  7. Known Issues / Limitations
  8. Test Result Reference

Definitions and Acronyms Used

List the definitions and acronyms used in this document in alphabetical order

Acronym / Item

Expansion / Definition

PWF

Packaged WorkFlow

 

 


‎Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of ITOps Release 2.0/2.0.1/2.0.2.

Executive Summary

End to end functional and regression testing of ITOps is completed.

Period of Testing: 07th June 2021 to 15th July 2021

Test Execution Summary

Sl No.

Area

Execution Status

Comments

1

ITOps PWF

Completed

 

Jira Defects Tested

ITOps

Jira ID/TFS ID

Test Status

Comments

SDI-510

Verified – QA Passed

 

SDI-926

Verified – QA Passed

 

SDI-875

Verified – QA Passed

 

SDI-555

Verified – QA Passed

 

3969

Verified – QA Passed

 

Outstanding Defects in Release 2.0

ITOps 2.0 Open Defects

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations(Deferred Defects)

ITOps PWF

  • Sync with ITSM for ticket updates will be affected, when there are multiple projects’ API channels in use simultaneously.

  • Alert Processing report from Alert console will show error message to choose shorter time range when API scanner channel polls empty results continuously for long time.

  • Email Channels should be created and authenticated from service account in incognito window to have it working.

  • Impact/Urgency cannot be set via ticket templates for the tickets created from ITOps

  • Multi realm projects’ dashboard should be accessed from different browsers if accessed simultaneously

  • Dashboard Alert Pattern - With top 5 alerts for each day with different alert names coming in for different days, selecting large duration in date filter - X axis data goes not readable

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, More information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Additional details if added to solarwinds mail format, gets added to previous field data in ES

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering larger value does not work as expected.

  • ITOps currently support iHUB channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API.

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

Test Result Reference

ITOps PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 2.0/2.0.1/2.0.2

50

50

0

2254

95%

0

1

22

4

Regression Release 2.0

 

 

 

397

98%