SmartOps v7.1 Test Summary Report
Click here to see this page in full context
  1. Navigate To: SmartOps Document Center >
  2. SmartOps Document Center >
  3. Release Documentation >
  4. Platform >
  5. 7.1 >
  6. Test Summary Report

 

SmartOps v7.1 Test Summary Report

FinOpsFinOps

Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of release 7.1 for Invoice Extraction PWF.

Executive Summary

End to end functional and regression testing of Invoice Extraction PWF is completed.

Period of Testing: 28th Oct 2020 to 15th Jan 2021

Test Execution Summary

Sl No.

Area

Execution Status

Open Defects if any

1

Invoice Extraction PWF

Completed


Jira Defects Tested

FinOps

Jira ID

Test Status

Comments

SDI-405

Verified – QA Passed


SDI-709

Verified – QA Passed


SDI-607

Verified – QA Passed


SDI-503

Verified – QA Passed


SDI-532

Verified – QA Passed


SDI-533

Verified – QA Passed


SDI-615

Verified – QA Passed


SDI-616

Verified – QA Passed


SDI-637

Verified – QA Passed


SDI-663

Verified – QA Passed


SDI-691, SDI-706

Verified – QA Passed


Outstanding Defects in Release 7.1


Innovation Engineering - FinOps - All Documents (sharepoint.com)


Areas not tested

N/A

Known Issues / Limitations

Module

Limitations

Invoice Extraction PWF

  • Point and Crop and Table Rebound will not work for new invoice

  • OCR data is incorrect, point and crop will not work properly

  • If field of interest are part of line items description, “Add to line items” will not work properly

  • Accuracy Issue associated with tesseract model, hence documents whose rotation angle is 180 will be inverted

  • Extraction is not possible for FOI which are in dark background

  • Accuracy issue for line item data

  • Invoice extraction in DU is failing intermittently with errors

  • When a batch is processed in DU there is a delay in reflecting the invoice count in run list and updating the flags

  • 5 to 7 minutes delay is observed in updating accuracy analytics after the corrections

  • Advanced page hangs when project is having more than 700 invoices

Test Result Reference

Invoice Extraction PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
‎Passed

Partially
‎Passed

Total

Passed %

Critical

High

Medium

Low

Release 7.0.0

31

29

2

375

77.00%

0

1

28

13

Regression Release 7.0.0


150

93.00%

0

0

0

12


ITOpsITOps

Purpose

The purpose of this document is to summarize the end to end testing activities performed by QA team as part of ITOps Release 7.1.

Executive Summary

End to end functional and regression testing of ITOps is completed.

Period of Testing: 19th Nov 2020 to 21st Dec 2020

Test Execution Summary

Sl No.

Area

Execution Status

Open Defects if any

1

ITOps PWF

Completed


Jira Defects Tested

ITOps

Jira ID

Test Status

Comments

SDI-559

Verified – QA Passed


SDI-534

Verified – QA Passed


SDI-630

Verified – QA Passed


SDI - 720

Verified – QA Passed


SDI-696, SDI-677

Verified – QA Passed


SDI-558

Verified – QA Passed


SDI-732

Verified – QA Passed


SDI-712, SDI-694

Verified – QA Passed


SDI-784

Verified – QA Passed


SDI-781

Verified – QA Passed


SDI-783

Verified – QA Passed


SDI-666

Verified – QA Passed


SDI-661

Verified – Reopened


Outstanding Defects in Release 7.1

ITOps 7.1 Open Defects

Areas not tested

N/A

Known Issues / Limitations

Module

Limitations(Deferred Defects)

ITOps PWF

  • Dashboard - Time left displayed is not accurate in the SLA graph.

  • Dashboard Alert Pattern - With top 5 alerts for each day with different alert names coming in for different days, selecting large duration in date filter - X axis data goes not readable

  • Email Template - fields been missed in incoming alert template, data in other fields gets added with wrong data.

  • Email Verba Template - Alert id, More information URL is not captured in Elastic search. Alert Attribute if not hostname OID, is not captured.

  • Email Template - Alert time not available in the template field, additional comma gets added to other Alert data in ES, resulting in failing to fetch inventory details

  • Email Template - Additional details if added to solarwinds mail format, gets added to previous field data in ES

  • Service now tickets created from ITOps, changing priority in SNOW is not changing severity of Alert back in ITOps.

  • Alert listing - Auto Refresh - No maximum limit validation available. Entering larger value does not work as expected.

  • ITOps currently support iHUB channel’s – Queue, API Scanner and Email only. Queue channel - Add/Edit is possible only via API.

  • RealTime workflow is no longer used in ITOps PWF

  • Same email id cannot be configured in multiple source emails of Extract email workflow.

  • 25% of alerts goes to correlation – incomplete, when we have more than 10 alerts coming in a minute.

Test Result Reference

ITOps PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
‎Passed

Partially
‎Passed

Total

Passed %

Critical

High

Medium

Low

Release 7.1

31

29

2

484

96%

0

7

43

27

Regression Release 7.1




403

100%

0

0

0

0



PlatformPlatform

Purpose

The purpose of this document is to summarize the end to end testing activities performed by Smartops E2E QA in platform as part of release 7.1

Executive Summary

End to end functional and TUC regression of Core Platform is completed.

Period of Testing: 28th Oct 2020 to 17th Dec 2020

Jira Defects tested

SmartOps platform

Jira ID

Test Status

Comments

SDI-123

Verified – QA Passed

 

SDI-398

Verified – QA Passed

 

 

VAPT Defects Tested

TFS ID

Test Status

Comments

38134

Closed

 

38135

Closed

 

Outstanding Defects in Release 7.1

Core Platform

Defect ID

Title

Severity

40931

7.1.2/QA/CorePlatform: Ihub email Imap special charectors in subject mail processing failed

3 – Medium

 

Areas not tested

Note: User stories 35734,37388 and 37581 not part of 7.1 execution, will be tested in 7.1.2 hot fix release.

Known Issues / Limitations

Not applicable
‎

Test Result Reference

SmartOps Platform

# of User Stories

# of Test Cases

# of Open Defects
‎ (Critical/High/Med/Low)

Total

Fully
‎ Passed

Partially
‎ Passed

Total

Passed %

Critical

High

Medium

Low

Release 7.1

27

27

0

308

99%

0

0

1

0

Release 7.1 Regression

 

 

 

485

99%

0

0

0

0

Feedback

Copyright © 2021 UST Global. All Rights Reserved.