Platform v6.4.0-Test Summary Report

Executive Summary

End to end functional and regression testing of Monitoring PWF , Invoice Extraction PWF and SmartOps platform changes is completed.

Period of Testing: 7th July 2020 to 13th July 2020

Test Execution Summary

Sl No.

Area

Execution Status

Open Defects if any

1

Invoice Extraction PWF

Completed

No Severity 1 and Severity 2

2

Monitoring PWF

Completed

No Severity 1 and Severity 2 excluding alert listing page

3

PWF and other SmartOps Platform Changes

Completed

No Severity 1 and Severity 2

Jira Defects Tested

FinOps

Jira ID

Test Status

Comments

SD -174

Verified – QA Passed

 

SDI-208

Verified – QA Passed

 

SDI-188

Verified – QA Passed

 

SDI-254

Verified – QA Passed

 

SDI-250

Verified – QA Passed

 

SDI-196

Verified – QA Passed

 

SDI-237

Verified – QA Passed

 

SDI-261

Verified – QA Passed

 

SDI-283

Verified – QA Passed

 

SDI-286

Verified – QA Passed

 

SDI-240

Verified – QA Passed

 

SDI-178

Verified – QA Passed

 

 

ITOps

Jira ID

Test Status

Comments

SDI-292

Verified – QA Passed

Known issues in Bug #30964

SDI-230

Verified – QA Passed

Known issues in Bug #30964

SDI-211

Verified – QA Passed

 

SDI-235

Verified – QA Passed

 

SDI-310

Verified – QA Passed

 

SD1-272

Not tested

End to end workflow for email alerts not available in test environment

SDI-218

Out of scoped for QA

Can be tested from the environment been raised. Not testable from QA

 

SDI-197

Not available for testing

 

VAPT Defects Tested

TFS ID

Test Status

Comments

29462

Closed

 

29463

Closed

 

29464

Closed

 

29465

Closed

 

Outstanding Defects in Release 6.4.3

Invoice Extraction PWF

Defect ID

Title

Severity

30999

 

6.4.3/QAPaaS/Invoice Extraction : Approved invoices are still shown in preview screen when user click on previous button

3 - Medium

31192

6.4.3/QAPaaS/Invoice Extraction Flickering issue in preview screen for extracted values

3 - Medium

 

Monitoring PWF

Defect ID

Title

Severity

31081

 

6.4.3/QAPaaS/Monitoring - Import topology - In import window - text 'browse files' is not horizontally aligned to the text before it - in 100% browser zoom

4 - Low

31364

6.4.3/QAPaaS/Monitoring - API for editing project description givens 200 OK for user 'monitoringuser', who has no admin privilege. Tested as part of vapt defect 29463

3 - Medium

31334

6.4.3/QAPaaS/Monitoring : Change in Filter Text Alignment in project Listing page

4 - Low

 

PWF and other SmartOps Platform Changes

Defect ID

Title

Severity

31395

6.4.3/QAPaaS/Clones: Alignment issue noticed in dashboard

4 - Low

Areas not tested

Monitoring PWF - Story 29748 is not tested due to blocked defects 30267, 30387

Defect ID

Title

Severity

30267

6.4.2/E2EPaaS/Monitoring - Alerts listing dashboard is showing a set of data for all projects which is not from QA Paas Environment. Alerts listing should be project specific.

1 - Critical

30387

 

6.4.2/E2EPaaS/Monitoring - As Monitoring User, he is not able to assign the ITSM ticket to self. Clicking on assign button against an alert just result in page hang. No action performed.

2 - High

Known Issues / Limitations

Module

Limitations

Monitoring PWF

  • Alert listing page is not listing Project specific Alerts.

  • Visualization for a new project is enabled from backend.

  • API/Email Channels are not integrated end to end. Currently up to log validation is done.

  • Metric beat integration has no visualization screen with consolidation component rules setup from backend.

  • User creating Scheduler field is not able to tag the corresponding automation story to be executed.

Invoice Extraction PWF

  • Accuracy Issue associated with tesseract model, hence documents whose rotation angle is 180 will be inverted

  • Extraction is not possible for FOI which are in dark background

  • Accuracy issue for field data and line item data

Test Result Reference

Invoice Extraction PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 6.4.3

7

6

1

95

98.95%

0

0

1

0

Regression Release 6.4.3

 

120

99.17%

0

0

1

0

Monitoring PWF 

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 6.4.3

3

3

0

33

100.00%

0

0

0

0

Regression Release 6.4.3

 

 

 

181

99.45%

0

0

1

2

Alert Listing page *

9

0

0

44

20.00%

0

8

16

0

SmartOps Platform (iHub and TUC)

# of User Stories

# of Test Cases

# of Open Defects (Critical/High/Med/Low)

Total

Fully
Passed

Partially
Passed

Total

Passed %

Critical

High

Medium

Low

Release 6.4.3 -iHub lite

2

2

0

34

100.00%

0

0

0

0

Regression Release 6.4.3 -iHub lite

 

 

 

5

100.00%

0

0

0

0

Release 6.4.3 -TUC

2

2

0

17

100.00%

0

0

0

1

Regression Release 6.4.3 - TUC

 

 

 

50

100.00%

0

0

0

0

* Alert Listing Page - No go for Stage

 

iHublite

https://ustglobal.sharepoint.com/teams/InnovationEngineering/Shared%20Documents/Forms/AllItems.aspx?viewid=f349a736%2D8a62%2D467f%2D8448%2D067be464bd59&id=%2Fteams%2FInnovationEngineering%2FShared%20Documents%2FTechnology%2FDesign%20Review%20Council%2FE2E%20Docs%2F6%2E4%2E3%2FTest%20Results%2FiHub%20Lite

Invoice Extraction PWF

https://ustglobal.sharepoint.com/teams/InnovationEngineering/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fteams%2FInnovationEngineering%2FShared%20Documents%2FTechnology%2FDesign%20Review%20Council%2FE2E%20Docs%2F6%2E4%2E3%2FTest%20Results%2FInvoice%20Extraction

Monitoring PWF

https://ustglobal.sharepoint.com/teams/InnovationEngineering/Shared%20Documents/Forms/AllItems.aspx?viewid=f349a736%2D8a62%2D467f%2D8448%2D067be464bd59&id=%2Fteams%2FInnovationEngineering%2FShared%20Documents%2FTechnology%2FDesign%20Review%20Council%2FE2E%20Docs%2F6%2E4%2E3%2FTest%20Results%2FMonitoring

Smartops Platform

https://ustglobal.sharepoint.com/teams/InnovationEngineering/Shared%20Documents/Forms/AllItems.aspx?viewid=f349a736%2D8a62%2D467f%2D8448%2D067be464bd59&id=%2Fteams%2FInnovationEngineering%2FShared%20Documents%2FTechnology%2FDesign%20Review%20Council%2FE2E%20Docs%2F6%2E4%2E3%2FTest%20Results%2FPWF%20and%20other%20SmartOps%20PlatformChanges

Feedback

Copyright © 2021 UST Global. All Rights Reserved.