SmartOps v6.3.2 Feature Clarification

Item

Currently Supported

Testing Details

Observation

Future Plans

Browser Compatibility

Google Chrome

Verified with Chrome version 76.0.3809.100 and above

All features many not work properly with other browsers or versions

No current plan to support other browsers

Item

Currently Supported

Testing Details

Observation

Future Plans

File size limit for Upload & Prediction

File size up to 1MB used in upload and predict works without performance lag

Verified with files < 1 MB and > 1 MB

Performance drop for documents with size greater than 1 MB

No current plan to improve this

Concurrent users

Supports up to 5 concurrent users without performance hit

Verified with 5 and more users. Current hardware: 16 core 64 GB RAM for Vision App + 8 Core 32 GB for dependencies

Performance drop when more than 5 users access the application concurrently

Not in current plan. Increasing the hardware can be one option

Unstructured document extraction accuracy

70% test accuracy reported, but lower accuracy on unseen documents

Verified using Annual Reports from US companies

Solution needs HITL, auto extraction solutions doesn’t give desired accuracy

HITL coming in 7.x to improve accuracy

File size limit for multi-page table extraction

Documents with size <= 1MB

Verified with files < 1 MB and > 1 MB. Current hardware: 16 core 64 GB RAM for Vision App + 8 Core 32 GB for dependencies

Multi page table extraction times out for documents with size greater than 1 MB

Not in current plan. Increasing the hardware can be one option

Preview for masked documents

Test page without preview

Verified test page by adjusting masking levels

No preview option available in the Test page to verify masked image

Not in current plan

Lambda & Throttler functions in Richly formatted document extraction

Provision to upload lambda and throttler functions in a file

Verified by uploading lambda and throttle functions via UI

Need pre-requisite knowledge on Fonduer framework and underlying data model

Not in current plan

Item

Currently Supported

Testing Details

Observation

Future Plans

Segregation Accuracy

40% accuracy in detecting cheques with handwritten characters

Verified with machine printed & handwritten cheques, blank and other type of documents

Accuracy issues with Cheque classification – Handwritten, printed, blanks and non-cheques

Improve with 3PP (HS) Integration

Identification of non-cheque documents

Non-cheque documents will be considered for segregation

Verified with cheque and non-cheque documents

Non-cheque documents are not classified

No current plan on this area

Handwritten data extraction

Not supported at present

NA

Handwritten data extraction from cheques are not supported

Support handwritten data extraction from cheques using 3PP (HS) integration

Item

Currently Supported

Testing Details

Observation

Future Plans

Field Extraction Accuracy Issues

Minimum resolution quality should be 300 DPI, Minimum dimension has to be 500 x 400 pixels, Non-blurred documents, document tilt not more than 5 degrees

Verified with machine printed and handwritten cheques

Field extraction accuracy issues for license & PAN documents due to insufficient training data

No current plans on this area

Supported Document types

Kerala drivers license in card format & PAN card

Verified with mentioned document types and others

Document types other than mentioned are not supported

No current plans on this area

KYC Application status updates

Submitted application shows as ‘Review Pending’ for 5-10 seconds in the listing

Verified submitting KYC application forms and validating status in listing page

Slight delay in overall status update when reviewer submits changes

No current plans on this area

Tilt & Skew correction of images

Document images which are not tilted or skewed

Verified with images tilted / skewed and proper images

Tilt & Skew correction of images are not available in this release

No current plans on this area

Handwritten data extraction from documents

Handwritten data extraction is available for application form available in the PWF

Verified with handwritten application forms and other types of documents

Handwritten data extraction will not work for document types other than specific application form available with PWF

Support using 3PP (HS) integration

Supported Document formats

Supports PDF, PNG and JPG formats

Verified with PDF, PNG and JPG files

Formats other than mentioned are not supported (e.g.: TIFF)

No current plans on this area

Item

Currently Supported

Testing Details

Observation

Future Plans

Support Engineer UI recommendations

ServiceNow ticket raised should have “Diagnosis” field set as Ticket Category

Verified with ServiceNow tickets with and without “Diagnosis” field as Ticket Category

Support UI will not show recommendations if Ticket Category is not set as “Diagnosis” field

No current plans on this area

Customizing Recommendations model

Works with pre-defined recommendation model

Verified recommendations for a custom ticket dump

Retraining of ML models to improve recommendations is currently not available

R&D in progress

Intent Retraining

Manual retraining of Conversation Designer project after changes to workflow mapping

Verified by making mapping changes

Conversation Designer project should be re-trained to reflect the changes in workflow mapping

R&D in progress

Similarity Scoring

Bigger ticket dumps give better similarity scores

Verified with big and small custom ticket dumps

Similarity model score varies for small sized Ticket dumps

R&D in progress

Periodic improvements to Resolution recommendations

Provides recommendations from pre-defined recommendations model

Verified by updating ticket dump after project creation

Periodic ticket dump updates are not affecting the resolution recommendations

R&D in progress

Item

Currently Supported

Testing Details

Observation

Future Plans

Project Visualisation

Visualisation needs to be configured from backend after project creation

Verified by creating monitoring project and

Visualization for a new project is enabled from backend

As part of 6.4.1, a new dashboard is planned

iHub Channel Integration

Queue is the only channel supported end to end

Verified by configuring iHub via Monitoring project screen

API/Email/HTTP Channels are not integrated end to end

Planned for 7.0

Visualization for Metric Beat Integration

Rule consolidation is done from backend and anomalies are verified from log files

Verified by enabling from Monitoring project

Metric beat integration has no visualization and rule consolidation screen

Planned for 7.0

Topology Correlation

Identifies root cause based on topology and mention it in the ticket details if already a ticket is open for parent

Verified by defining topology and simulating ticket creation for parent and child nodes

Correlation fails intermittently due to GRAKN connectivity issue

No current plan on this

Item

Currently Supported

Testing Details

Observation

Future Plans

Data extraction accuracy

Field level accuracy below

Verified by uploading MSAs, SOWs, Addendums from different vendors

Accuracy issues in MSA/SOW/Addendum data extraction

Moving towards Vespa based solution in 7.x

Volume Rebates Recalculate

Volume rebates data extracted from MSA and applied in the total contract value to get rebate value

Verified by uploading MSAs with rebate details and SOWs linked to MSA

User will not be able to change wrong linking and recalculate rebate value

Not in current plan

UI for Addendum Details

Addendum details are used in document linkage and total contract value calculation

Verified by uploading MSAs, SOWs, Addendums from different vendors

Addendum details are not available in details page

Moving towards Vespa based solution in 7.x

Performance

Takes 30 minutes average for 10 contract documents of 10-20 pages each

Verified by uploading MSAs, SOWs, Addendums from different vendors

Delay in processing contract documents & extract data

Not in scope, since it’s an offline solution

Volume rebate table extraction

Supports rebates tables with following 4 columns - Tier, Minimum, Maximum and % of Level

Verified by uploading MSAs with rebate details represented in different forms

Rebate tables with difference in column titles or number of columns and rebate information represented in non-tabular format are not extracted

Moving towards Vespa based solution in 7.x

First Party

Second Party

Effective Date

End Date

Jurisdiction

Rebate Available

Payment Terms

Warranties

Termination

Indemnification

100

100

94

21

81

43

38

47

25

15

First Party

Second Party

Effective Date

MSA Date

End Date

Duration

Contract Value

100

91

91

91

50

50

83

First Party

Second Party

Effective Date

MSA Date

SOW Date

End Date

Purpose

Original SOW End Date

Revised SOW End Date

Contract Value

81

54

38

57

42

50

50

67

67

33

Item

Currently Supported

Testing Details

Observation

Future Plans

Data extraction accuracy for Annual Reports

Field level accuracy below

Verified by uploading annual reports from multiple companies

Field accuracy needs to be improved for Annual Reports extraction

HILT coming in 7.x to improve accuracy

Field selection change

Supported fields are pre-selected for extraction

Verified by creating DU projects for annual reports and invoices

Fields selected for extraction cannot be changed

Not in current plan

Pre-processing for annual reports

Annual report extraction done without applying pre-processors

Verified by creating DU project for annual report extraction and applying pre-processors

Pre-processor configurations are not applicable for Annual reports

Not in current plan

Orientation Correction

Orientation correction is supported only for non-searchable PDFs

Verified by uploading PDFs rotated clockwise, anti-clockwise and upside down

Orientation correction is not applied on searchable PDFs

Not in current plan

Company Name

Report Year

CEO

Total Assets

Total Liabilities

73

93

86

66

20

Item

Currently Supported

Testing Details

Observation

Future Plans

Pending documents selection for assignment

Supervisor will have to select individual invoices for assignment

Verified assignment from Invoice PWF dashboard

Not possible to select all ‘Waiting For Approval’ documents by Supervisor user

This will be available in 6.4.1

Fields in exports

Data extracted can be exported by downloading excel file or through report generation

Data export verification

Few fields are missing in excels downloaded and created via report generation

This will be available in 6.4.2

Line item preview

User can view line items in document preview page

Verified by navigating to document preview page for an invoice

Line item window is not movable in Preview Page

Not in current plan

Confidence score manually entered invoices

User can correct a mis-split invoice by adding as a new invoice

Verified by manually entering a new invoice

Confidence Score is displayed for newly added (via ‘Add Invoice’) invoices which is not relevant

Confidence Score is moved to Accuracy scores in HILT

Reassign Invoices

Supervisor can assign an invoice only once to a clerk

Verified by assigning invoices via Supervisor dashboard

Reassign functionality is not working for assigned invoices

This will be available in 6.4.1

Fields in Document Preview page

Only limited fields are available in preview

Verified by navigating to document preview page for an invoice

All fields selected in project configuration are not getting displayed in Preview Page

This will be fixed in 6.4.0

Invoice Extraction Time

Extraction time for a single page invoice with 1 line item is 142 seconds

Verified by uploading a single page invoice and calculating time based on UI status

More than 2 minutes for a single page can slow down multi page invoice extraction in real scenario

Not in current plan. Higher speeds would need GPU

Field data accuracy

Field level accuracy below

Verified by validating against Tolas and PP invoices for Navistar

Field wise extraction accuracy is not good for all 14 fields

HILT coming in 7.x to improve accuracy

Line item data accuracy

Line item extraction works for tables with strong closed borders containing in a single page

Verified by validating against invoices containing line items in tabular and non-tabular format

Line item extraction will not work for invoices with partially closed bordered tables, multi-page tables and non-tabular representations

In R&D

Extraction time calculation

Extraction time in analytics is total time in DU - which includes queue wait time and execution time

Verified by validating extraction time in Admin Analytics section

Extraction time for an invoice will not be consistent; as the queue time can vary

No current plan for this

Invoice Date

Invoice Number

Scan Number

Ship To Code

Sub Total

Total Amount

Vendor Code

Customer PO Number

Extended Cost

Part Number

Quantity

Unit Rate

88

53

100

100

80

80

91

43

28

0

37

0

Item

Currently Supported

Testing Details

Observation

Future Plans

Supported file size of PWF project for import

Import functionality works with a file size limit of 1 MB

Verified by importing PWF projects

Import functionality will not work with files above 1 MB

Not in current plan

Import Messages

Message shows source file details

Verified by importing PWF projects

Import pop up window message to be corrected to reflect target file details

Planned for 6.4.1