Contents
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Browser Compatibility |
Google Chrome |
Verified with Chrome version 76.0.3809.100 and above |
All features many not work properly with other browsers or versions |
No current plan to support other browsers |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
File size limit for Upload & Prediction |
File size up to 1MB used in upload and predict works without performance lag |
Verified with files < 1 MB and > 1 MB |
Performance drop for documents with size greater than 1 MB |
No current plan to improve this |
|
Concurrent users |
Supports up to 5 concurrent users without performance hit |
Verified with 5 and more users. Current hardware: 16 core 64 GB RAM for Vision App + 8 Core 32 GB for dependencies |
Performance drop when more than 5 users access the application concurrently |
Not in current plan. Increasing the hardware can be one option |
|
Unstructured document extraction accuracy |
70% test accuracy reported, but lower accuracy on unseen documents |
Verified using Annual Reports from US companies |
Solution needs HITL, auto extraction solutions doesn’t give desired accuracy |
HITL coming in 7.x to improve accuracy |
|
File size limit for multi-page table extraction |
Documents with size <= 1MB |
Verified with files < 1 MB and > 1 MB. Current hardware: 16 core 64 GB RAM for Vision App + 8 Core 32 GB for dependencies |
Multi page table extraction times out for documents with size greater than 1 MB |
Not in current plan. Increasing the hardware can be one option |
|
Preview for masked documents |
Test page without preview |
Verified test page by adjusting masking levels |
No preview option available in the Test page to verify masked image |
Not in current plan |
|
Lambda & Throttler functions in Richly formatted document extraction |
Provision to upload lambda and throttler functions in a file |
Verified by uploading lambda and throttle functions via UI |
Need pre-requisite knowledge on Fonduer framework and underlying data model |
Not in current plan |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Segregation Accuracy |
40% accuracy in detecting cheques with handwritten characters |
Verified with machine printed & handwritten cheques, blank and other type of documents |
Accuracy issues with Cheque classification – Handwritten, printed, blanks and non-cheques |
Improve with 3PP (HS) Integration |
|
Identification of non-cheque documents |
Non-cheque documents will be considered for segregation |
Verified with cheque and non-cheque documents |
Non-cheque documents are not classified |
No current plan on this area |
|
Handwritten data extraction |
Not supported at present |
NA |
Handwritten data extraction from cheques are not supported |
Support handwritten data extraction from cheques using 3PP (HS) integration |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Field Extraction Accuracy Issues |
Minimum resolution quality should be 300 DPI, Minimum dimension has to be 500 x 400 pixels, Non-blurred documents, document tilt not more than 5 degrees |
Verified with machine printed and handwritten cheques |
Field extraction accuracy issues for license & PAN documents due to insufficient training data |
No current plans on this area |
|
Supported Document types |
Kerala drivers license in card format & PAN card |
Verified with mentioned document types and others |
Document types other than mentioned are not supported |
No current plans on this area |
|
KYC Application status updates |
Submitted application shows as ‘Review Pending’ for 5-10 seconds in the listing |
Verified submitting KYC application forms and validating status in listing page |
Slight delay in overall status update when reviewer submits changes |
No current plans on this area |
|
Tilt & Skew correction of images |
Document images which are not tilted or skewed |
Verified with images tilted / skewed and proper images |
Tilt & Skew correction of images are not available in this release |
No current plans on this area |
|
Handwritten data extraction from documents |
Handwritten data extraction is available for application form available in the PWF |
Verified with handwritten application forms and other types of documents |
Handwritten data extraction will not work for document types other than specific application form available with PWF |
Support using 3PP (HS) integration |
|
Supported Document formats |
Supports PDF, PNG and JPG formats |
Verified with PDF, PNG and JPG files |
Formats other than mentioned are not supported (e.g.: TIFF) |
No current plans on this area |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Support Engineer UI recommendations |
ServiceNow ticket raised should have “Diagnosis” field set as Ticket Category |
Verified with ServiceNow tickets with and without “Diagnosis” field as Ticket Category |
Support UI will not show recommendations if Ticket Category is not set as “Diagnosis” field |
No current plans on this area |
|
Customizing Recommendations model |
Works with pre-defined recommendation model |
Verified recommendations for a custom ticket dump |
Retraining of ML models to improve recommendations is currently not available |
R&D in progress |
|
Intent Retraining |
Manual retraining of Conversation Designer project after changes to workflow mapping |
Verified by making mapping changes |
Conversation Designer project should be re-trained to reflect the changes in workflow mapping |
R&D in progress |
|
Similarity Scoring |
Bigger ticket dumps give better similarity scores |
Verified with big and small custom ticket dumps |
Similarity model score varies for small sized Ticket dumps |
R&D in progress |
|
Periodic improvements to Resolution recommendations |
Provides recommendations from pre-defined recommendations model |
Verified by updating ticket dump after project creation |
Periodic ticket dump updates are not affecting the resolution recommendations |
R&D in progress |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Project Visualisation |
Visualisation needs to be configured from backend after project creation |
Verified by creating monitoring project |
Visualization for a new project is enabled from backend |
Planned for 7.0 |
|
iHub Channel Integration |
Queue & HTTP are the only channels supported end to end |
Verified by configuring iHub via Monitoring project screen |
API/Email Channels are not integrated end to end
|
No current plan on this |
|
Visualization for Metric Beat Integration |
Rule consolidation is done from backend and anomalies are verified from log files |
Verified by enabling from Monitoring project |
Metric beat integration has no visualization and rule consolidation screen |
No current plan on this |
|
Scheduling for Ticket Status update |
Supports 6-digit cron expression to schedule pre-configured automation story |
Verified by creating a new monitoring PWF project |
Cannot specify an automation story to get executed for ticket status updates |
No current plan on this |
|
Alert listing |
Alerts listing page shows data |
Verified by creating a new monitoring PWF project |
Alert listing page is not listing Project specific Alerts |
|
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Data extraction accuracy |
Field level accuracy below |
Verified by uploading MSAs, SOWs, Addendums from different vendors |
Accuracy issues in MSA/SOW/Addendum data extraction |
Moving towards Vespa based solution in 7.x |
|
Volume Rebates Recalculate |
Volume rebates data extracted from MSA and applied in the total contract value to get rebate value |
Verified by uploading MSAs with rebate details and SOWs linked to MSA |
User will not be able to change wrong linking and recalculate rebate value |
Not in current plan |
|
UI for Addendum Details |
Addendum details are used in document linkage and total contract value calculation |
Verified by uploading MSAs, SOWs, Addendums from different vendors |
Addendum details are not available in details page |
Moving towards Vespa based solution in 7.x |
|
Performance |
Takes 30 minutes average for 10 contract documents of 10-20 pages each |
Verified by uploading MSAs, SOWs, Addendums from different vendors |
Delay in processing contract documents & extract data |
Not in scope, since it’s an offline solution |
|
Volume rebate table extraction |
Supports rebates tables with following 4 columns - Tier, Minimum, Maximum and % of Level |
Verified by uploading MSAs with rebate details represented in different forms |
Rebate tables with difference in column titles or number of columns and rebate information represented in non-tabular format are not extracted |
Moving towards Vespa based solution in 7.x |
|
First Party |
Second Party |
Effective Date |
End Date |
Jurisdiction |
Rebate Available |
Payment Terms |
Warranties |
Termination |
Indemnification |
|
100 |
100 |
94 |
21 |
81 |
43 |
38 |
47 |
25 |
15 |
|
First Party |
Second Party |
Effective Date |
MSA Date |
End Date |
Duration |
Contract Value |
|
100 |
91 |
91 |
91 |
50 |
50 |
83 |
|
First Party |
Second Party |
Effective Date |
MSA Date |
SOW Date |
End Date |
Purpose |
Original SOW End Date |
Revised SOW End Date |
Contract Value |
|
81 |
54 |
38 |
57 |
42 |
50 |
50 |
67 |
67 |
33 |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Data extraction accuracy for Annual Reports |
Field level accuracy below |
Verified by uploading annual reports from multiple companies |
Field accuracy needs to be improved for Annual Reports extraction |
EITL coming in 7.x to improve accuracy |
|
Field selection change |
Supported fields are pre-selected for extraction |
Verified by creating DU projects for annual reports and invoices |
Fields selected for extraction cannot be changed |
Not in current plan |
|
Pre-processing for annual reports |
Annual report extraction done without applying pre-processors |
Verified by creating DU project for annual report extraction and applying pre-processors |
Pre-processor configurations are not applicable for Annual reports |
Not in current plan |
|
Orientation Correction |
Orientation correction is supported only for non-searchable PDFs |
Verified by uploading PDFs rotated clockwise, anti-clockwise and upside down |
Orientation correction is not applied on searchable PDFs |
Not in current plan |
|
Company Name |
Report Year |
CEO |
Total Assets |
Total Liabilities |
|
73 |
93 |
86 |
66 |
20 |
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Line item preview |
User can view line items in document preview page |
Verified by navigating to document preview page for an invoice |
Line item window is not movable in Preview Page |
Not in current plan |
|
Invoice Extraction Time |
Extraction time for a single page invoice is 4 minutes |
Verified by uploading a single page invoice and calculating time based on UI status |
More than 2 minutes for a single page can slow down multi page invoice extraction in real scenario |
Not in current plan. Higher speeds would need GPU |
|
Field data accuracy |
Field level accuracy below |
Verified by validating against Tolas and PP invoices for Navistar |
Field wise extraction accuracy is not good for all 14 fields |
EITL coming in 7.x to improve accuracy |
|
Line item data accuracy |
Line item extraction works for tables with strong closed borders containing in a single page |
Verified by validating against invoices containing line items in tabular and non-tabular format |
Line item extraction will not work for invoices with partially closed bordered tables, multi-page tables and non-tabular representations |
In R&D |
|
Accuracy for fields with dark background |
Fields with light or no background |
Verified by validating against Tolas and PP invoices for Navistar |
Extraction is not possible for FOI data in dark background
|
Not in current plan |
|
Extraction time calculation |
Extraction time in analytics is total time in DU - which includes queue wait time and execution time |
Verified by validating extraction time in Admin Analytics section |
Extraction time for an invoice will not be consistent; as the queue time can vary |
No current plan for this |
|
Orientation correction accuracy |
Landscape oriented images will be converted to portrait mode |
Verified by validating with a landscape-oriented invoice |
2 out of 75 invoice pages had wrong orientation correction applied |
No current plan for this |
|
Invoice Date |
Invoice Number |
Scan Number |
Ship To Code |
Sub Total |
Total Amount |
Vendor Code |
PO Number |
Freight Amount |
Company Address |
Company Name |
|
77 |
68 |
98 |
75 |
75 |
86 |
100 |
87 |
82 |
61 |
56 |
* Field accuracy calculated excluding files with OCR quality issues
|
Part Number |
Quantity |
Unit Rate |
Extended Cost |
|
50 |
50 |
46 |
76 |
* Field accuracy calculated excluding files with OCR quality issues
|
Item |
Currently Supported |
Testing Details |
Observation |
Future Plans |
|
Supported file size of PWF project for import |
Import functionality works with a file size limit of 1 MB |
Verified by importing PWF projects |
Import functionality will not work with files above 1 MB |
Not in current plan |