Looking at the future of Intelligent Document Processing
How did document processing become intelligent and what does it mean for your organisation now and in the future?
Document processing has come a long way over the years - from being essentially a production line task that relies on manual data entry via keyboard, to a process that harnesses technologies like AI and machine learning to extract actionable insights from the data within documents.
But despite the progress, organisations still face real challenges when it comes to accessing and extracting data from documents, connecting to processes, and being able to make decisions in a timely manner. Many organisations are exploring intelligent document processing (IDP) as a way to meet these challenges.
According to Gartner, the intelligent document processing market will hit US$4.8 billion in 2022. It's part of the much larger intelligent process automation market IDC estimates at $17.3 billion. So how exactly did document processing become intelligent? And how do the latest technologies impact how organisations can process documents to boost business efficiencies and outcomes?
WHAT IS DOCUMENT PROCESSING?
In brief, document processing is a field of research and a set of processes aimed at making an analogue document digital. It can be manually intensive. For example, faced with a need to process millions of visa and citizenship applications in 2007, a U.S. immigration agency turned to an army of about 1,000 contract workers to handle mailroom and data entry work.
More than merely photographing or scanning a document, document processing aims to make it digitally intelligible. The ability to extract and classify information contained within paper and digital documents empowers organisations to manage and perform a wide range of business operations, such as purchasing, ordering, and on-boarding.
Document processing is especially critical in delivering goods to consumers. When a customer completes an order online, the manufacturing company can then coordinate the process of acquiring materials, creating the product, and delivering it. Each step entails the use of documents between departments and their teams.
This sequence of events depends on a high degree of accuracy and reliability. If any small piece of data from the documents isn't processed correctly, the result can be a domino effect of failures, from losing an invoice or payment to losing an order altogether. Ultimately, an inefficient and inaccurate operation can affect all aspects of a business.
WHAT IS IDP?
Over the years, document processing technology has added more intelligent capabilities. ISG, a leading global research and advisory firm, defines intelligent document processing (IDP) as software that uses AI technologies such as natural language processing (NLP), machine learning (ML), computer vision, and deep learning to filter and analyse large volumes of unstructured data from multiple formats for further processing, storage, and use in other applications.
IDP has its roots in the first optical character recognition (OCR) solutions, which converted character images into machine-encoded text. While OCR-templated approaches can handle highly structured data, they cannot handle unstructured data, which hides in "documents" like emails, PDFs, invoices, purchase orders, contracts, IDs, corporate tax filings, and more. This is significant because at least 80% of an organisation's data is unstructured.
IDP for years remained a siloed mailroom or business process optimisation (BPO) function, isolated from critical engagements with customers, suppliers, and employees. Today, that is changing. IDP encompasses the processes in which content is used as part of customer and employee experiences. It begins with a re-thinking of these processes, then targets modern AI-based document processing "skills" to make the content available and actionable when and where it's needed.
WHY IS THIS IMPORTANT?
Across all industries, businesses are facing pressure to do more, faster, and with fewer skilled resources. At the same time, more organisations are focusing on improving experiences for customers and employees as a key to improving revenue, margins, and retention.
IDP can deliver significant benefits on both counts. By applying document skills in a way that closely supports the way humans understand and manage content, IDP saves time and money while also reducing the risk of costly errors. From an experience perspective, employees are freed from the tedious tasks of reading and manually processing documents, while customers benefit from greater efficiency in document-based processes and decisions.
For example, one leading beverage company's Accounts Payable department was struggling with manual, error-prone processes for keying invoice and credit data into SAP. The team needed an automated capture solution - something that could support multiple languages and currencies, as well as large-volume batch processing and complex data fields. Their new IDP solution scanned paper invoices, identified them with the correct data, and recognised them. From there, the digital documents and data were sent to human in the loop verification stations, and then forwarded for approval. Finally, the data is exported in an XML file to the company's ERP solution. In its first three months, the IDP technology processed 2,000 batches, over 21,000 documents, and nearly 40,000 pages - in a mix of five languages - without issues.
IDP can also augment robotic process automation (RPA) for tasks such as transferring data from an invoice into an ERP system. However, for this to work, bots need to be smart enough to "read," "understand," and "make decisions" about the data they are processing. RPA on its own cannot understand unstructured documents. It needs AI to enable bots to have "content intelligence," so that they can do things like read and categorise documents, route documents, extract and validate data from documents, and other tasks related to understanding and processing unstructured content.
Simple OCR tools that capture data simply don't cut it anymore. Organisations need solutions that transform the content, context, relationships, and entities buried in documents into meaningful structured data, enabling vision, understanding, and insight.
WHAT NEXT FOR IDP?
The increase in digitisation is spurring companies to use IDP solutions to simplify and accelerate their document processing procedures. Intelligent automation platforms like RPA, BPM and CRM can, used correctly, speed processes and ready an organisation to add more experiential opportunities to engage with customers such as interactive mobile apps, cognitive virtual assistants that combine voice and conversational AI, and chatbots.
IDP solutions need to keep pace with modern enterprises that have transitioned to low-code/no-code platforms and the cloud, especially in the age of hybrid workforces. Modern IDP solutions offer powerful features such as "skill designers" that make it easy for citizen developers to design, train, and publish document skills for all types of structured and unstructured documents with no coding required. They can turn documents into actionable data with pre-trained, ready-to-use skills for a variety of documents, saving development time and gaining quicker ROI.
Modern IDP solutions should also easily integrate out of the box with intelligent automation platforms while making skills auto-discoverable from RPA, BPM, chatbots, mobile apps, and more. And they should improve accuracy with continuous learning, making business processes more efficient over time and requiring less human intervention.
THE FUTURE IS NOW
As the future of work fast becomes the present, IDP is playing a significant role in the digital transformation of a broad variety of industries - whether re-thinking document-centric processes for the financial services industry, automating shipping processes in transportation and logistics, impacting the automation of accounts payable, loan and insurance processing, and more.
The combination of low-code/no-code platforms, the cloud, and a citizen-developer-friendly user experience is revolutionising the way AI-enabled skills are created, trained, and published, thereby speeding learning curves and making it easy to apply IDP technology across many business functions and automation systems.
Bill Galusha, Vice President, Product Marketing for AI Document Processing at ABBYY
Please feel free to contact us at
