Get ready for documents that will actively help you

Coming to your business soon, today’s practical AI (Artificial Intelligence) is ushering in a new era of 'sentient documents'.

While a significant operational advancement, the core concept is straightforward— allowing us to engage with connected information sources— yesteryear’s ‘documents’— as if they possess consciousness, and can comprehend and respond dynamically.

So, expect AI that enables all types of documents—contracts, orders, policies, even handwritten notes—to assist us effortlessly, like an eager young apprentice. You could interact with them naturally, asking ‘What are you?’ or ‘What's your subject?’ and the document itself would comprehend and respond, almost as if it were a conscious entity. Rather than using code, you'll interact through natural language. And note that while I've described this concept in English, sentient documents will transcend language barriers—even if a document is written in a language you don’t understand, you'll be able to interact with it naturally in your home language.

To grasp the potential, imagine being faced with a very complex legal contract that could even be in a foreign language. Imagine being able to ask ‘what is this contract?’, ‘who are the contracting parties?’, ‘what is the expiry date?’ or ‘what are the penalty clauses for breach?’ and receiving a full answer in your own language. And this same principle can apply to orders, invoices, job applications, change of address forms, customer onboarding forms – and just about any form of document. It can even go into domain specialisms, such as legal, pharmaceutical, medical or other areas, with domain-specific questions and answers.

This new generation of smart documents represents the next step in the evolution of document management. Sentient documents represent the evolution of both what we’ve been doing for decades in ECM (enterprise content management) and the next step in content services platforms, where organisations store, manage, search, or archive documents and other useful content.

Defining sentience in business information networks

Document sentience emerges from automating content processes like auto-responding to invoices in finance or using business rules for employee onboarding over in HR. However, the game-changer here involves harnessing diverse AI capabilities beyond just conversational AI. It requires combining natural language understanding with generative AI, large language models, and machine learning for pattern recognition.

The magic is in the combination of each of these intelligent technologies, which together give us a framework for a highly functional new form of content platform. Not only will it provide traditional enterprise content management features, but it will also (at last) truly understand the content it is being tasked to look after. And at the same, it will offer an easy and familiar kind of conversational interface we are accustomed to with virtual assistants like Siri or Alexa, but tailored to intelligent document and content engagement rather than consumer tasks.

A complex back end—and multiple forms of ‘dark’ content

And it’s arriving in the nick of time. The business world has gone from a largely monolithic approach to ERP and ecommerce to best-of-breed. So, rather than everything running on an all-in-one ERP suite, organisations now piece together different best-in-class solutions—SAP for ERP, Salesforce for CRM, ServiceNow for service automation, and a mix of Google Workspace and Microsoft Office productivity for other parts of the organisation.

If a back-end architecture utilising specialised solutions from multiple providers is the new norm, the problem is that each of those best-of-breed solutions provides specialised capabilities—which is why you have them—but they're also from different tech ecosystems. As a result, we’re at risk of recreating information and process silos, where critical documents, data and processes become trapped within their respective walled gardens.

To save us from this inefficiency organisations need an intelligent content automation framework. This allows an organisation’s set of sentient documents to be able to access everything across those ecosystem boundaries. As a customer told me last week, ‘we needed the glue to connect all of these best-of-breed platforms together’. Moreover, since the pandemic ushered in remote collaboration tools like Teams and Zoom, enterprises are grappling with massive amounts of ‘dark’ content.

Dark content in this context is a catch-all term for valuable business information trapped within recorded meetings, chat logs, and presentations that currently sits there passively and underutilised. Imagine all the organisational wisdom locked away in there! An intelligent content layer could illuminate this dark data, making the insights buried within those digital interactions accessible and actionable.

Sentient document technology will ingest and make this dark data useful by incentivising corporations to meticulously record and cross-reference information, to construct comprehensive organisational memory banks. AI needs to extract descriptive metadata whenever content gets created or ingested, forming a centralised 'metadata kernel' that powers AI bots and analytics capabilities.

This metadata kernel will fuel the sentient document revolution that organisations seek. Sentience will also make housekeeping and version control simpler, which any information manager knows is a huge part of their work—all versions will be retained, but the specific version needed can be easily retrieved through a simple request, like ‘show me the latest version’ or ‘display the July draft.’ Sentience at the stored document level will instantly provide concise summaries, highlighting the most salient points across document versions.

Will our future document managers look after us?

This level of convenience and assistance does raise the danger with sentient documents: could they end up being so powerful and ubiquitous that they run our lives? I’m reminded of a famous Isaac Asimov SF story where computers are so dominant that rediscovering the joy of doing arithmetic by hand is viewed as liberating.

I think a potential overreliance on AI systems is a valid concern. However, in my view AI is not a substitute for humans, but a highly useful tool, and its true potential lies in magnifying the power of the knowledge worker by 10 to 100 times.

The history of IT tells us that we will always need human oversight at the highest level. Allowing AI to make unqualified decisions blind is not going to be a good way forward, and regulations will need to mandate human involvement at critical junctures, particularly in areas like healthcare.

Time for the ECM market to get sentient

It is our belief that sentient documents are coming, and will be a fantastic way to help people understand and summarise business information. They will allow knowledge workers the world over to quickly grasp key points and make truly data-driven decisions without getting mired in excessive details, saving time, money and brainpower.

They will also provide a much-needed solution to manage all the complexity, untapped dark content, and an endless firehose of unstructured data streaming into businesses today. And we wouldn’t be able to afford the army of people we’d need to extract insights from vast data repositories otherwise.

So, get acquainted with sentient document thinking. This is way more practical than generalised AI hype, and paves the way for intelligent document-processing prowess for you and your users.

Dr John Bates is a former Cambridge Don who became addicted to starting and growing businesses, often linked to his primary research field of AI, smart algorithms, and intelligent analytics. Today he is the CEO of SER, which specialises in intelligent content automation (ICA)—the convergence of content management (capturing, storing, searching, archiving, and managing enterprise content), business process automation, and AI-powered content understanding.

‍