NLP’s Legal Limit
Natural Language Processing (NLP) is routinely hailed as one of the most promising technological fields. Once NLP innovators perfect software that can intelligently read documents, the potential is limitless for the automation of our society. From the DMV to Goldman Sachs, this technology offers great promise, but currently there are still significant limitations to the technology and its current potential application.
One must first understand the nature of the technology. NLP is a delicate instrument that must be precisely directed. When creating NLP software, the application is extremely relevant to the potential outcome. So says NLP expert and former Microsoft & Googler Dima Korolev, “NLQ or Natural Language Query is doable in high performance, but NLP is too close to strong Ai and will make mistakes in a high performance setting.”
Strong Ai is a field of Ai that is trying to make the machine’s intellectual functions as strong as a human’s intellect. However, this field is still in an exploratory stage as most Ai is still quite far off from anything that resembles the cognitive functions of a human.
Fields that are sensitive to perturbations such as the legal field, where NLP has been extremely hyped, will have a tough time applying the technology. This is particularly relevant to one of the most document-centric fields out there: law.
Corporate lawyers focus on the details of lengthy, complex documents. Mergers and acquisition agreements, for example, are written by attorneys for parties for parties engaged in an adversarial dance with one another. So rather than a standard template that is easy for a simple NLP system to understand, the AI will need to pull together a clause in the merger agreement with a defined term 50 pages away, and then adjust that according to an amendment filed later in a separate document.
According to Logan Beirne, professor at Yale Law School and CEO of legal technology company Matterhorn Transactions, “In the law, details and accuracy are everything.” Matterhorn provides comprehensive databases, analyses, and reports to thousands of law firms across the US, UK, and Canada, allowing lawyers to compare the market terms of complex transactions. “Attorneys rely on us to draft complicated deals and advise clients on what is market for terms, so we need 100% accuracy - something NLP hasn’t been able to achieve. Yet.”
Beirne has largely relied on his own team of attorneys to digest the deals and accurately enter the granular information into the company’s databases. “The varied and complicated nature of legal documents has required a human touch. However, after years of our attorneys analyzing over 50,000 documents, we have a unique treasure trove of data which we are currently using to train AI tools.” Beirne is optimistic for the future of NLP, adding, “We are on the cusp of using NLP to achieve the impeccable accuracy that attorneys need.”
Unlike in law, NLP has been successful in finance, where a document can have a positive or negative interpretation based on far fewer factors. For example, consider the terms “increase” vs “decrease” and the correlation each word’s usage in a press release would have with the potential forward 90 day price return.
In the legal world there are many twists and challenges and NLP will take a few more years to master the craft.