A new legal aid. Advisor and mentor for young attorneys. IBM Argumentor Powered by the Debater Technology Boaz Carmeli, IBM Research – Haifa [email protected]
A new legal aid. Advisor and mentor for young attorneys.
IBM ArgumentorPowered by the Debater Technology
Boaz Carmeli, IBM Research – Haifa
Agenda – things that I am intending to cover
� Short introduction and background: the Debater grand challenge
� Our technology focus: Pro-con analysis via Machine learning (deep learning) and NLP
� The Argumentor application: Pro-con analysis at the legal domain
� Argumentor: Short demo
� The university relations aspects: legal experts for requirements definition and legal text annotation
� Sumary and questions….
Background – The debater grand challenge
� The Debater Grand Challenge aims on developing technology that assist
humans to debate and reason...
� The Debater vision is that of an intelligent system able to take raw information and
digest and reason on that information, to understand the context, and to construct
arguments pro and con any subject…
� The Debater uses complex analytic pipeline a.k.a Argument Construction Engine (ACE)
3
Pro/Con Analysis� Goal:
– given two related sentences e.g.,
� A claim and an evidence or,
� A thesis and a fact
– determine for each pair:
� Whether the second supports the first (PRO) or contests it (CON)
� Example
– Claim:
� An evidence obtained from a search without warrant cannot be used in court
– Evidence:
� “The Fourth Amendment provides that "the right of the people to be secure in their
persons, houses, papers, and effects, against unreasonable searches and seizures,
shall not be violated.” (PRO)
� “It is frequently argued that in dealing with the rapidly unfolding and often dangerous
situations on city streets the police are in need of an escalating set of flexible responses,
graduated in relation to the amount of information they possess.“ (CON)
Deep Learning for Pro-Con Analysis
� Deep learning is a promising machine learning subfield that gains huge momentum lately
– Beats state-of-the-art algorithms in areas such as computer vison, speech recognition
and natural language processing
Sentence one Sentence two
Deep learning 2 i.e., Recurrent
Neural Net (RNN)
Deep learning 1 i.e., Recurrent
Neural Net (RNN)
Deep Learning layer 3 i.e., Classification
Pro-con results
Argumentor’s Target Professional
� Young attorney, less than 5 years of experience
� Usually takes a role of a Junior associate in mid-large law firms
� Chosen based on a market research by Watson group:
6
http://ace1.haifa.ibm.com:8080/arguMentor/?tryme
Argumentor Demo
Web client
supremecourtdatabase.org Supreme court decisions 1937-1975 (FLITE)
Keyword andConcept
extraction
Annotation tool
Annota
tionArgument detection
Debater claim detection service
ActiveLearningvia Exemplar
Clustering
Annotation management
Alchemy Langage
NLClassifierArgumentclustering
Argumentclassification
Solution Architecture
8
IBMArgumentor
Cognitive Elements� Argument Construction
– Deep natural language processing - Argumentor processes legal cases for
detecting arguments, based on technology from the Debater grand challenge
� Natural language input
– Argumentor receives a short case brief as its input
� Interactivity
– Argumentor integrates in current workflow of the legal professional
� Processes email as input
� Allows human feedback at each step
� Learns argument classification based on several examples from the user
� Helps the user focus on what is of her interest by interactive highlighting
� Supervised Machine Learning
– For this hackathon, we annotated sample data of real legal cases to teach the
computer to classify and evaluate arguments
9
University Relations Aspects
� Won a “Legal Argumentation Structuring and Gamification” UR country project
� Gave two lectures at the “Institute for Legal Implications of Emerging Technologies”
program at Interdisciplinary Center, (IDC) Herzliya, Israel
� Instructing a group of 4 students on ‘Legal and AI’ project
– Legal csaes annotation guidelines
– User requirements
� Search collaboration with Robert-Jan Sips and the Netherlands academy
– Application for the IBM University Programs to fund a PhD fellowship for crowdsourcing
and nichsourcing research in the legal domain
– Investigating annotation platforms such as Crowdflower and Watson Knowledge Studio
(WKS)
Summary
� Debater provides leading technology for argument construction
� Argumentor is a cognitive application that assists young attorney in her legal
research tasks
– Based on argument construction pipeline, namely claim detection and pro-con
analysis
� Collaboration with law schools and university provides a jump-start into this
complex domain
Hackathon Team
Thanks to:
IBM Research -Haifa
•Machine Learning Technologies group
•Medical Imaging Analytics
Watson Emerging Products Design
Watson Implementations
12
Backup
Backoffice work
� Downloaded 7,500 US Supreme Court cases from 1937-1975
� Used ACELab and GATE to annotate legal data
– Annotated by our legal professional on board – a lawyer team member from Watson
Implementations
� Created three exemplary use cases
� Conducted search for relevant cases for all three cases
� Annotated arguments from Supreme Court cases
– Created guidelines for annotation of legal argument
� Trained NLClassifier based on winning and losing arguments
� Downloaded structured data of US Supreme court cases http://supremecourtdatabase.org/
� Preprocessed all 7,500 cases by running the Debater pipeline offline due to performance
considerations
14
IBM Argumentor Flow - Input
� Attorney pastes an email or brief
that describes his case
� Argumentor uses AlchemyAPI to
extract keywords and concepts
� Argumentor reranks and filters
results to keep only relevant legal
concepts
� Attorney can add and remove
keywords
15
IBM Argumentor Flow – Argument Construction
� Argumentor uses Debater’s ACE
to construct arguments
� Argumentor augments arguments
data with case data
� Attorney can look at the relevant
cases Argumentor found
� Attorney can focus on arguments
using facets based on structured
data
16
IBM Argumentor Flow – global cases view
� Argumentor presents relevant
cases taxonomy in a treemap
� Argumentor presents wining and
losing parties statistics
� Attorney can interact with views so
Argumentor will focus on certain
arguments
17
IBM Argumentor Flow – cases view
� Argumentor presents relevant
color coded cases in a list
� Argumentor provides a case view
in a glance:
– Case relevancy
– Case characteristics
– Case taxonomy
� Attorney can interact with views so
Argumentor will focus on
arguments from a specific case
18
IBM Argumentor Flow – arguments pro/con analysis
� Argumentor presents arguments
sorted by relevancy
� Argumentor provides a quick link
between an argument and its case
� Attorney can place a few
arguments in folders - arguments
that support his client or against
him.
� Argumentor learns to classify the
rest of the arguments automatically
19
� https://wpncatalog.stage1.mybluemix.net/assets/assets_debater_claim_detection_service
� https://wpncatalog.stage1.mybluemix.net/assets/assets_activelearning_via_exemplarcluster
ing
20