HITIQA: Scenario Based Question Answering Tomek Strzalkowski, et al The State University of New York at Albany Paul Kantor, et al Rutgers University Boris.

HITIQA: Scenario Based Question Answering

Tomek Strzalkowski, et alThe State University of New York at Albany

Paul Kantor, et alRutgers University

Boris YamromLehman College of City University of New York

(HLT-NAACL2004 Workshop)

Abstract HITIQA: High Quality Interactive

Question Answering HITIQA is a open-domain QA In this paper we describe some

preliminary results of qualitative evaluation.

It uses novel data-driven semantics to conduct a clarification dialogue.

Introduction(1/3) HITIQA allows user to submit exploratory, a

nalytical Qs.( eg: What has been Russia’s reaction to U.S. bombing of Kosovo?)

The answer is heavily conditioned by what information is in fact available on the topic, background knowledge of the user while certain type is expected (eg: diplomatic statements).

Introduction(2/3)

HITIQA is Dialogue with Information Analytical task: topic + context (time, reci

pient, purpose) Detect, follow, anticipate, negotiate strat

egy turns and shifts

Introduction(3/3)

Acknowledgements Based on work supported by ARDA(Ad

vanced Research and Development Activity)’s AQUAINT(Advanced QA for intelligence)

Data Driven Semantics of Questions

We use a combination of Google and InQuery to retrieve the top 50-200 documents from a large document database.

1.ranked:on target ,near misses,outliers

2.extracted into topic frames.

Partial structuring of text data Frames are defined structures repre-se

nting various “event types” 3 broad domain frames of “Weapon of

mass destruction proliferation domain.”

WMDtransfer WMDDevelop WMD-Tready

All frames contain small num of core attribute using BBN extract 24 kind.

Partial structuring of text data

the Goal frame generated from the question, “Where does al-Qaida have training facilities?”

How it works?

HITIQA Dialogue capabilities System actions: offers, suggestions, clarifications

Offering closely related events (time, space, etc.): Imports from Korea? Missile developments there?

Suggesting similar events involving different players Sarin production in Iran? In Pakistan?

Analyst’s actions affecting answer space Direct: accept, partially accept, or reject offers Indirect: strategy changes, diversions

Support hypothesis formation Explicit:

HITIQA suggestions and offers Analyst information saving actions

Implicit: HITIQA: Direction of dialogue and answer formation Analyst: Deciding what question to ask next

Analyst: What is the history of the nuclear arms program between Russia and Iraq?

HITIQA: Do you want to see material on cooperation on development of weapons between Iraq and Russia?

Analyst: Yes … Who has helped finance the nuclear arms program in Iraq?

HITIQA: Would you be interested in information on weapons in Iraq? Analyst: Yes … What type of nuclear weapons does Iraq possess?HITIQA: Would you be interested in information on dollars in Russia rel

ative to your question? Analyst: Yes … What is the finance debt that Iraq owes Russia?

Answer generation Current Frame-based summaries

Frames in the answer space rendered into headlines Passages sorted by “threads”: target, time, location, etc.

Build more coherence into the answer Use elements Discourse Structure Theory

Applied to passages from multiple documents Passages are output to form a more logical flow

Maximize lucidity of the answer Use dialogue history to structure the answer → folders Compute rhetorical relations between answer elements

justification, elaboration, evidence, contradiction, etc. Answer Summaries

Summarize answer passages using XDoX Summarizer

Workshop Result It is important to determine the realism of

the scenarios used during the workshop relative to the analysts current work tasks in order for any results to be meaningful.

Each analyst was asked a series of five questions These 5 questions were all relative to the realism and difficulty of the scenario tasks used a scale of 1 to 5 based on their agreement with the statement.

Using Camtasia record the session.

Result

“Having HITIQA at work would help

me find information faster than I can currently find it”,

our mean score was 3.83.

Integrated Visual interface