Corpus resources for learning Arabic to understand the Quran Higher Education Academy workshop on "The Role of Corpora in LSP (Language for Specific Purposes) Learning and Teaching" Parkinson B08, University of Leeds, Monday 23rd July 2012. Eric Atwell I-AIBS Institute for Artificial Intelligence and Biological Systems School of Computing University of Leeds
41
Embed
Corpus resources for learning Arabic to understand the Quran Higher Education Academy workshop on "The Role of Corpora in LSP (Language for Specific Purposes)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Corpus resources for learning Arabic to understand the Quran
Higher Education Academy workshop on "The Role of Corporain LSP (Language for Specific Purposes) Learning and Teaching"
Parkinson B08, University of Leeds, Monday 23rd July 2012.
Eric Atwell I-AIBS Institute for Artificial Intelligence
- Orthography (diacritics and vowelization)- Etymology (Semitic roots)- Morphology (derivation and inflection)- Syntax (origins of dependency grammar)- Discourse Analysis & Rhetoric- Semantics & Pragmatics
Originated in Arabs studying the language of the Quran (scientific analysis for at least 1000 years – a lot older than English language!):
(3) ComputingQuran is online, for keyword searchBUT verse-by-verse translations are interpretationsMuslims should access the “true” Classical Arabic source
(3) Computing
Example question-answering dialog system:
QuestionHow long should I breastfeed my child for?
Answer Mothers should suckle their offspring for two years, if the father wishes to complete the term (The Holy Quran, Verse 2:233).
- How far can we go?- An Artificial Intelligence system which “understands” the Quran?
An Artificial Intelligence approach to understanding the Quran
Central HypothesisAugmenting the text of the Quran with rich linguistic annotation will lead to a more intelligent/accurate AI systems.
- Prepare the data by annotating the Quran.- Use the data to build an AI system for concept search and question-answering.
Corpus resources for learning Arabic to understand the Quran
Augmenting the Arabic text of the Quran with rich linguistic annotation will help learners to understand Quranic Arabic.
- Annotate the Quranic Arabic Corpus.- Teacher and Learners use the annotations for deeper understanding of Quranic Arabic.
Straw Poll: LSP for religious texts?• How many Muslims in the audience?
– How many read/recite Classical Arabic Quran?– How many would like to?
• How many Jews in the audience?– How many read/recite Classical Hebrew Tanakh?– How many would like to?
• How many Christians in the audience?– How many read/recite Classical Hebrew/Greek Bible?– How many would like to?
• Have I left anyone out?
Annotating the QuranChallenges
Orthography - Complex non-standard script
Morphology (word structure) - Arabic is highly inflected, challenging to analyze
Grammar - Phrase structure, dependency
Semantics – Ontology of Entities and Concepts referred to by pronouns and nouns
Annotating the QuranSolutions
- Computing advances have made annotation possible, to high accuracy
- Leverage existing resources from Traditional Arabic Grammar
-Machine-Learning annotation followed by manual verification
-- Community effort using online volunteers
Recent Advances: Orthography
Google Search for verse (68:38) on Jan 21, 2008 shows many typos
- Morphological Analysis of the Quran at the University of Haifa (Shuly Wintner, 2004)
- Lexeme & feature based morphological representation of Arabic (Nizar Habash, 2006)
The Haifa Corpus (2004)
Multiple analysis for each word (up to 5)rbb+fa&l+Noun+Triptotic+Masc+Sg+Pron+Dependent+1P+Sgrbb+fa&l+Noun+Triptotic+Masc+Sg+Gen
Not manually verifiedAuthors reports an F-measure of 86%
Non-standard annotation scheme not familiar to Arabic linguists e.g. extracting a list of all verbs is non-trivial
Arabic text is only encoded phonetically not familiar to Arabic linguists e.g. searching for a specific root is not easy
The Quranic Arabic Corpus http://corpus.quran.com/
Kais Dukes – PhD (part-time)
word structure - colour-coded morphological analysis translation – verse, word-for-word English translations grammar- dependency parse following Arabic tradition semantics – ontology of entities and concepts Machine Learning - annotations used for A.I. training Impact - dozens of researchers have collaborated/cited, and over a million visitors use the website per year
- Unicode Uthmani Script- Sourced from the verified Tanzil project
The Quranic Arabic CorpusPhonetics (faja'alnāhumu)
- Phonetic transcription generated algorithmically- Guided by Arabic vowelized diacritics
The Quranic Arabic CorpusInterlinear translation
- Word-for-word translation from accepted sources- Interlinear translation scheme
The Quranic Arabic CorpusLocation Reference (21:70:4)
- Common standard for verses (Chapter:Verse)- Extended in the QAC corpus to include word numbers and segment numbers, e.g. (21:70:4:2)
The Quranic Arabic CorpusMorphological Segmentation
- Division of a single word into multiple segments- Part-of-speech tag assigned to each segment- Traditional Arabic Grammar rules used for division
The Quranic Arabic CorpusMorphological segment features
The Quranic Arabic CorpusArabic Grammar Summary
The Quranic Arabic CorpusSyntactic Annotation
- Dependency Grammar based onإعراب (i'rāb)- Syntactico-semantic roles for each word
The Quranic Arabic CorpusOntology of entities and concepts
- linked to/from nouns and pronouns in the text
The Quranic Arabic CorpusFramework for collaboration
User Interaction via Message Board:“If you come across a word and you feel that a better analysis could be provided, you can suggest a correction online by clicking on an Arabic word”(5000+ resolved messages)
Resources:Publications; Citations, Reviews, FAQs, Feedback,Data Download, Software download, Mailing list
The Quranic Arabic CorpusUsers: researchers, public
- Artificial Intelligence and Computational Linguistics- Arabic linguistics-Quranic and Islamic Studies-Classical literature analysis-Anyone who wants to appreciate the Quran
The Quranic Arabic Corpusnew Computational Linguistics
- First Treebank of Classical Arabic
- Free Treebank of the Quran
- First formal representation of Traditional Arabic Grammar using constituency/dependency graphs
- Machine-Learning parser
User Feedback (300+ comments)“I would like to applaud you for your effort” Prof
Behnam Sadeghi, Stanford University“We are big admirers of the work” Prof Gregory
Crane, Classics Dept, Tufts University “I regularly use your work on the Qur'an and read
it whenever I can.” Prof Yousuf Islam, Director, Daffodil International University
“Congratulations to all concerned on this project” - Prof Michael Arthur, VC, Leeds Uni
Most users are teachers and learners of Quranic Arabic
Over a million users already, and growing; many unforseen social benefits, eg:
“I work as a chaplain in correctional centers in the State of Missouri, U.S.A. Thanks for your permission to use the Quranic Arabic Corpus in these correctional centers” Tadar Wazir.
AI for understanding the Quran
Qurany the first Quran "search for a concept" website http://xyzqurany.appspot.com/
If you choose from the tree of concepts on the left hand side concept "Pillars of Islam" then "The Prayers" then "Performing the Prayers" then "Friday Prayers“...you get Quran verses on this topic in the upper right frame and Hadith on this topic in the lower right frame.
Nora Abbas, Qurany: A Tool to Search for Concepts in the Quran (PDF). 2009
“Google Qurany" html version athttp://www.comp.leeds.ac.uk/nora/html Store each Quran or Haddith verse as a separate web-page, andannotate each web-page with English translations and concept-tags.
Then search is enabled via Google, but "keywords” can be concept-tags and/or English words and/or Arabic words.
Google "Jesus site:http://www.comp.leeds.ac.uk/nora/html“ Google “Friday Prayers site:http://www.comp.leeds.ac.uk/nora/html”
AI for understanding the QuranSALMA – Sawalha Atwell Leeds Morphological Analyser SALMA Morphological analysis of Quran text
Majdi Sawalha, Eric Atwell (2010). Fine-Grain Morphological Analyzer and Part-of-Speech Tagger for Arabic Text. Proc LREC’2010, Valetta, Malta
Majdi Sawalha, Eric Atwell (2010). Constructing and Using Broad-Coverage Lexical Resource for Enhancing Morphological Analysis of Arabic. Proc LREC’2010, Valetta, Malta
AI for understanding the QuranBoundary-Annotated Quran - Tagged with prosodic annotation scheme from Tajwīd
(recitation) mark-up in the Qur'an
Claire Brierley, Majdi Sawalha and Eric Atwell (2012). Open-Source Boundary-Annotated Corpus for Arabic Speech and Language Processing. Proc LREC’2012, Istanbul
Majdi Sawalha, Claire Brierley and Eric Atwell (2012). Predicting Phrase Breaks in Classical and Modern Standard Arabic Text. Proc LREC’2012, Istanbul
AI for understanding the QuranThe Quranic Arabic Corpus - the first online annotated
linguistic resource which shows the Arabic "irab" morphology and grammar for each word and verse in the Holy Quran, including word-by-word morphology and English gloss, and Ontology of Quranic concepts
Kais Dukes, Eric Atwell and Nizar Habash (2011). Supervised Collaboration for Syntactic Annotation of Quranic Arabic. Language Resources and Evaluation Journal (LREJ).
Kais Dukes and Eric Atwell (2012). LAMP: A Multimodal Web Platform for Collaborative Linguistic Analysis. Proc LREC’2012, Istanbul
ConclusionAugmenting the Arabic text of the Quran with rich linguistic annotation will help learners to understand Quranic Arabic.http://corpus.quran.com/
Eric Atwell, Nora Abbas, Claire Brierley, Kais
Dukes, Majdi Sawalha, Abdul-Baquee Sharaf I-AIBS Institute for Artificial Intelligence and