Simon Lia-Jonassen CHATBOTS ARE COMING!
Why now?
• Excessive use of messaging services.• Market transition from Apps to IM.• Theoretical advancements in KD, DM,
ML and IR.• Massive amounts of information.• Nearly unlimited computational capacity.
Messaging service boomhttp://w
ww
.slideshare.net/chatbotscomm
unity/2016-chatbots-com
munity-sections-61784553
• Motivators: free, effortless, fast, asynchronous, persistent, searchable, etc.
Business model
• Apple Siri, Amazon Echo, Facebook M, Google Now, Microsoft Xiaoice/Cortana, and hundreds of startups.
Answering strategies
• Retrieval based methods– use a set of predefined responses.– E.g., search or template based methods
• Generation based methods– generate a new response from scratch.– E.g., ontology or deep learning based methods
• Multi-turn versus single-turn dialogues
Simple chatbots
• IR and rule based methods: – Patterns, templates, similarity…
• Example libraries:– https://github.com/gunthercox/ChatterBot– https://sourceforge.net/projects/chatscript
• See also https://www.youtube.com/watch?v=IjKDO3ThlSc
– http://www.alicebot.org/aiml.html
Traditional dialogue systems
• NLP, knowledge bases, state management.• Example libraries:
• http://www.ling.gu.se/projekt/trindi/trindikit/– A toolkit for building dialogue systems. Support for speech recognition, text-
to-speech synthesis, natural language understanding and natural language generation.
• http://www.ling.gu.se/grupper/dialoglab/godis/– A dialogue system implemented with TrindiKit– Can handle grounding, implicature, correction, clarification, etc.– See also
http://dspace.ou.nl/bitstream/1820/5390/1/INF_20140617_Woudenberg.pdf
• https://github.com/google/cayley– Inspired by the graph database behind Freebase and
Google's Knowledge Graph.
Deep learning based systems
• Lots of ongoing research on different NNapproaches for question answering:– Convolutional, recursive or recurrent neural networks.
• General trend:– Lots of training data and processing power, and deep NN.
• Starting points:– http://www.hangli-hl.com/recent-publications.html
• http://www.hangli-hl.com/uploads/3/1/6/8/3168008/cikm_2015_hang_li.pdf
– http://dblp.uni-trier.de/pers/hd/t/Tian:Fei
Autoencoders
• Read an input sentence one character/token at a time and produce another sentence one character/token at a time.
Autoencoders
• Example libraries:– https://www.tensorflow.org/versions/r0.9/tutorials/seq2seq
• See also http://arxiv.org/pdf/1506.05869v2.pdf– https://github.com/karpathy/char-rnn– http://research.baidu.com/warp-ctc
• Further reading:– https://arxiv.org/pdf/1503.02364v2.pdf – http://arxiv.org/pdf/1409.3215v3.pdf– http://colah.github.io/posts/2015-08-Understanding-LSTMs– http://karpathy.github.io/2015/05/21/rnn-effectiveness– https://skillsmatter.com/skillscasts/6611-visualizing-and-understanding-
recurrent-networks
SAAS products such as wit.ai
• Techniques:– Extract certain predefined entities, intent or sentiment.– Define and extract own entities.
• See also:– http://blogs.aspect.com/comparison-of-bot-frameworks-on-the-market– https://stanfy.com/blog/advanced-natural-language-processing-tools-for-bot-makers
Recap – the main components
1. Input analyzer– NLP, syntactic, semantic, pragmatic, and sentiment analysis,
categorization, entity recognition, etc.2. Dialogue manager
– The current state, history, etc.
3. Specific agent(s)– Task execution, querying, etc.
4. Answer generator
Interesting use cases
• Technical support• Virtual sales• Marketing research• Personal assistance• Application controls
– e.g., Appear.in• M2M/IOT interface