What Open Data and Open Source can do for Sri Lanka? Srinath Perera, Ph.D VP Research, WSO2 Member, Apache Foundation @srinath_perera
What Open Data and Open Source can do for Sri
Lanka?
Srinath Perera, Ph.DVP Research, WSO2
Member, Apache Foundation@srinath_perera
A Day in Your Life
Success Stories• Money Ball ( Baseball drafting) • Nate Silver predicted outcomes
in 49 of the 50 states in the 2008 U.S. Presidential election
• Cancer detection from Biopsy cells ( Big Data find 12 patterns while we only knew 9), http://go.ted.com/CseS
• Bristol-Myers Squibb reduced the time it takes to run clinical trial simulations by 98%
• Xerox used big data to reduce the attrition rate in its call centers by 20%.
• Kroger Loyalty programs ( growth in 45 consecutive quarters)
If you collect data about your business, and feed it to a Big Data system, you will find useful insights
that will provide competitive advantage– (e.g. Analysis of data sets can find new correlations to
"spot business trends, prevent diseases, combat crime and so on”. [Wikipedia])
Putting Analytics to Work
What happened? And Why? ( Hindsight)
What is Happening right now? ( oversight)
What will happen? (Foresight)
Open Source Market Share
• Apache (60%)• Linux (Servers 16%)• Firefox (25%)• Tomcat and most of
middleware • Android (43%)• Even Microsoft
looking favorably at Opensource projects
• There are lot of open source projects bundled inside the proprietary products
Gartner Predicted that by 2012 most systems will use open source
components Copyright kafka4prez and licensed for reuse under CC License ,
http://www.flickr.com/photos/kafka4prez/198465913
What is Open Source?• Most commercial
software does not distribute the source code, and developed and managed in a closed world.
• Idea of open source is to have the code in the open, and to improve it though volunteer contributions using “open development”
• Idea is that the project becomes a eco-system– More ideas– More developers– More Testers– More Bug fixers
“There is no delight in owning anything
unshared.” Seneca (Roman philosopher,
mid-1st century AD)
How does a Open Source Work?• Open code repository (SVN
or Git etc.)• Two parts of the community
– Developer Community– User Community
• Communication through Mailing lists / IRC Channel– Develop mailing list– User mailing list
• Bug tracking database to track errors (Jira, Bugzilla)
• People submit improvements as patches through Jira etc.
Committers have write access to repository Committers review and apply patches, and when you
submit lot of them, they will make you a committer.
History of Opensource• 1970s – UNIX, Emacs• 1984-85 - GNU project
and Free Software Foundation
• 1990 - GNU project almost complete .. well not OS
• 1991 - Linus Torvalds announce Linux, Phython
• 1993 - Net BSD and Free BSD
• 1994-95 - Linux 1.0 released
• 1995 - Apache, KDE, PHP
• 1997 - Genome• 1999 Linux 2.2,
OpenOffice• 2003 - Firefox, Android
http://www.geograph.org.uk/photo/916456http://www.fotopedia.com/items/flickr-3320704544
Why People Contribute?• As a way to improve your
profile (looking for a Job)• To gain experience• To work with “like minded”
People• To be part of something bigger• To be a “Geek”• As a Job – if you a well known
open source developer, chances are that you will get payed for contribution
• As a competitive strategy
Great investments need faith and patience
Copyright U. S. Fish and Wildlife Service and licensed for reuse under CC License , http://www.flickr.com/photos/usfwsnortheast/4754624921 and Copyright WxMom and licensed
for reuse under CC License , http://www.flickr.com/photos/wxmom/1359996991.
• Sahahna• Apache Axis2 and
other projects
http://www.geograph.org.uk/photo/1842872
LKA Success Stories
Why People use Open Source Software?
• It is cheaper• It is better• Because it is open source
(Religiously)• More visibility into the code, better
security, auditing • If there is a problem, I can fix it • More control over releases, roadmap• Patches become available faster • Easy to understand how it works• Can fork the code if needed • Not own by one person, less risk to
depend on it. • Do not have to maintain the code
Big Data and Opensource Most Big data tools are
free Even the state of the
art is being released as opensource
Give countries like a unique opportunity with a level playing field
Open Data
Make the data public Advanced form of the RTI act Opensource idea applied to data
science E.g. programs like “Code for America”
Code Red: US healthcare.gov Rescue
$300M project, that is failing and small group of volunteers go to hackathon mode to fix it, and fix it.
See http://radar.oreilly.com/
2014/03/code-red_-they-have-no-use-for-someone-who-looks-and-dresses-like-me.html
http://content.time.com/time/magazine/article/0,9171,2166770-1,00.html
Filtering Information with Big Data Big Data can filter
information (e.g. SPAM)
Rank Information ( show most relevant articles)
Find Anomalies ( detect Fraud)
Make recommendations ( product recommendations)
Handle reputations (e.g. Ebay, Amazon)
George Caleb Bingham, 1846
Example: Reddit, Hacker News( Ranking)
Keep Your Customers
Get New Customers
Improve Operations
Monetize your data
Example: Amazon, Ebay, Abnb, (reputation)
Urban Planning and Policy Decisions• Urban Planning
– People distribution – Mobility – Waste Management– Parking
• Policy Decision– What if we change
minimum wage? – What are economic
impact of a new law?
By Aqwis - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=6810430
Example: Big Data for Development• Done using CDR data• People density noon vs.
midnight (red => increased, blue => decreased)
From: http://lirneasia.net/2014/08/what-does-big-data-say-about-sri-lanka/
Traffic Lot of us waste time
on traffic Know where is
traffic ( Google traffic does that)
Emergency Response
Know the traffic patterns
Long term planning
Manage Donors and Charities
Sri Lanka donates a lot (even the poorest)
Does the money goes to intended place Can we track how money is spent?
https://iwringer.files.wordpress.com/2015/09/traffic2.jpg?w=656
Day to day Maintenance
Does the news papers are the best way to get day to day things done?
Can crowd sourcing help? How to stop false tickets?
Disease spread Earlier Malaria and now dengue Know current situation Know overall trends ( focus on
problematic areas) Emergency Response
Summary• There are lot Opensource,
Open data, and Big Data can do for Sri Lanka
• Some cases needs money!! And might be beyond us
• But not for many cases – e.g. Sahana– Hackathon to build an app
to decide what topics to take up in the parliament
• What we really need is collaborations between domain experts and computer scientists