NO SQL WITH CACHING, SEARCH AND REAL-TIME ANALYTICS James Gorlick Basho bit.ly/bdp-on-the-under-ln @paegun
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NOSQL WITH CACHING, SEARCH AND REAL-TIME
ANALYTICSJames Gorlick
Basho
bit.ly/bdp-on-the-under-ln @paegun
About Me
Application Developer with Lead and Project Management experience in a wide variety of business applications. Particularly interested in straight-through processing, removing data duplication, and increasing information availability throughout the organization in a secure manner. Previously with Dimensional Fund Advisors performed data science with much Fortran and many data sources while helping DFA "buy the market". Developed a number of in-house solutions for Myspace to serve purposes similar to what BDP does and will provide.
@paegun
Word Count
Just as developer activities extend beyond writing and testing software into what were typically operational concerns, data solutions are extending beyond storing and retrieving data from a single database management system into the deployment and management of heterogeneous services.
A walk through the development and operational support of a reference application utilizing the Basho Data Platform, gains within the development and operations cycles will be demonstrated.
Excellent DataNLC Origin TLC Station Name Government Office Region (GOR)County or Unitary AuthorityDistrict or Unitary AuthorityNUTS2 Spatial_Unit CodeNUTS2 Spatial Unit Station Facility OwnerStation Group
5131 ABW Abbey Wood London Greater London Greenwich UKI2 Outer London Southeastern
3813 ABE Aber Wales - Cymru Caerphilly - CaerffiliCaerphilly - Caerffili UKL1
West Wales and the Valleys Arriva Trains Wales
3801 ACY Abercynon Wales - CymruRhondda Cynon Taff - Rhondda Cynon Taf
Rhondda Cynon Taff - Rhondda Cynon Taf UKL1
West Wales and the Valleys Arriva Trains Wales
3982 ABA Aberdare Wales - CymruRhondda Cynon Taff - Rhondda Cynon Taf
Rhondda Cynon Taff - Rhondda Cynon Taf UKL1
West Wales and the Valleys Arriva Trains Wales
8976 ABD Aberdeen Scotland Aberdeen City Aberdeen City UKM1 North Eastern Scotland First ScotRail9090 AUR Aberdour Scotland Fife Fife UKM2 Eastern Scotland First ScotRail
4435 AVY Aberdovey Wales - Cymru Gwynedd - GwyneddGwynedd - Gwynedd UKL1
West Wales and the Valleys Arriva Trains Wales
4440 ABH Abererch Wales - Cymru Gwynedd - GwyneddGwynedd - Gwynedd UKL1
West Wales and the Valleys Arriva Trains Wales
3602 AGV Abergavenny Wales - Cymru Monmouthshire - Sir FynwyMonmouthshire - Sir Fynwy UKL2 East Wales Arriva Trains Wales
2448 AGL Abergele & Pensarn Wales - Cymru Conwy - Conwy Conwy - Conwy UKL1West Wales and the Valleys Arriva Trains Wales
4303 AYW Aberystwyth Wales - Cymru Ceredigion - Sir CeredigionCeredigion - Sir Ceredigion UKL1
West Wales and the Valleys Arriva Trains Wales
2552 ACR Accrington North West Lancashire Hyndburn UKD4 Lancashire Northern Rail8600 AAT Achanalt Scotland Highland Highland UKM4 Highlands and Islands First ScotRail8601 ACN Achnasheen Scotland Highland Highland UKM4 Highlands and Islands First ScotRail8602 ACH Achnashellach Scotland Highland Highland UKM4 Highlands and Islands First ScotRail
7662 ACK Acklington North East Northumberland Alnwick UKC2Northumberland and Tyne and Wear Northern Rail
7286 ACL Acle East Norfolk Broadland UKH1 East AngliaAbellio Greater Anglia
4529 ACG Acocks Green West Midlands West Midlands Birmingham UKG3 West MidlandsLondon Midland Trains
2287 ACB Acton Bridge North West Cheshire Vale Royal UKD2 CheshireLondon Midland Trains
Period and
Financial year
Reporting Period
Days in period
Period beginning
Period ending
Bus journeys (m)
Underground journeys (m)
DLR Journeys (m)
Tram Journeys (m)
Overground Journeys (m)
Emirates Airline
Journeys (m)
01_10/11 1 31 1-Apr-10 1-May-10 189.1 90.5 6.3 2.3
02_10/11 2 28 2-May-10 29-May-10 181.6 84.5 5.8 2.203_10/11 3 28 30-May-10 26-Jun-10 175.9 84.3 5.8 2.104_10/11 4 28 27-Jun-10 24-Jul-10 183.4 86.5 6.1 2.105_10/11 5 28 25-Jul-10 21-Aug-10 160.4 82.9 5.8 2.006_10/11 6 28 22-Aug-10 18-Sep-10 175.8 80.9 5.5 2.007_10/11 7 28 19-Sep-10 16-Oct-10 189.8 88.7 6.3 2.308_10/11 8 28 17-Oct-10 13-Nov-10 179.9 90.3 6.7 2.2 5.609_10/11 9 28 14-Nov-10 11-Dec-10 178.8 90.6 6.4 2.3 5.410_10/11 10 28 12-Dec-10 8-Jan-11 140.1 72.5 4.8 1.8 3.511_10/11 11 28 9-Jan-11 5-Feb-11 183.0 84.4 6.3 2.1 5.212_10/11 12 28 6-Feb-11 5-Mar-11 177.2 87.8 6.5 2.2 5.213_10/11 13 26 6-Mar-11 31-Mar-11 173.9 83.5 6.0 2.1 5.8
01_11/12 1 30 1-Apr-11 30-Apr-11 183.8 91.2 6.4 2.1 6.3
02_11/12 2 28 1-May-11 28-May-11 186.1 87.8 6.3 2.2 6.403_11/12 3 28 29-May-11 25-Jun-11 181.7 88.9 6.1 2.2 6.904_11/12 4 28 26-Jun-11 23-Jul-11 186.7 92.5 6.4 2.3 7.605_11/12 5 28 24-Jul-11 20-Aug-11 161.1 85.5 6.2 1.9 7.706_11/12 6 28 21-Aug-11 17-Sep-11 173.9 85.3 6.4 2.1 7.707_11/12 7 28 18-Sep-11 15-Oct-11 193.4 93.1 7.5 2.4 8.708_11/12 8 28 16-Oct-11 12-Nov-11 185.2 95.8 7.3 2.3 8.809_11/12 9 28 13-Nov-11 10-Dec-11 189.4 97.1 7.1 2.4 9.010_11/12 10 28 11-Dec-11 7-Jan-12 151.2 79.3 5.2 1.9 6.911_11/12 11 28 8-Jan-12 4-Feb-12 181.4 89.8 6.9 2.2 8.712_11/12 12 28 5-Feb-12 3-Mar-12 179.5 91.5 7.0 2.0 8.813_11/12 13 28 4-Mar-12 31-Mar-12 191.2 92.7 7.3 2.3 9.1
01_12/13 1 28 1-Apr-12 28-Apr-12 171.8 88.7 7.0 2.1 8.5
02_12/13 2 28 29-Apr-12 26-May-12 189.2 92.6 7.4 2.3 8.903_12/13 3 28 27-May-12 23-Jun-12 176.5 89.7 7.0 2.2 8.604_12/13 4 28 24-Jun-12 21-Jul-12 189.9 96.9 7.9 2.5 8.9 0.2405_12/13 5 28 22-Jul-12 18-Aug-12 167.1 103.0 10.6 2.3 10.5 0.5306_12/13 6 28 19-Aug-12 15-Sep-12 177.0 94.0 8.7 2.4 9.8 0.38
Period and Financial
yearReporting
PeriodDays in period
Period ending Month
Number of Lost Customer
Hours
Operated Kms (Peak and Off
Peak)
% of Scheduled Operated
Excess Journey Time
(mins)
02_11/12 2 28 28-May-11 May-11 2,322,089 5,444,365 97.3% 5.90
03_11/12 3 28 25-Jun-11 Jun-11 1,929,445 5,463,580 97.6% 5.79
04_11/12 4 28 23-Jul-11 Jul-11 2,300,769 5,465,180 96.8% 6.22
05_11/12 5 28 20-Aug-11 Aug-11 2,059,198 5,595,332 97.1% 5.32
06_11/12 6 28 17-Sep-11 Sep-11 2,462,085 5,516,502 96.8% 5.58
07_11/12 7 28 15-Oct-11 Oct-11 2,155,102 5,512,264 97.5% 6.29
08_11/12 8 28 12-Nov-11 Nov-11 2,054,499 5,657,701 97.5% 6.08
09_11/12 9 28 10-Dec-11 Dec-11 1,989,066 5,694,703 97.4% 5.69
10_11/12 10 28 7-Jan-12 Jan-12 2,043,901 5,275,158 96.4% 5.07
11_11/12 11 28 4-Feb-12 Feb-12 2,073,884 5,730,948 97.1% 5.59
12_11/12 12 28 3-Mar-12 Mar-12 2,604,341 5,559,547 95.9% 6.59
13_11/12 13 28 31-Mar-12 Mar-12 1,997,431 5,682,740 97.4% 5.81
01_12/13 1 28 28-Apr-12
E_TOOMANYOPTIONS
Prescriptive, Open
Storage
Consistent Hash (Route)
Ring Around The Rosie
Multi Data Center
CACHE
Read-Through
Write-Around
Excuse Me Miss
Search
Ridealong
Multi Service
Analysis
Ship The Compute?
Spread The Load
No Pets
Graph Reality
Window Seat
QUEUE
HOST
For More Information
• London Datastore - http://data.london.gov.uk/
• Basho Data Platform - http://docs.basho.com/dataplatform/latest/
• Spark - http://spark.apache.org/
• Redis - http://redis.io
• Kafka - http://kafka.apache.org/
• Solr - http://lucene.apache.org/solr/
• Riak - http://docs.basho.com/riak/latest/
REHASH?
Related Documents
