Top Banner
65

Mateusz herych content search problem on android

Nov 29, 2014

Download

Software

apps4allru

 
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Mateusz herych content search problem on android
Page 2: Mateusz herych content search problem on android

Mateusz Herych

Android Developer - Base CRMCo-organizer - GDG KrakowCo-organizer - KrakDroid

Page 3: Mateusz herych content search problem on android

Stats

Page 4: Mateusz herych content search problem on android

LIKE ‘%smth%’

Page 5: Mateusz herych content search problem on android

LIKE ‘%smth%’is not the way.

Page 6: Mateusz herych content search problem on android

Search

Page 7: Mateusz herych content search problem on android

SearchOffline.

Page 8: Mateusz herych content search problem on android

Why?

Page 9: Mateusz herych content search problem on android

Why?Let the backend guys do the job

Page 10: Mateusz herych content search problem on android

Why?Internet is not everywhere.

Page 11: Mateusz herych content search problem on android

Why?Internet is not everywhere.It takes time. (especially SSL)

Page 12: Mateusz herych content search problem on android

Why?Internet is not everywhere.It takes time. (especially SSL)And sometimes it’s shitty.

Page 13: Mateusz herych content search problem on android

Why?Internet is not everywhere.It takes time. (especially SSL)And sometimes it’s shitty.

Page 14: Mateusz herych content search problem on android

Sure, some apps don’t really need it

You need an Internet to order that taxi anyway

Page 15: Mateusz herych content search problem on android

Do you keep offline content?

Let your users navigate fast.

Page 16: Mateusz herych content search problem on android

Did I say fast?

Page 17: Mateusz herych content search problem on android

How?Let’s go deeper.

Page 18: Mateusz herych content search problem on android

Context

Page 19: Mateusz herych content search problem on android

CRM- Contacts- Deals- Notes- ...

Page 20: Mateusz herych content search problem on android

CRM- Contacts (~100)- Deals (~50)- Notes (~100)- ... 2011

Page 21: Mateusz herych content search problem on android

select id from deals where name LIKE ‘%something%’

Page 22: Mateusz herych content search problem on android

CRM- Contacts (~40K)- Deals (~20K)- Notes (~300K)- ...

Page 23: Mateusz herych content search problem on android
Page 24: Mateusz herych content search problem on android

HOW DOES “LIKE” WORKS LIKE?

Page 25: Mateusz herych content search problem on android

Docs saying

Page 26: Mateusz herych content search problem on android

I tried to put all the conditions that need to be satisfied so SQLite can use indices combined with LIKE

operator.

Docs saying

Page 27: Mateusz herych content search problem on android

They didn’t fit.

Docs saying

Page 28: Mateusz herych content search problem on android

http://www.sqlite.org/optoverview.html

Docs saying

Page 29: Mateusz herych content search problem on android

Hey, you, SQLite!

EXPLAIN (my) QUERY PLAN

Page 30: Mateusz herych content search problem on android

PRAGMA case_sensitive_like=1;

Page 31: Mateusz herych content search problem on android

PRAGMA case_sensitive_like=1;CREATE INDEX search_index on deals(name);

Page 32: Mateusz herych content search problem on android

PRAGMA case_sensitive_like=1;CREATE INDEX search_index on deals(name);SELECT id FROM deals WHERE name LIKE ‘Some%’;

Page 33: Mateusz herych content search problem on android

EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘Some%’;

SEARCH TABLE deals USING COVERING INDEX search_index (name>? AND name<?) (~31250 rows)

Page 34: Mateusz herych content search problem on android

EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘%Some%’;

Page 35: Mateusz herych content search problem on android

EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘%Some%’;

SCAN TABLE deals (~500000 rows)

Page 36: Mateusz herych content search problem on android

EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘%Some%’;

SCAN TABLE deals (~500000 rows)

(And then you die)

Page 37: Mateusz herych content search problem on android

first_name || ‘ ‘ || last_name?complicated queries, VIEWs?

Like is NOT the way to go.

Page 38: Mateusz herych content search problem on android

What people think SQLite is

Page 39: Mateusz herych content search problem on android

What SQLite really is

Page 40: Mateusz herych content search problem on android

SQLite is powerful

Not kidding.

Page 41: Mateusz herych content search problem on android

FTS3Full Text Search

Page 42: Mateusz herych content search problem on android

CREATE VIRTUAL TABLE search USING fts3 (tokens)

Page 43: Mateusz herych content search problem on android

?

CREATE VIRTUAL TABLE search USING fts3 (tokens INT)

Page 44: Mateusz herych content search problem on android

Nope.

PRAGMA table_info(search);cid|name|type|notnull|dflt_value|pk0|word||0||0

Page 45: Mateusz herych content search problem on android

All is TEXT, except for hidden rowid.

Page 46: Mateusz herych content search problem on android

What is virtual table?Imagine it’s a Java interface.interface VirtualTable { void insert(Params p); void update(Params p); // etc, also createTable.}

Page 47: Mateusz herych content search problem on android

What is a virtual table?

class Fts3 implements VirtualTable { // …}

Page 48: Mateusz herych content search problem on android
Page 49: Mateusz herych content search problem on android

MATCHLet’s go make some magic.

Page 50: Mateusz herych content search problem on android

SELECT * FROM search WHERE content MATCH ‘something’

Page 51: Mateusz herych content search problem on android

SELECT rowid, * FROM search WHERE content MATCH ‘something’rowid|word1|something2|not something special3|SoMeThInG

Page 52: Mateusz herych content search problem on android

SELECT rowid, * FROM search WHERE contentMATCH ‘some* spe*’

rowid|word2|not something special

Page 53: Mateusz herych content search problem on android

CREATE VIRTUAL TABLE search USING fts3 (author, lyrics)

Page 54: Mateusz herych content search problem on android

SELECT * FROM search WHERE lyrics MATCH ‘author:Giorgio Synthesizer

author |lyricsGiorgio Moroder|..Why don’t I use a synthesizer...

Page 55: Mateusz herych content search problem on android

Cool?

Page 56: Mateusz herych content search problem on android

Cool?Look at this.

Page 57: Mateusz herych content search problem on android

SELECT * FROM search WHERE lyrics MATCH ‘why NEAR synthesizer’

author |lyricsGiorgio Moroder|..Why don’t I use synthesizer...

Page 58: Mateusz herych content search problem on android

SELECT * FROM search WHERE lyrics MATCH ‘why NEAR/3 synthesizer’

author |lyricsGiorgio Moroder|..Why don’t I use synthesizer...

Page 59: Mateusz herych content search problem on android

Tips.

Page 60: Mateusz herych content search problem on android

1. Your FTS vtable should contain only tokens. Divided into sections.

Page 61: Mateusz herych content search problem on android

2. Link your FTS table’s records with other table (containing real object’s id and type) using rowid.

Page 62: Mateusz herych content search problem on android

3. Remember. FTS is fast enough for searching purposes. But it’s always slower than ‘=’ based query on indexed field.

Page 63: Mateusz herych content search problem on android

4. EXPLAIN QUERY PLAN doesn’t work for fts tables. Try to measure it with .timer ON.

Page 64: Mateusz herych content search problem on android

5. ???

Page 65: Mateusz herych content search problem on android

6. QUESTIONS TIME!