1 media.netlogix.de October 2, 2010 Clemens Kalb, Lienhart Woitok, T3CON10
Building a large eCommerce application with Extbase, Fluid and Apache Solr
May 19, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
2
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
3
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
IBC SOLAR: Selling
everything from a
solar module to
complete solar plants
Working with retail
partners in Germany
and throughout the
world
857 million turnover
in 2009
4-figure number of
products x product
variants x retail
partners
4
media.netlogix.de
„Build yet another Shop System for TYPO3 – are you
insane?“
All of us & the customer (before the start of the project)
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
5
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Previous system based on Commerce
• scalability issues
tt_products? Hell no! ;-)
Specialized functionality lacking in all systems
Magento looked fairly nice – but not integrated in
TYPO3
6
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
7
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
8
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
9
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
10
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
All data comes from the company‘s ERP system
Custom import functionality for complete orders
Individual prices for customers
No need for retail functions like online payment
processing
11
media.netlogix.de
standing on the shoulders of giants
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
12
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
How do you design a generic domain model for a
shop system?
13
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
You don‘t …(unless you have too much time on your hands)
14
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
…because it already exists:
15
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
BMEcat
16
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Proven & open standard for product catalogues
Very well documented (specification >600 pages)
Originally an XML data exchange format
17
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
The Extbase kickstarter:
a great way to create lots of model objects
18
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
19
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
20
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
The kickstarter is not an editor!
21
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Using SCM to cherry-pick changes:
1. Save in kickstarter
2. svn revert <fileThatShouldNotBeTouched>
3. Eclipse diff view to manually merge files
22
media.netlogix.de
Apache Solr as a Storage Backend
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
23
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Java based search
server
Searches millions of
documents (products,
text entries, prices…)
in very little time
Provides faceted
search facilities
24
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
The naive way:
writing direct querys to the Solr server from the
application code
25
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Repository QueryExtbase
persistencelayer
Storage Backend
Background storage
(DB, Solr…)
NO
FACETED
QUERIES
Faceted
Query
Solr storage
backend
26
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Tx_Extbase_Persistence_Query
$query->equals(‘foobar’, $foobar)
$query->logicalAnd($statement1,
$statement2)
$query->in(‘foobar’, array($item1,
$item2))
27
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Tx_Nxsolrbackend_Persistence_Backend_
SolrBackend
foobar:$foobar
($statement1 AND $statement2)
foobar:($item1 $item2)
28
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Tx_Nxsolrbackend_Persistence_FacetedQuery
extends Tx_Extbase_Persistence_Query
$query->facet($facetConfiguration)
$query->execute(): Tx_Nxsolrbackend_Persistence_
FacetedQueryResult
$facetedQueryResult->getObjects()
$facetedQueryResult->getFacets()
29
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Structural differences: database vs. index
Data duplication: database and Solr index
Extbase knows only one StorageBackend
30
media.netlogix.de
Pentaho Data Integration
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
31
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
How to feed data into the database
from external sources?
32
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
(Photo courtesy of trenttsd/flickr)
33
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Looking for the right tool:
Integrate different data sources homogenously
Source data from Microsoft Dynamics NAV
(MSSQL)
CSV file to specify additional files for products
like datasheets and the like
Need to understand it 3 years later
34
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Pentaho Data Integration
Extract, Transform and Load
JDBC + plugins for different data sources/targets
35
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Configurability: import into different sysfolders,
configure parameters and default values
Where needed: Use JS and external code for
special transformations
36
media.netlogix.de
How fast is Extbase?
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
37
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
First implementations were painfully slow due to…
Working with thousands of objects
Bug in Extbase: most objects considered dirty at
any time
38
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Build first, optimize later!(Might be scary at times, but the approach does actually work…)
39
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Optimization #1:
Make heavy use of lazy loading, load objects only
when needed
40
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Optimization #2:
Fix Extbase: Objects no longer dirty ;-)
41
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Optimization #3:
Not everything needs to be an object:
If you don‘t need domain logic,
you might just render raw query results
42
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Optimization #4:
Use the good old TYPO3 caching mechanisms,
but be especially careful
43
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
44
media.netlogix.de
October 2, 2010
Clemens Kalb, Lienhart Woitok, T3CON10
Related Documents
