Using Document Databases with TYPO3 Flow

Inspiring people to

shareInspiring people to

share

Using Document Databases with

TYPO3 Flow

Karsten Dambekalns

TYPO3 Neos and Flow developer

35 years old

lives in Lübeck, Germany

1 wife, 3 sons

1 espresso machine

likes canoeing & climbing

Persistence Basicsin Flow

Flow uses Doctrine 2 ORM

MySQL PostgreSQL SQLite Oracle

Native and PDO DB drivers

Doctrine DBAL

Doctrine ORM

TYPO3 Flow

Flow provides a centralized PersistenceManager

Native and PDO DB drivers

Doctrine DBAL

Doctrine ORM

Doctrine\PersistenceManager

Repository Doctrine\Repository

Repositories encapsulate persistence concerns

Doctrine\PersistenceManager

Repository Doctrine\Repository

Your Application Code

Models do not know about persistence

Models do not know (a lot)

about persistence(internals)

Properties

/** * @Flow\Entity */class Organisation {

/** * The name * @var string * @Flow\Validate(type="NotEmpty") * @Flow\Validate(type="StringLength", options={"maximum"=40}) * @ORM\Column(length=40) */ protected $name;

Relations

/** * @Flow\Entity */class Organisation {

/** * @var \Doctrine\Common\Collections\Collection <\Acme\Demo\Domain\Model\Alias> * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"}) * @ORM\OrderBy({"name" = "ASC"}) * @Flow\Lazy */ protected $aliases;

Document Databases

Provide some powerful advantages

• schema-less storage

• usually scale well

• really good for non-relational data

No more Object-Relational Mapping, now you have Object-Document Mapping

Queries, but not as you know them

•Most document databases do not allow queries on arbitary properties

Which Database to Choose

Many different document databases exist

Which one is right for your project depends on many factors

•MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back.

• CouchDB: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important.

A nice overview with much more can be found onhttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

CouchDB

Is one of the more widespread products

Stores documents in JSON

Provides a REST interface

Provides master-master replication and MVCC

Futon on CouchDB

CouchDB REST API

Create a database

curl -X PUT http://127.0.0.1:5984/demo

{"ok":true}

Create documents

curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}'

{"ok":true,"id":"8843faaf0b831d364278331bc3001bd8","rev":"1-33b9fbce46930280dab37d672bbc8bb9"}

CouchDB Basics

Fetch a document

curl -X GET http://127.0.0.1:5984/demo/8843faaf0b831d364278331bc3001bd8

{"_id":"8843faaf0b831d364278331bc3001bd8","_rev":"1-33b9fbce46930280dab37d672bbc8bb9","company":"Example, Inc."}

CouchDB Basics

Create a database using Futon or with

curl -X PUT http://127.0.0.1:5984/demo

Create documents using Futon or with

curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}'

Fetch documents using Futon

If you do manual work on the shell, try HTTPie instead of cURL

Read guide.couchdb.org and docs.couchdb.org

CouchDB Basics

Read guide.couchdb.org and docs.couchdb.org

Learn about views and map/reduce

Hint: A better cURL

If you do manual work on the shell, try HTTPie instead of cURL

TYPO3.CouchDB

Developed for Rossmann by networkteam

Fully replaces the ORM persistence

Model annotations stay the same

Provides basic QOM-to-View mapping

Note: Not working with TYPO3 Flow 2.0 right now

Installation

cd Packages/Applicationgit clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git

No composer package, so just clone it

Configuration

TYPO3: FLOW3: persistence: # Options for the CouchDB backend backendOptions: # database: 'database_name' dataSourceName: 'http://127.0.0.1:5984' username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~

Usage – Repository

class TestEntityRepository extends Repository {

/** * @param string $name * @return \TYPO3\FLOW3\Persistence\QueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like('name', $name) ); return $query->execute(); }}

Usage – LuceneIndex

class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex {

/** * Configure the index * * @return void */ public function configure() { $this->forEntity('Acme\Demo\Domain\Model\SomeEntity') ->indexProperty('name') ->indexProperty('relatedValueObject.color'); }

}

Usage – Views & Design Docs

Views can be defined by

• implementing TYPO3\CouchDB\ViewInterface

•configuring TYPO3\CouchDB\QueryView

Design documents can be defined by extending TYPO3\CouchDB\DesignDocument

Now for somebad news

Only CouchDB is supported

Kristina AlexandersonCC BY-NC-SA 2.0

One Backend at a Time

Doctrine DBAL

Doctrine ORM

Doctrine\PersistenceManager

Repository Doctrine\Repository

One Backend at a Time

Doctrine DBAL

Doctrine ORM

Doctrine\PersistenceManager

Repository Doctrine\Repository

You can work around this!

In your repositories you can basically do what you want

• Open a direct database connection

• Read CSV files

• Connect to a Solr server

• Instantiate another Doctrine EntityManager

Do something more advanced, let Radmiraal.CouchDB inspire you

As long as you encapsulate nicely!

And now somegood news

Radmiraal.CouchDB

Radmiraal.CouchDB

Developed for some projects having similar needs

Started out as an extension to TYPO3.CouchDB

Since then refactored & now based on Doctrine 2 CouchDB ODM

Can be used alongside the usual ORM

Radmiraal.CouchDB

Radmiraal.CouchDB

"repositories": [ { "type": "git", "url": "https://github.com/radmiraal/Radmiraal.CouchDB.git" }],

"require": { "radmiraal/couchdb": "dev-doctrine",}

Installation

TYPO3: Flow: persistence: backendOptions: … as usual

Radmiraal: CouchDB: persistence: backendOptions: databaseName: 'mycouchdb' username: 'mycouchuser' password: 'mycouchpassw0rd'

Configuration

Usage – Models

use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM;

/** * @ODM\Document(indexed=true) */class DataSet {

/** * @var string * @ODM\Id(type="string") */ protected $id;

/** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") * @ODM\Index */ protected $organisation;

/** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;

Usage – Models

/** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") * @ODM\Index */ protected $organisation;

/** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;

Usage – Repository

class OrganisationBoundaryRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository {

public function findAll() { return $this->createQuery( 'organisationboundaries', 'by_organisation_id') ->execute(); }}

Usage – Repository

class DataSetRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository {

public function findByQuux(Quux $quux) { return $this->createQuery('datasets', 'most_recent', 'native') ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute()}

Usage – Views & Design Docs

And now someeven better news

Multiple backends in parallel

Doctrine DBAL

Doctrine ORM

PersistenceManager

Doctrine\Repository

MongoDB\Repository

Doctrine MongoDB

MongoDB ODM

Doctrine\Repository

The Future

TYPO3 Flow will support multiple persistence backends in parallel:

•of the same type: 2 MySQL and 1 PostgreSQL databases

•of different types: MySQL and MongoDB

The Future

Configuration extended but backwards compatible

Consistent use independent of backend used:Annotations, result interfaces & queries stay consistent

References across persistence backends? Eventually, but not right away…

You can help, too…

The work on a proper ODM integration so far has been done and made possible by (applause!):

•Beech IT

•BKWI

•Rens Admiraal

•myself

More work is needed, so if you think this is the way to go… contact me!

questionsyour

now!

Thank You!

These slides can be found at:http://speakerdeck.com/kdambekalnshttp://slideshare.net/kfish

Give me feedback: [email protected] | [email protected]

Book me: [email protected]

Follow me on twitter: @kdambekalns

Support me using

My Amazon wishlist: http://bit.ly/kdambekalns-wishlist