DVTDS
Christian Hollstein, TeraCortex
www.teracortex.com
Presentation of DVTDS
Distributed Virtual Transaction Directory Server
By TeraCortex
● Background
● Architecture
● Virtualization
● Performance
Background: LDAP in Mobile Networks
MediaServerIMSAS
CSCF
IMSdomain
Provisioning System
LDAPTransactions
LDAP
LDAP
SGSN
GGSN
MSC
3GnetworkHLR
MME
HSS
4Gnetwork
3Com CoreBuilder 5000TM Switching Hub
mgt fb fb fb fb fb fb fb fbtpl6tpl6 5302m
SDMDirectory
LDAP
LDAP based Subscriber Data Management
● 3GPP standard rules LDAP as central repository
● Several hundred mobile operators / deployments worldwide
● Major vendors: Ericsson, Huawei, NSN, ZTE, Alcatel
● NSN alone serves 3.2 billion subscriber records
● Several dozen entries per subscriber record
● Probably largest directories worldwide
Consequences for Directory Products
● Millions of subscriber records → billions of entries
● Data federation / distribution
● High availability → geo -redundant deployment / replication
● Consistent provisioning → transaction safeness
● Update signaling to applications → triggers
● Multi application environments → data model virtualization
● High volume traffic → near real time behavior
DVTDS
New Solution Coming Up:
DVTDS Distributed Architecture
Client… > 1000
Client Client Client
DVTDS (chained)
LDAPChaining
...
LDAP
LDAPChaining
• LDAP protocol for chaining• Multi level hierarchy• Leaves may be any LDAP server• Sessions span over several servers• Servers may be replicated• Distributed transactions
Possiblesessionpath
DVTDS 1000 million keyson 64 GB machine (mirrored)
...
(chained, mirrord)
...
Data Replication
(Mirror 0)
LDAP Mirror
• Symmetrical Multi Master Replication• No single point of failure• Logical DSA concept• Compatible with LDAP chaining• Priority based conflict resolving, real time• LDAP protocol• Up to eight servers per DSA, fully meshed• Transaction safe
Logical DSA
Client
Sessionpath
LDAPconnection toany of themirrors
(Mirror 1)
(Mirror 2) (Mirror 3)
Replication and Conflict Resolving
DVTDSSite APrio 2
UserPrio 7
LDAPDeletePrio 0
LDAPMirror
• Conflicts recognized and handled in real time• Based on request, user and server priority• Keeps to ACID paradigm• Data consistent across sites under attack• Winner gets “Success”. Looser gets “Busy”
Sessionpath
DVTDSSite BPrio 5
LDAPModifyPrio 1
Sessionpath
UserPrio 4
ObjectResolverObject
ResolverObject
ResolverObject
Resolver
System Integrationand External Interfaces
...Client Port
AdminPort
LDAP
...Capture Port
Log FileTrigger...
...DataFederation
...DataReplication
...
Restore / DataMigration
...
Reports...
Backup / DataMigration
...
SOAP/HTTP
CSV
LDAP
LDAP LDAP
CSV
LDIF
CSV
LDIF
LDIF
CSVLDIF
LDAP
BinaryASN.1
OAMSystem
Applications /Provisioning
Internal Architecture
...Client Ports Capture Ports
Session ...
Protocol Stack
Object Resolver
Execution Unit
Protocol Stack
Object Resolver
Execution Unit
Protocol Stack
Object Resolver
Execution Unit...
Interfaces:TriggerBackupRestoreMigrationReportsAdminLog filesChainingReplication
ConfigurationSchemaBackup/RestoreTraffic controlTuningDNSLicensesLogging/Audits
Interlocking sub system
DirectoryInformation Tree
Central Data Area
DVTDS
Session, queuecontrol
Hard disk sub system
Architectural Features
● Free configurable client ports
● Each client port serves a number of sessions
● Each session lives inside its own worker thread
● Object level locking system
● Direct data allocation on memory mapped hard disk volumes
● Volumes maybe cooked or raw file space
LDAP Data Model Virtualization
HLRMMS
HSS
PCRFFixedNet
MNP
AAA IMS
M2M
Social Networks
Data access viaapplication views
Physical dataaccess (No views)
ViewLayer
Provisioning System
Application Data
CoreData
Supported LDAP View Mechanisms
● Transparent aliases
● Rule based bidirectional DN conversion
● Virtual objects
● Virtual and real attributes can be mixed in any object
● Soon: Rule based bidirectional attribute/value conversion
● Integrated in the DVTDS kernel → little overhead
● Online configurable → no service interruption
ou=MOBILE ou=FiXEDou=EMAIL
Data Aggregation by Virtualization:Physical Telco Model
UID=777888000000001
oc: inetOrgPerson
oc: imsiUidAlias
dc=IMSI
oc: dcObject
o=<BusinessUnit>
oc: organization
ou=subscriberData
oc: organizationalUnitt IMSI=777888000000001
Access Path
ou=IDENTITY
oc: imsiAlias
dc=IMSI
oc: dcObject
IMSI=262011100000001
oc: imsiUidAlias
dc=IMSI
oc: dcObject
IMSI=777888000000001
oc: msisdnAlias
dc=MSISDN
oc: dcObject
MSISDN=4916096220958
oc: imsiUidAlias
dc=IMSI
oc: dcObject
IMSI=777888000000001
oc: mailAlias
dc=EMAIL
oc: dcObject
dc=IMSI
oc: dcObject
dc=Enterprise
oc: dcObject
dc=IMSI
oc: dcObject
dc=configurableViews
oc: dcObject
oc: imsiUidAlias
dc=IMSI
oc: dcObject
IMSI=777888000000001
oc: accountlAlias
dc=ACCOUNT
oc: dcObject
account=1234abcd
dc=FIXED
oc: fixedNetDataparam4: real valueparam5: real value...Fixed Net: reference
dc=IDENTITY
oc: identityDataparam6: real valueparam7: real value...Identy: reference
dc=EMAIL
oc: eMailDataparam2: real valueparam3: real value...Email: referencedc=MOBILE
oc: mobileDataparam0: real valueparam1: real value...Mobile: reference
Fixednet data
SubscriberIdentities
Email DataMobile
Data
...
View Mechanism Properties
● Each subscriber has individual data below uid=...
● Accessed via transparent aliases
● Application view data outside of subscriber data
● Found by two stage resolving algorithm
● Different applications can share physical data
Example: Server – Side DN Conversion
Server Side Conversion Rule:
clientDn: *,impi=(sip):([0-9]+)@(ims.telekom.de),dc=IMPI
serverDn: imsi=#3(2),dc=IMSI
DN as sent by the client:
ou=mobile,impi=sip:[email protected],dc=IMPI
DN as used by the server:
ou=mobile,imsi=262000000000000,dc=IMSI
Throughput in absolute numbers
200000
Entryload
LDAPSearch
LDAPModify
Op
erat
ions
/ s
DVTDSIntel I7 4960X6 Cores @4.6 GHz32 GB RAM7 x SATA 7200 RPM28 Million entries
Oracle OIDSparc T5-232 cores @3.6 GHz512 GB RAMFlash disk array50 million entries
LDAPCompare
LDAPAdd
100000
300000
400000
500000
600000
700000
800000
900000
1000000
Throughput per GHz CPU speed
6000
Entryload
LDAPSearch
LDAPModify
Op
erat
ions
/ s
DVTDSIntel I7 4960X6 Cores @4.6 GHz= 27.6 GHz
Oracle OIDSparc T5-232 cores @3.6 GHz= 115.2 GHz
LDAPCompare
LDAPAdd
3000
9000
12000
15000
18000
21000
24000
27000
Throughput Scaling
Notes on 3D Server Throughput Diagram
● 2 Variables: queue length and number of clients
● Throughput increases with bigger queue length
● Throughput scales by number of cores and clients
● Saturation on 6 core machine at 6 clients
● Degradation when operated beyond saturation
● Linear scaling if not bottle - necked by memory bandwidth
Scaling the Data
• 540 million entries in less than 2 hours• Naming attribute was indexed• Indexing time included, no setup time• Multi threaded object loader• LDAP protocol / BER object format• 30 GB RAM, 366 GB data base size
540 Million entries
inetOrgPerson22 AttributesLDIF size: 532 bytes
114 Minutes load time
Linear sc
aling
Roadmap 2014
● Automatic replica reconciliation after mirror network faults
● Free configurable indices
● User level documentation
● Free demo version download
Thank you for your attention!
www.teracortex.com