Implementing MLP

Implementing MLP

Thomas Lindgren / Fredrik Linder / Magnus Eklund

CellPoint AB

speaker

Location services

• Distinctive feature of mobile services

• Measurements from network collected in a location server (spec: 03.71)

• Clients access information via HTTP/XML interface to server

• Clients are portals, resellers, operators, … which then provide info to end-users

Standardization

• Standardized info format from mobile net => location server

• Plethora of protocols for accessing info location server => client

• Location interoperability forum (LIF) founded to bring some order– Equipment manufacturers, operators, …– Mobile Location Protocol (MLP) = unifier

The MLP Protocol

• Mobile Location Protocol 3.0.0 based on HTTP/1.1 and XML– About half a dozen DTDs define data

• Many features– Desired accuracy, max age, …– Presentation coordinate system– Varying geometric shapes in reply– Multitude of data formats

• ”Every feature is optional”

SIMPL2 and MLP

• SIMPL2 developed by Cellpoint as an MLP compatible protocol– Removed some features

• Zoning, triggering• Still had to signal that features not supported

– Added some features• CLPT-specific extensions for charging records

• Many changes in MLP and SIMPL2 during development– Fourteen SIMPL2 draft specifications prior to release

• Seven released during May-June

SIMPL2 request• <?xml version ="1.0" ?>• <!DOCTYPE svc_init SYSTEM "MLP_SVC_INIT_300.DTD">• <svc_init ver="3.0.0">• <hdr ver="3.0.0">• <client>• <id>application_4</id>• <pwd>secret</pwd>• </client>• </hdr>• <slir ver="3.0.0">• <msids>• <msid type="MSISDN">46702711038</msid>• </msids>• <geo_info>• <CoordinateReferenceSystem>• <Identifier>• <code>4326</code>• <codeSpace>www.epsg.org</codeSpace>• <edition>6.1</edition>• </Identifier>• </CoordinateReferenceSystem>• </geo_info>• </slir>• </svc_init>

SIMPL2 reply• <?xml version ="1.0" ?>• <!DOCTYPE svc_result SYSTEM "MLP_SVC_RESULT_300.DTD">• <svc_result ver="3.0.0">• <slia ver="3.0.0">• <pos>• <msid>46702711038</msid>• <pd>• <time utc_off="+0200">20020623134453</time>• <shape>• <CircularArea srsName="www.epsg.org#4326">• <coord>• <X>20 30 5.4W</X>• <Y>0 0 3.5N</Y>• </coord>• <radius>570</radius>• </CircularArea>• </shape>• </pd>• </pos>• </slia>• </svc_result>

Implementing MLP

• Integrate inets and xmerl• Act as server (MLS)• Act as proxy (MLB)

– Must still handle all of MLP for proxying non-CLPT location servers

• Thoroughly validate all incoming data– Requests and replies

• Translate external internal format• Dispatch to other server or compute location

The bad part

• Aggressive dev schedule (5-6 months x 3 people)– Goal: Release soon after MLP specification is finalized

• Specification mutates quickly– Major and minor syntax and feature changes– Data formats– Error codes– Specification bug fixes

• Only protocol syntax specified, not semantics– Semantics sometimes unclear

The first-half score

• Unsatisfactory code– Specification changes => patch upon patch

• Unsatisfactory testing– Hard for developers to test such a big protocol

– Hard for QA to know what worked in a given release

• Time spent on reacting to trouble reports and specification changes– Release date approaching, but not release

Second-half kickoff

• We needed to get bugs and changes under control– Code must become easy to change

• Esp. XML validation

– Code must have high quality before QA begins• Fixing bugs via QA too slow and unhappy for us and QA

• Must quickly resolve arguments about TRs– Many specification changes => many arguments about

what was valid => lost time

Abstraction

• Get rid of records in crucial places– Use abstract data types instead– Encapsulate data representation in module– Can check that data are consistent when created– Operations are named => legible code, intention clear– (Programming 101 …)

• Why weren’t ADTs used already?– Record notation more convenient (e.g., pattern match)– Records already provide representation independence

(sort of)

Quick and dirty encapsulation

#request_rec{misc = Priv, subscr_i=X#subscr_rec.f3} =>

request_adt:set_subscriber_info(Priv, X)

-module(request_adt).

set_subscriber_info(Priv, X) ->

#request_req{misc=Priv, subscr_i=X#subscr_rec.f3}.

XML validation

• Original code: traverse xml tree, check values• Rewrite for fast change

– Use a rule interpreter + separate validation rule set

– Easy to check that rules obey current specification

– Easy to rewrite or extend without intro bugs• Pace of development made this crucial

• Many draft specs => each must be integrated quickly

Validation rules

-define(tag_specs, [{'svc_init', blank, [{'ver', {'or', [{member, ["3.0.0"]}, {unsupported, ver}]}}]}, ... ]).

apply_rule(blank, Value) -> [] == strip_whites(Value);apply_rule({'or', Rules}, Value) -> lists:any(fun(Rule) -> apply_rule(Rule, Value) end,

Rules);...

Improve quality

• Go for (pre-QA) automated testing– Not a new idea, but awesomely useful (again)– Wrote tester from scratch

• effort paid back very quickly

– Exercise all protocol features– Regression test case added when TR appears– Detects integration bugs as well

• SIMPL2 ”on top of food chain”

Test case specification

test_series(1, 1) ->

Clients = [{"service_a", "secret", ?OK}, ...],

MSID = "...",

[ {Expect,

?svc_init(?hdr_client(Name, Pwd),

?slir(?msids(MSID),

?default_geo_info))

|| {Name, Pwd, Expect} <- Clients ];

...

Social issues

• Some XP practices used (see paper)

• Pair programming worked well

• More refactoring should have been done– Elegance is (should not be) optional– Hard to take the time

Evaluation

• Worked very well in this setting– Bug fixing reduced

– Bug hunting reduced

– Faster turnaround

• We could direct efforts to the right areas– Resolve grey areas of SIMPL2 and MLP

– Tighten code

– Add test cases

– ”Virtuous circle”

Performance evaluation

• Measured SIMPL2 by running test suite– Single request at a time on unloaded system

– Varying cases rather than weighted to normal

• Results:– 15% time in scanning and parsing

– 8% in validation

– 18% in port_command, port_close, …• (tests include acting as proxy or server, Oracle access, …)

– So any speedup from optimizing validation is limited

Future work

• Extend data driven design– Rule-driven translation from/to internal format?

• Generate a validating XML parser given a validation spec?– Extend xmerl

Final score

• ADTs > Records• Data driven design (= validation rules)

reduce complexity• Automated testing• XP-practices (pair programming)• SIMPL2 released to customer same week as

final MLP specification was released• Another grisprotokoll bites the dust