WebSocket MicroService vs. REST Microservice

Rick Hightower - Mammatus Technology Inc.

WebSocket MicroService vs. REST Microservice

How does the performance of a REST microservice compare to a WebSocket MicorService?

Comparison of Speed

❖ WebSocket is faster and uses less resources for more calls

❖ REST is slower due to each connection needing to wait for a response before it can send another request

REST/HTTP vs WebSocket❖ With REST you have to wait for a reply before

you can send another request (over the same connection)❖ Unless you use pipelining which seems to

work better in benchmarks than the real world❖ With WebSocket you can stream requests and

stream responses and then marry them together❖ Still request/reply but WebSocket can send

responses while it is getting other requests

Example uses QBit❖ QBit supports REST and WebSocket RPC❖ QBit allows you to stream requests over the wire

and receive responses over the wire. It employs micro-batching to batch requests and responses into larger chunks to minimize IO and thread hand-off costs

❖ QBit allows async REST calls and async WebSocket calls. (As well as internal services based on queues)

wrk Testing tool for REST/HTTP

❖ To perf test REST code we use wrk ❖ wrk load tester capable of generating significant

load when run on a single multi-core CPU❖ wrk load tester uses multithreaded and event

notification systems ❖ epoll and kqueue

❖ wrk maximizes the number of HTTP requests per second

Trade Service

wrk LUA script

Caveats About the Test❖ We can tweak server a bit and reduce the flush rate or reduce the batch size to get

higher throughput with lower connections❖ We can also tweak the OS so we can have more ephemeral ports available and then

use a lot more connections❖ experience tells me that we can get close to 90K TPS or so on a MacBook Pro

❖ We could test from two machines ❖ one of those machines being a Linux server with a tweaked stack

❖ This test has disadvantage of all being run on same machine❖ but same disadvantage that the WebSocket version will have so it is somewhat

fair❖ We could also employ HTTP pipelining to increase the throughput,

❖ great trick for benchmarks rarely works in production environments with real clients

❖ On commodity Linux server, we can get close to 150K TPS to 200K TPS from experience

WebSocket Test❖ This will be a bit harder❖ No tool like wrk to test WebSocket RPC calls so

we will write our own❖ RPC calls are QBit centric and rely on JSON plus

some custom ASCII protocol ❖ Some overhead in the protocol because we have

to match responses to clients and requests need unique identifier (as well as the client needing a unique identifier)

To use WebSocket we will need Async interface to service

WebSocket PERF Test code

WebSocket Perf code continued

How does the WebSocket microservice do?

1 Million messages a second.

700K messages for a single WebSocket connection.

7.5x better than REST version.

Overhead❖ Running code through perf showed it was waiting for IO

a lot. Better IO. Better NIC card. More TCP/IP stack tuning. You can expect better throughput.

❖ QBit has to marry requests with responses/client id combinations. There is overhead in doing this. QBit also uses JSON/ASCII protocol which could be optimized to reduce IO (gzip it or use message pack).

❖ Custom more efficient streaming can be implemented if needed (from experience, this is another 2x to 4x)

❖ But in general, WebSocket will do better than HTTP

More Caveats❖ When using WebSocket, you also need service

discovery, and you need to implement your own failover ❖ You can’t just throw a bunch of microservices

behind a load balancer❖ QBit integrates with DNS and consul.io to

provide service discovery (also allows you to use push)

Follow up❖ This slide deck as a blog post:

❖ http://rick-hightower.blogspot.com/2015/12/websocket-microservice-vs-rest.html

❖ The QBit project: ❖ https://github.com/advantageous/qbit

❖ Mammatus Technologies ❖ http://www.mammatustech.com/

❖ Rick Hightower❖ https://twitter.com/RickHigh❖ https://www.linkedin.com/in/rickhigh

There is a lot more to QBit than this

QBit supports reactive stream over WebSocket

QBit implements Service Actors

Check out QBithttps://github.com/advantageous/qbit