18166746-NeverBlock-RubyKaigi2009

1

RubyKaigi2009

Fast, simple I/O concurrency for RubyRubyのための、高速・簡単な並行 IO

www.espace.com.eg

2

Helloこんにちは

3

Egypt's #1 Ruby shop. Embraced Ruby (and Rails) in 2006 and it currently keeps a team of 20 talented developers

busy serving clients in 4 continents

4

● Not the boxer!● eSpace's CTO● 11 years of C, C++, Java, PHP and JavaScript● Met Ruby in 2005, it's been 4 years now ● Researching software concurrency models● Backed up by a wonderful team of developers

Mohammad Ali ( モハメドアリ )

5

NeverBlock is a Fiber based library that provides I/O concurrency facilities

NeverBlockは Fiberベースのライブラリで、非同期イベントループの複雑さを隠しつつ I/O並行性の性質をアプリケーションに提供する。

What is NeverBlock?

6

How can my Ruby program perform I/O concurrently?Rubyプログラムはどうやって I/Oを並行に実行するのか？

Q

7

Concurrently (並行に )? What is that?

A1

1000.times do socket = TCPSocket.open(host, port) socket.write(request) response = socket.read(chunk)...

socket.closeend

8

Illustration図解

9

Object 1

10

● Trivial to understand and implement

とても簡単に理解できるし、実装もできる● No need to worry about synchronization

同期について考える必要がない● The fastest if your servers have zero latencies

もしサーバがレイテンシゼロなら最速

The Good

11

The Bad● Introduce the slightest latency and it crawls

微少なテイテンシを持ち込み、一定幅をとり続ける

● Large latencies render it unusable

大きく遅延すると利用不能に

12

1000.times do if !fork socket = TCPSocket.open(host, port) socket.write(request) response = socket.read(chunk)

... socket.close exit endendProcess.waitall

Use multiple processesマルチプロセスを使う

A2

13

● Easy to understand and implement

簡単に理解し、実装できる● No synchronization needed (no shared resources)

同期処理が不要（共有リソースがない）● Scales to multiple CPUs!

マルチ CPUだとスケールする！● Scheduling (スケジューリング ) is done by the OS

The Good

14

● A heavy operation that can hurt if excessively used

過度に使うと実行環境にダメージを与えるほど重い処理● Much harder if you want to share resources

リソースの共有が極めて難しい● Communication becomes a lot slower

プロセス間通信を行うと、とても遅くなる

The Bad

15

www.espace.com.eg

@threads = []1000.times do @threads << Thread.new do socket = TCPSocket.open(host, port) socket.write(request) response = socket.read(chunk)

... socket.close [email protected]{|th|th.join}

Use multiple threadsマルチスレッドを使う

A3

16

Illustration図解

17

Object 6

18

● Simple implementations (実装しよう ) are easy● The logic inside the thread body is straight forward

スレッド本体内のロジックは一直線● Creating threads is much faster than forking (~200x)

スレッド生成はフォークに比べて極めて速い（～ 200倍）● Scheduling is done by the OS● Communication (間通信 ) among threads is very fast

The Good

19

● Synchronization for shared data is tricky 共有データの同期はトリッキーな方法が必要● Big locks cause idleness and eventually deadlocks 巨大ロックはアイドル状態を生むし、場合によってはデッドロックする● Many Tiny locks cause degradation and races たくさんの小さなロックは性能低下とレースの原因になる● Ruby 1.9.x does not like to have too many threads

Ruby 1.9.xは多くのスレッドを抱えることを嫌う

The Bad

20

(reactor = Reactor::Base.new).run do 1000.times do socket = TCPSocket.open(host, port) reactor.attach(:write, socket) do socket.write(request) reactor.detach(:write, socket) reactor.attach(:read, socket) do response = socket.read(chunk) socket.close reactor.detach(:read, socket) end end endend

Use an event loopイベントループを使う

A4

21

● Extremely fast, no context switching overhead 恐ろしく速いし、コンテキストスイッチのオーバーヘッドもない● Can scale enormously (if epoll/kqueue are used)

果てしなくスケールする ( もし epoll/kqueueが使えれば )● Uses much less system resources than threads スレッドと比べ、ほとんどシステムリソースを使わない

The Good

22

● Non continuous flow, hard to understand and use 不連続なフローになるため、把握しづらく、使いにくい● If a single action blocks the whole loop is blocked １つのアクションがブロックすれば、全体がブロックされる● Deadlocks can happen (with very careless coding) デッドロックが起こりうる (とてもいい加減なコーディングの場合 )

The Bad

23

• Processes are easy but expensive

マルチプロセスは簡単だが高くつく• Threads are cheaper but tricky

マルチスレッドは比較的低コストだがトリッキー• Event loops are the best performers but too hard

イベントループはベストな性能だが難しすぎる

In Summary (要するに )

24

What about those fibers in Ruby 1.9.x?Ruby 1.9.xの fiberはどうなのだろう？

Q

25

(reactor = Reactor::Base.new).run do 1000.times do Fiber.new do fiber = Fiber.current socket = TCPSocket.open(host, port) reactor.attach(:write, socket){fiber.resume} Fiber.yield reactor.detach(:write, socket) socket.write(request) reactor.attach(:read, socket){fiber.resume} Fiber.yield reactor.detach(:read, socket) response = socket.read(chunk) socket.close end.resume endend

Yes, Ruby 1.9.x can use fibersA

26

● Very fast, switch context only when needed とても速い。必要なときだけコンテキストを切り替える● Can scale almost as much as the event loop can do ほぼイベントループ並にスケールすることができる● Low memory overhead, 4KB stack space per fiber メモリはも低い。 1 fiberあたり 4KB程度のスタックスペース● No race conditions in non I/O code I/O以外のコードでは競合状態が発生しない

The Good

27

● Scheduling fibers is done manually fiberのスケジューリングは手動で行う● The event loop is still explicitly managed イベントループは明示的に管理されている● Recursive calls can easily run out of stack 再帰呼び出しは、簡単にスタックを使い果たす● Context switching (mem)copies the fiber stacks コンテキストスイッチは fiber スタックをコピる

The Bad

28

That's it? No other ways to do concurrency in Ruby?それだけ？ Rubyには他の並行性のやり方は

ないのか？

Q

29

● Revactor, for example, is an Actor library implemented using Fibers

例えば、 Revactorは Fiberで実装した Actorライブラ

である。● And of course, NeverBlock.

そして、勿論、 NeverBlock。

There are many (たくさんある )A

30

NB::Reactor.new do 1000.times do NB::Fiber.new do socket = TCPSocket.open(host, port) socket.write(request) response = socket.read(chunk) socket.close end.resume endend

NeverBlock In Action

31

● Developers can write normal “blocking” Ruby code 開発者は、普通の "blocking" Rubyコードを書ける● Scheduling is handled almost transparently スケジューリングはほぼ透過的に扱われる● Looks a lot like threads but without synchronization 同期しない複数のスレッドのように見える● Generally faster than processes and threads 通常、マルチスレッドや、マルチプロセスより速い

The Good

32

● A bit slower than Event Loops

イベントループよりわずかに遅い

● Uses more memory

より多くのメモリを使う

● CPU operations block

CPU演算中は切り替わらない

The Bad

33

Examples?例は？

34

# @pipes is an array of processes to ping

NB::Reactor.run do

@pipes.each do |pipe|

NB::Fiber.run do

loop do

pipe.write('ping')

sleep 3

end

end

end

end

Writing to a pipe every 3 secondsパイプに３秒ごとに書き出す

35

# @images is an array of images to process

# concurrently

NB::Reactor.run do

@images.each do |image|

NB::Fiber.run do

system('slow_processor', image.path)

end

end

end

Calling slow external programs遅い外部プログラムを呼び出す

36

# connecting to a server that might not respond in time

NB::Reactor.run do

@servers.each do |s|

NB::Fiber.run do

timeout(3) do

conn = TCPSocket.connect(s.host, s.port)

conn.write(request)

response = conn.gets('\r\n\r\n')

conn.close

end

end

end

end

Network I/O with timeout タイムアウトなしのネットワーク I/O

36

37

# sending a large response in chunks

NB::Reactor.run do

@connections.each do |conn|

NB::Fiber.run do

response.each_chunk do |chunk|

conn.send(chunk)

end

end

end

end

Large data transfers巨大なデータの送信

38

And how does that work?どうやって動い

ている？

Q

39

# the reactor supports fiber assisted

# waiting for notifications

class NB::Reactor

def wait(mode, io)

fiber = Fiber.current

self.attach(mode, io){fiber.resume}

Fiber.yield

self.detach(mode, io)

end

end

It's all Fibers and Event loopsすべてが Fiberで、イベントループ

40

# The I/O classes use NeverBlock for transparent

# concurrency

class MySQL

def query(sql)

send_query(sql)

# this method selects the appropriate reactor and

# calls it

NB.wait(:read, self.socket)

# the query method continues once the socket is ready

get_restult

end

end

Along with modified I/O classes改良された I/Oクラスもある

41

All hidden from client codeクライントコードからみえない

# The query method will yield the current

# fiber till the query results are ready.

# This gives way for other fibers to run.

mysql.query(sql).each{|r|..}

# note that this requires the use of

# the MySQLPlus adapter which extends

# the original adapter with an async

# query API.

42

Illustration図解

43

Object 7

44

Case Study:

Thin static file serving performance

45

Thin vs Mongrel (small static files)

~200 Bytes ~1 KB ~30KB ~126KB0

500

1000

1500

2000

2500

3000

3500

MongrelThin

File Size

Req

uest

s / S

ec

46

Thin vs Mongrel (large static file, ~170MB)

10/40 20/40 40/400

50

100

150

200

250

300

MongrelThin

Concurrency/Requests

MB

/sec

47

# Send the response

@response.each do |chunk|

trace { chunk }

send_data chunk

end

The offending code(lib/thin/connection.rb, line #103)

48

All file data will be buffered in memory before Eventmachine is given a

chance to run

The problem?

49

Build a new backend that uses NeverBlock instead of EventMachine

The solution?

50

Thin vs Mongrel (large static file, ~170MB)

10/40 20/40 40/400

50

100

150

200

250

300

350

MongrelThin NBThin

Concurrency / Requests

MB

/ Se

c

51

What about small files?

52

~200 Bytes ~1 KB ~30KB ~126KB0

5001000150020002500300035004000

MongrelThinThin NB

File Size

Req

uest

s / S

ec

Thin vs Mongrel (small static files)

53

What exactly do I get with NeverBlock?を使うことで何がで

きる？

?

54

● Concurrent Socket and Pipe I/O

並行 Socket & Pipe I/O● Concurrent File I/O (by delegating to threads)

並行な File I/O (スレッドへの委譲を使う )● Kernel#system, Kernel#sleep and Timeout.timeout

Supported Concurrency Featuresサポートされている並行性の特徴

55

● A Fiber pool class

Fiberプールクラス● A generic connection pool class

汎用コネクションプールクラス● A reactor backend

reactorバックエンド

Other Features (その他の特徴 )

56

● Net/HTTP● ActiveRecord● ActiveResource● Thin Web Server

Supported Librariesサポートしているライブラリ

57

Any performance figures or benchmarks?性能に関する数値や

ベンチマークは？

58

Servers with zero delay

1000/100000

2000

4000

6000

8000

10000

12000

BlockingForkingThreadedReactorNeverBlock


Req

uest

s / S

ec

59

Servers with 10ms delay

1000/100000

2000

4000

6000

8000

10000

12000



Req

uest

s / S

ec

60

Servers with 1 second delay

1000/10000

50100150200250300350400450



Req

uest

s / S

ec

61

Findings● Reactor pattern provides highest performance● NeverBlock is generally the second fastest● There is a room for more performance with faster fiber context switching

62

NeverBlock enables

Conclusiontransparent async event loops 透過的な非同期イベントループ

synchronization free threads同期フリーなマルチスレッド

automatic fiber scheduling自動的な fiberスケジューリングhigh I/O performanceハイ I/Oパフォーマンス

63

Wait,there is more

もう少し聞いて。まだある。

64

● Phusion Passenger support● Sequel Support● AIO support for file operations● epoll/kqueue support via ktools● Deeper integration with Ruby

Planned for NeverBlock

65

● Faster fiber switching● Using (set/get)context.● Wish for context switching performance similar to Erlang processes or Stackless Python tasklets

Wish list

66

● Reactor, a pure Ruby event loop● Arabesque, a new Ruby queuing library● Shihabd, the ultimate Rack server ● And more

Wanted to a share a few exciting projects undergoing at eSpaceeSpaceで現在進行中のエキサイティングなプロジェクトについて紹介したい

67

Thank Youشكرا�

ありがとう

● http://www.espace.com.eg/neverblock ● http://github.com/oldmoe/neverblock● http://github.com/oldmoe/mysqlplus● http://oldmoe.blogspot.com● [email protected]

http://github.com/oldmoe/neverblock

http://github.com/oldmoe/mysqlplus

http://oldmoe.blogspot.com/

18166746-NeverBlock-RubyKaigi2009

Documents