Berlin,2019 - Firebird · Fastbackup/restore.Conclusions. Index activation time Restore time B/R time Backup time Index activation time B/R time MAX(Backup, Restore) time 10xfasterindex

Berlin, 2019

Firebird Conference 2019Berlin, 17-19 October

Contents

3/36 | www.red-soft.ru |

Description of the loaded system

Experience

5/36 | www.red-soft.ru |

Short history of IDES

6/36 | www.reddatabase.ru |

Requirements to IDES

7/36 | www.red-soft.ru |

Architecture of IDES

8/36 | www.reddatabase.ru |

A role of the segments

9/36 | www.red-soft.ru |

Example of heavy loaded segment

10/36 | www.reddatabase.ru |

Tricks

General tuning

12/36 | www.reddatabase.ru |

Limits

| www.red-soft.ru |13/36

Binding processes to physical CPU

14/36 | www.reddatabase.ru |

How to find a problem with NUMA

| www.red-soft.ru |15/36

# numastatnode0 node1 node2 node3

numa_hit 35483991701 25907973778 43439858254 23399971622numa_miss 13235884421 16606874744 16180699231 18803917332numa_foreign 17220754371 11119698073 24867593917 11619329374interleave_hit 53690 53720 53709 53739local_node 35483981570 25907909025 43439790189 23399907916other_node 13235894552 16606939497 16180767296 18803981038

Example (numastat)

# numastatnode0 node1 node2 node3

numa_hit 224300115716 293940768830 289461636335 296555251853numa_miss 516913107 380035968 451756983 486206474numa_foreign 371950817 457785062 488683152 516493501interleave_hit 53615 53592 53603 53624local_node 224299280174 293940256731 289461093452 296554599720other_node 517748649 380548067 452299866 486858607

16/36 | www.reddatabase.ru |

Example (graphics)

| www.red-soft.ru |17/36

Fast backup/restore. Method.

18/36 | www.reddatabase.ru |

Fast backup/restore. Results.

| www.red-soft.ru |19/36

Fast backup/restore. Conclusions.

IndexactivationtimeRestoretimeB/Rtime Backuptime

IndexactivationtimeB/Rtime

MAX(Backup,Restore)time

10x faster indexactivation3x faster B/R

20/36 | www.reddatabase.ru |

gbak/nbackup via fifo

| www.red-soft.ru |21/36

System improvements

Snapshot request & Garbage collection

| www.red-soft.ru |

More details in Vlad's talkshttps://firebirdsql.org/file/community/conference-2016/statement-level-read-consistency.pdfNew transactions' features and changes in garbage collection in Firebird 4

23/36

Improvements in restore via GBAK

24/36 | www.reddatabase.ru |

Improvements in restore via GBAK. Example.

| www.red-soft.ru |

create procedure P1_ROUND (NUM float)returns (RES float)asbeginRES = round(NUM);suspend;end^create procedure P2asdeclare variable VAR float;beginVAR = (select RES from P1_ROUND(4.4564));end^alter procedure P1_ROUND (NUM float, SCALEinteger)returns (RES float)asbeginRES = round(NUM, SCALE);suspend;end^

gbak:restoring stored procedure P1_ROUND...gbak:restoring stored procedure P2...gbak:committing metadatagbak: ERROR:invalid request BLR at offset 50gbak: ERROR: Input parameter mismatch for procedure P1_ROUND...gbak:Exiting before completion due to errorsgbak:stopped at Thu Oct 3 11:00:11 2019

gbak:restoring stored procedure P1_ROUND...gbak:restoring stored procedure P2...gbak:committing metadatagbak: ERROR:Error while parsing procedure P2's BLRgbak: ERROR: invalid request BLR at offset 50gbak: ERROR: Input parameter mismatch for procedure P1_ROUND...gbak: WARNING:Database is not online due to failure to restore one ormore objects.gbak: WARNING:Run gfix -online to bring database online.gbak:stopped at Tue Oct 8 16:48:00 2019

25/36

Improvements in restore via GBAK. Fixing.select RDB$PROCEDURE_SOURCE from RDB$PROCEDURES where RDB$PROCEDURE_BLR is NULL;select RDB$FUNCTION_SOURCE from RDB$FUNCTIONS where RDB$FUNCTION_BLR is NULL;select RDB$TRIGGER_SOURCE from RDB$TRIGGERS where RDB$TRIGGER_BLR is NULL;...ALTER PROCEDURE/FUNCTION/TRIGGER ...

26/36 | www.reddatabase.ru |

Improvements in fb_lock_print

| www.red-soft.ru |27/36

While it haspending request

Dead owner search algorithm

grantedrequestlock requestowner pendingrequest

Themostpendinglock

28/36 | www.reddatabase.ru |

Binary trace file

| www.red-soft.ru |

• All events• The fastest way• Low overhead

• Big file

29/36

Binary trace file. Usage.

30/36 | www.reddatabase.ru |

Binary trace file. Example.

| www.red-soft.ru |31/36

Binary trace file. Result.SQL SUM AVG COUNT

select sum(pl.delta_amt) amt, pl.submit_date carry_date,doc.doc_date, doc.doc_number, doc.opertype_id, 549207 61023 9select sum(bh.day_credit) as Financing from expbudget expjoin budgrest br on (br.budgetline_id=exp.budgetline_id)join account a on 443561 62 7042

select sum(bh.day_credit) as Financing from expbudget expjoin budgrest br on (br.budgetline_id=exp.budgetline_id)join account a on 427189 71198 1

32/36 | www.reddatabase.ru |

Load in a laboratory - big challenge

| www.red-soft.ru |35/36

Questions?