Binary deprotection with metasm - and stuffdslrouter.sourceforge.net/stuff/HTB/D1T2 - Alexander Gazet and Yoa… · Breaking obfuscation Breaking code virtualization Putting the pieces

Binary deprotection with metasmand stuff

Alexandre GazetSogeti / ESEC R&D

alexandre.gazet(at)sogeti.com

Yoann GuillotSogeti / ESEC R&D

yoann.guillot(at)sogeti.com

HITB 2009

MetasmAnalysis of a protection

Decompilation

DebuggerCompilerDisassembler

Plan

1 MetasmDebuggerCompilerDisassembler

BindingBacktracking

2 Analysis of a protection

3 Decompilation

A. Gazet, Y. Guillot Binary deprotection with metasm 2/55

http://www.esec.fr.sogeti.com/


Decompilation


Metasm

a pure ruby opensource framework

assembler/dissassembler

Ia32 (16/32/64bits), mipsEven supports cr7

debugger

linux, windows, remote

compiler/decompiler (more or less :)

GUI included !




Decompilation


Plan


BindingBacktracking

2 Analysis of a protectionBreaking obfuscationBreaking code virtualizationPutting the pieces togetherConclusion(s)

3 Decompilation




Decompilation


Debugger




Decompilation


Debugger




Decompilation


Features

Direct manipulation of the OS primitives

sys ptraceWaitForDebugEvent

Very fine & low-level control

Unified high-level interface

Linux, Windows, GDBserverConditionnal breakpoints, callback. . .




Decompilation


Plan


BindingBacktracking


3 Decompilation




Decompilation


C Compiler

Rudimentary C compiler

x86 only

Framework integration easy to leverage

Easy to customize e.g. dynamic symbol resolution




Decompilation


Plan


BindingBacktracking


3 Decompilation




Decompilation


Disassembler




Decompilation


Disassembly

The reference: IDA Pro

Excellent on unobfuscated binaries

Not so useful on protected code

No code interpretationStrong hypothesis

Hypothesis

Both branches are taken on a conditionnal jump

Two instructions never overlap

A subfunction call returns




Decompilation


Disassembly

The reference: IDA Pro

Excellent on unobfuscated binaries

Not so useful on protected code

No code interpretationStrong hypothesis

Hypothesis

Both branches are taken on a conditionnal jump

Two instructions never overlap

A subfunction call returns




Decompilation


Hypothesis: all call returns

. t e x t : 00403E9F loc 403E9F : ; CODE XREF: .text: loc_40CDEF

. t e x t : 00403E9F push ebp

. t e x t : 00403EA0 push ecx

. t e x t : 00403EA1 push ebp

. t e x t : 00403EA2 c a l l sub 40BECD

. t e x t : 00403EA7 outsb

. t e x t : 00403EA8 cmp edx , esp

. t e x t : 00403EAA push esp

. t e x t : 00403EAB i n c e s i




Decompilation


Failure

. t e x t : 0040BECD sub 40BECD proc near ; CODE XREF: .text :00403 EA2

. t e x t : 0040BECD cmp eax , ebp

. t e x t : 0040BECF add dword p t r [ esp +0] , 1

. t e x t : 0040BED4 t e s t ebx , 1E2h

. t e x t : 0040BEDA r e t n 0Ch

. t e x t : 0040BEDA sub 40BECD endp




Decompilation


Binding

Our solution:

Express instruction effects through symbolic expressions. Thisassociates semantics to each instruction.

Instruction ADD:

r e s = E x p r e s s i o n [ [ a [ 0 ] , :& , mask ] , :+ , [ a [ 1 ] , :& , mask ] ]b i n d i n g [ a [ 0 ] ] = r e sb i n d i n g [ : e f l a g z ] = E x p r e s s i o n [ [ res , :& , mask ] , :==, 0 ]b i n d i n g [ : e f l a g s ] = s i g n [ r e s ]b i n d i n g [ : e f l a g c ] = E x p r e s s i o n [ res , :> , mask ]b i n d i n g [ : e f l a g o ] = E x p r e s s i o n [ [ s i g n [ a [ 0 ] ] , :==, s i g n [ a [ 1 ] ] ] ,

: ’&&’ , [ s i g n [ a [ 0 ] ] , : ’!=’ , s i g n [ r e s ] ] ]




Decompilation


Binding

Our solution:

Express instruction effects through symbolic expressions. Thisassociates semantics to each instruction.

Instruction ADD:

r e s = E x p r e s s i o n [ [ a [ 0 ] , :& , mask ] , :+ , [ a [ 1 ] , :& , mask ] ]b i n d i n g [ a [ 0 ] ] = r e sb i n d i n g [ : e f l a g z ] = E x p r e s s i o n [ [ res , :& , mask ] , :==, 0 ]b i n d i n g [ : e f l a g s ] = s i g n [ r e s ]b i n d i n g [ : e f l a g c ] = E x p r e s s i o n [ res , :> , mask ]b i n d i n g [ : e f l a g o ] = E x p r e s s i o n [ [ s i g n [ a [ 0 ] ] , :==, s i g n [ a [ 1 ] ] ] ,

: ’&&’ , [ s i g n [ a [ 0 ] ] , : ’!=’ , s i g n [ r e s ] ] ]




Decompilation


Binding

Instruction CALL:

b i n d i n g [ : esp ] = E x p r e s s i o n [ : esp , :− , opsz ]b i n d i n g [ I n d i r e c t i o n [ : esp , 4 ] ] = d i . n e x t a d d r

For exemple:

dword p t r [ esp ] = 0x403EA7esp = esp−4

Instruction RDTSC:

b i n d i n g [ : eax ] = E x p r e s s i o n : : Unknownb i n d i n g [ : edx ] = E x p r e s s i o n : : Unknown




Decompilation


Binding

Instruction CALL:


For exemple:


Instruction RDTSC:





Decompilation


Binding

Instruction CALL:


For exemple:


Instruction RDTSC:





Decompilation


Backtracking, the theory

Definition

Symbolic emulation by walking the instruction flow backwards.




Decompilation


Backtracking, the facts

Execution flow:

c a l l l o c 4 0 b e c d h ; @403ea2h e826800000

[ . . . ]cmp eax , ebp ; @40becdh 39e8

add dword p t r [ esp +0] , 1 ; @40becfh 8344240001

t e s t ebx , 1e2h ; @40bed4h f7c3e2010000

r e t 0ch ; @40bedah c20c00

Backtracing x dword ptr [esp] for 40bedah ret 0ch

1 backtrace 40becfh dword ptr [esp] => dword ptr [esp]+1

2 backtrace up 40becdh->403ea2h dword ptr [esp]+1

3 backtrace 403ea2h dword ptr [esp]+1 => 403ea8h

4 backtrace result: 403ea8h




Decompilation


Metasm

Result:

l o c 4 0 3 e 9 f h :push ebp ; @403e9fh 55

push ecx ; @403ea0h 51

push ebp ; @403ea1h 55

c a l l l o c 4 0 b e c d h ; @403ea2h e826800000 noreturn

db 6eh ; @403ea7h

// X r e f s : 40bedahloc 403ea8h :

cmp edx , esp ; @403ea8h 39e2

push esp ; @403eaah 54

[ . . . ]

// X r e f s : 403 ea2hl o c 4 0 b e c d h :

cmp eax , ebp ; @40becdh 39e8

add dword p t r [ esp +0] , 1 ; @40becfh 8344240001

t e s t ebx , 1e2h ; @40bed4h f7c3e2010000

r e t 0ch ; @40bedah c20c00 x: loc_403ea8h




Decompilation

Breaking obfuscationBreaking code virtualizationPutting the pieces togetherConclusion(s)

Plan

1 Metasm


3 Decompilation




Decompilation


Code protection?

People want to be able to hide the functionning of their code

Using code obfuscationUsing code virtualizationBoth




Decompilation


Defeating code protection

Need to understand the virtual machine architecture

The VM interpreter is obfuscated

Must defeat code obfuscation

Generate a translator from virtual bytecode to easy to readcode

We’ll introduce a framework to assist is those steps




Decompilation


Previous work: T2 2007

On the fly deobfuscation with patterns

Virtual processor generator

Automatic semantics analysis




Decompilation


The target

A realworld virtualization based protection:

Some protected chunks of code are virtualizedVirtualized code (bytecode) is executed using an embeddedinterpreterEach virtual machine instance is unique (polymorphism)

Code is massively obfuscated

List of all the instruction handlers is trivial to get




Decompilation


Starting point

Need of automation

What we already have:

Filtering processorControl flow graph (CFG) walkingRewriting rules applicationOn-the-fly CFG modification

Manual analysis of obfuscated code

Trying to find patterns manually?

PainfulLoss of genericityPossibly ineffective: polymorphism




Decompilation


Plan


BindingBacktracking


3 Decompilation




Decompilation


Compiler approach

Our needs

Semantics preservation

Rewrite code into a simpler form

Dead code elimination

etc.

Compilers already do this kind of jobs: optimization

Our optimization criteria: code conciseness




Decompilation


Using compiler optimisation

Proposed approach

Addition of an optimization module to our CFGwalkthrough module

For each handler:1 Its code is recovered2 Then optimized




Decompilation


Constant propagation

c f h mov a l , 12h67h mov c l , 46h69h x o r c l , a l

c f h mov a l , 12h67h mov c l , 46h69h x o r c l , 12h

Figure: Propagation of 12h through al.




Decompilation


Constant folding

c f h mov a l , 12h67h mov c l , 46h69h x o r c l , 12h

c f h mov a l , 12h67h mov c l , 54h

Figure: cl value folding.




Decompilation


Operation folding

4 f h add a l , −7f h51h add a l , b l53h add a l , −70h

4 f h add a l , 11h51h add a l , b l

Figure: add operation folding.




Decompilation


Demonstration

Optimization of a handler




Decompilation


Next step

Now we are ready to analyse the virtual machine architecture




Decompilation


Plan


BindingBacktracking


3 Decompilation




Decompilation


At the beginning

2nd Futamura projection

Given two languages La and Lb, it is possible to find a compilerfrom Lb to La, if we know an interpreter of Lb written in La

Once again, compiler approach is the answer!

It’s only a proof of existence

How to generate it?




Decompilation


Semantical analysis of the handlers

code binding: method from the Disassembler object

Example of an optimized handler

l o d s dxo r eax , ebxadd eax , 859 f c f a e hsub ebx , eaxpush eax

Semantics (aka binding)

dword p t r [ esp ] := ( dword p t r [ e s i ] ˆ ebx )+859 f c f a e heax := ( dword p t r [ e s i ] ˆ ebx )+859 f c f a e hebx := ebx+−(dword p t r [ e s i ] ˆ ebx )−859 f c f a e he s i := e s i+4esp := esp−4




Decompilation


Semantical analysis

When a handler is encountered for the first time:

Its code is optimizedIts semantics is computed and stored

We progressively build the description of the semantics ofthe interpreter !

Given the current program state, we are able to emulate thenext step of bytecode.




Decompilation


From static to (almost) dynamic

Handler’s binding:

dword p t r [ esp ] := ( dword p t r [ e s i ] ˆ ebx )+859 f c f a e heax := ( dword p t r [ e s i ] ˆ ebx )+859 f c f a e hebx := ebx−(dword p t r [ e s i ] ˆ ebx )−859 f c f a e he s i := e s i+4esp := esp−4

Current context (partial)

eax := 93hebx := 0 fd8dhe s i := 100167 beh[ . . . ]

Contextualized binding:

dword p t r [ esp ] := 0c0000001heax := 0c0000001hebx := 4000 fd8che s i := 100167 c2hesp := esp−4




Decompilation


From static to (almost) dynamic (2)

Based on contextualized binding:

Generate corresponding assembly:

push 0c0000001h

Symbolic execution: compute context after execution of thecurrent handler

Follow bytecode execution flow

Support virtual calls and jumps (conditionnal or not)

⇒ Recover the whole chunk of code innative Ia32 assembly




Decompilation


Demonstration

Symbolic execution and assembly generation




Decompilation


Results

The whole chunk of bytecode is compiled, on-the-fly, intonative Ia32 assembly

Compiled bytecode itself is still obfuscated

Still many references to the virtual machine’s context

Looks like a stack automaton

Next steps

⇒ Re-use optimisation module + inject abstraction




Decompilation


Abstraction injection

Processor extension

l i s t = Reg . i t o s [ 3 2 ] . concat ( %w[ v i r t e a x ] )Reg . i t o s [ 1 6 ] . concat ( %w[ v i r t a x ] )Reg . i t o s [ 8 ] . concat ( %w[ v i r t a l ] )

Reg . s t o i . c l e a rReg . i t o s . each { | sz , hh |

hh . e a c h w i t h i n d e x { | r , i |Reg . s t o i [ r ] = [ i , s z ]

}}Reg : : Sym . r e p l a c e l i s t .map { | s | s . to sym }




Decompilation


Demonstration

Optimized chunk with virtual registers




Decompilation


Final pass

Injection of virtual registers

Applying optimizations ⇒Stack automaton aspect totally removedCode is expressed using virtual registers only

Virtual registers are then mapped back to native registers

Compilation and links edition

⇒ original, unprotected chunk of code is retrieved




Decompilation


Plan


BindingBacktracking


3 Decompilation




Decompilation


Demonstration

Unvirtualized code, mapped into theoriginal binary




Decompilation


Plan


BindingBacktracking


3 Decompilation




Decompilation


Conclusion(s) 1/2

Optimisation (rewriting rules)

Quite effectiveOur implementation is limited

Local optimizationsLake of an intermediate representationUnsuited to control flow obfuscation

Partial evaluation or specializationPre-computation of all the static elements:

Data transfers within the obfuscated codeApplication of the interpreter to the bytecode

Generic ApproachRequire relatively heavy computation




Decompilation


Conclusion(s) 2/2

Integration and re-usability of the deobfuscatorActual code is still a “prototype”Being integrated into the framework as a pluginUsable on x86 code, with some cross-platform parts




Decompilation

Plan

1 Metasm

2 Analysis of a protection

3 Decompilation




Decompilation

Advantages

Arch-specific code reduced to the minimum

C code is much more expressive than asm

Standard loops are simpler to handle

Semantics is often simpler

No side-effects on flags




Decompilation

Limitations

Some asm constructs are difficult to express in C

rol, rorjmp eax

Needs that the code has certain properties

Split in function/subfunctionsFollows C ABIs/calling conventions

Those last things can be worked around

Custom attribute




Decompilation

Demo

Decompilation




Decompilation

Reminder

Metasmhttps://metasm.cr0.org/

Bloghttp://esec.fr.sogeti.com/blog/



https://metasm.cr0.org/

http://esec.fr.sogeti.com/blog/


Decompilation

Conclusion

Thanks for listening.

Questions ?



Binary deprotection with metasm - and stuffdslrouter.sourceforge.net/stuff/HTB/D1T2 - Alexander Gazet and Yoa… · Breaking obfuscation Breaking code virtualization Putting the pieces

Documents