Statistical Deobfuscation for Android Applications · Statistical Deobfuscation for Android Applications Benjamin Bichsel Veselin Raychev Petar Tsankov Martin Vechev Department of

StatisticalDeobfuscation forAndroidApplications

BenjaminBichsel

VeselinRaychev

PetarTsankov

MartinVechev

DepartmentofComputerScience

WhyDe-obfuscate?

GooglePlay

Androidbinaries(APKs)(nocodeavailable)

NumberofAPKsonGooglePlay 2.4MAPKs

’10 ’12 ’14 ’16

LayoutObfuscationinAndroid

Obfuscate

Non-descriptivenames

Namesprovidekeysemanticinformation

package com.example.dbhelper

class DBHelper extends SQLiteHelper {SQLiteDatabase db;

public DBHelper(Context ctx) {db = getWritableDatabase();

}

Cursor execSQL(String str) {return db.rawQuery(str);

}}

package a.b.c

class a extends SQLiteHelper {SQLiteDatabase b;

public a(Context ctx) {b = getWritableDatabase();

}

Cursor c(String str) {return b.rawQuery(str);

}}

Somenamesremain


Obfuscate






}


}}

package a.b.c



}


}}

Somenamesremain

SecurityChallenges

CodeInspection

Third-partyLibraryDetection

… manyothers

LayoutObfuscationinAndroidNon-descriptive

names


package a.b.c



}


}}




}


}}

Somenamesremain

Canwereverselayoutobfuscation





}


}}

package a.b.c



}


}}


NamesprovidekeysemanticinformationYes,withroughly80%accuracy!

www.apk-deguard.com

www.apk-deguard.com

Releasedlastweek,sofar:>5Kusers >5GBAPKs

Redditposts/comments Tweets

... ...

HowDoesDeGuard Work?

DeGuard:SystemOverview

Staticanalysis TransformMAP

Inference

class a extends SQLiteHelper {SQLiteDatabase b;public a(Context ctx) {

b = getWritableDB();}

}

class DBHelper extends SQLiteHelper{SQLiteDatabase db;public DBHelper(Context ctx) {

db = getWritableDB();}

}

PredictionPhase

Open-source,unobfuscatedapplications

LearningPhase

Staticanalysis Training

Probabilisticmodel𝑃 )

Semanticrepresentation

ObfuscatedCode De-obfuscatedCode

ProbabilisticGraphicalModels

SQLiteHelper

getWritableDB

aextends

getsfield-in

b

ProbabilisticGraphicalModelsname1 name2 weight

𝑓% SQLiteHelper DBUtils 0.3𝑓& SQLiteHelper DBHelper 0.2

name1 name2 weight𝑓' getWritableDB db 0.7𝑓( getWritableDB instance 0.4

name1 name2 weight𝑓) DBUtils instance 0.5𝑓* DBHelper db 0.4𝑓+ … … …

Graph+featuresdefineaprobabilisticgraphicalmodel

𝑂 𝐾𝑃 ) == 𝑃 𝑎, 𝑏 𝑆𝑄𝐿𝑖𝑡𝑒𝐻𝑒𝑙𝑝𝑒𝑟, 𝑔𝑒𝑡𝑊𝑟𝑖𝑡𝑎𝑏𝑙𝑒𝐷𝐵)

= 1𝑍 exp(0.3 I 𝑓% 𝑆𝑄𝐿𝑖𝑡𝑒𝐻𝑒𝑙𝑝𝑒𝑟, 𝑎

+0.2 I 𝑓& 𝑆𝑄𝐿𝑖𝑡𝑒𝐻𝑒𝑙𝑝𝑒𝑟, 𝑎 +⋯ )

class a extends SQLiteHelper {SQLiteDatabase b;public a(Context ctx) {b = getWritableDB();

}}

𝑂 Unknownvariables

Knownvariables

𝑓%, 𝑓&, . .

𝐾

Featurefunctions

SQLiteHelper

getWritableDB

aextends

getsfield-in

b

ProbabilisticGraphicalModelsname1 name2 weight

𝑓% SQLiteHelper DBUtils 0.3𝑓& SQLiteHelper DBHelper 0.2

name1 name2 weight𝑓' getWritableDB db 0.7𝑓( getWritableDB instance 0.4

name1 name2 weight𝑓) DBUtils instance 0.5𝑓* DBHelper db 0.4𝑓+ … … …

Graph+featuresdefineaprobabilisticgraphicalmodel

𝑂 𝐾𝑃 ) == 𝑃 𝑎, 𝑏 𝑆𝑄𝐿𝑖𝑡𝑒𝐻𝑒𝑙𝑝𝑒𝑟, 𝑔𝑒𝑡𝑊𝑟𝑖𝑡𝑎𝑏𝑙𝑒𝐷𝐵)

= 1𝑍 exp(0.3 I 𝑓% 𝑆𝑄𝐿𝑖𝑡𝑒𝐻𝑒𝑙𝑝𝑒𝑟, 𝑎

+0.2 I 𝑓& 𝑆𝑄𝐿𝑖𝑡𝑒𝐻𝑒𝑙𝑝𝑒𝑟, 𝑎 +⋯ )


}}

𝑂 Unknownvariables

Knownvariables

𝑓%, 𝑓&, . .

𝐾

Featurefunctions

NextHowaretheweightsandfeatureslearned?

Learning

Learning

UnobfuscatedAPKs

name1 name2 weight𝑓% SQLiteHelper DBUtils 0.3𝑓& SQLiteHelper DBHelper 0.2𝑓' getWritableDB db 0.7𝑓( getWritableDB instance 0.4𝑓) DBUtils instance 0.5𝑓* DBHelper db 0.4𝑓+ … … …

name1 name2𝑓% SQLiteHelper DBUtils𝑓& SQLiteHelper DBHelper𝑓' getWritableDB db𝑓( getWritableDB instance𝑓) DBUtils instance𝑓* DBHelper db𝑓+ … …

Computeweightsthatmaximize𝑃 𝑂 = 𝑜N 𝐾 = 𝑘N foralltrainingsamples(𝑜N, 𝑘N)

Staticanalysis

TrainModel

Featuretemplates

Features(withcandidatenames)

Dependencygraphs

28templates

Actualgraphshave>1,000nodes

>2,000

>100,000



Inference



}



}

PredictionPhase


LearningPhase






Inference



}



}

PredictionPhaseObfuscatedCode De-obfuscatedCode


ObfuscatedCode

SQLiteHelper

getWritableDB

aextends

getsfield-in

b

PredictionPhasename1 name2 weightSQLiteHelper DBUtils 0.3SQLiteHelper DBHelper 0.2

name1 name2 weightgetWritableDB db 0.7getWritableDB instance 0.4

name1 name2 weightDBUtils instance 0.5DBHelper db 0.4DBUtils db 0.2DBHelper instance 0.2


}}

Staticanalysis

ObfuscatedCode

SQLiteHelper

getWritableDB

aextends

getsfield-in

b





}}

Programanalysis

MAPInference

Candidate assignment𝒐 𝑷 𝒐 𝒌)*a =DBUtils b =instance 1.2a =DBHelper b =db 1.3a =DBUtils b =db 0.8a =DBHelper b =instance 1.2

*Non-normalized

�⃗� = 𝑎𝑟𝑔𝑚𝑎𝑥𝑃 𝑂 = �⃗�′ 𝐾 = 𝑘�⃗�′ ∈ Ω

ObfuscatedCode

SQLiteHelper

getWritableDB

aextends

getsfield-in

b





}}

Programanalysis

MAPInference

Candidate assignment𝒐 𝑷 𝒐 𝒌)*a =DBUtils b =instance 1.2a =DBHelper b =db 1.3a =DBUtils b =db 0.8a =DBHelper b =instance 1.2

*Non-normalized

�⃗� = 𝑎𝑟𝑔𝑚𝑎𝑥𝑃 𝑂 = �⃗�′ 𝐾 = 𝑘�⃗�′ ∈ Ω

ObfuscatedCode

SQLiteHelper

getWritableDB

DBHelperextends

getsfield-in

db





}}

Staticanalysis

Deobfuscated Code

class DBHelper extends SQLiteHelper {SQLiteDatabase db; public DBHelper(Context ctx) {db = getWritableDB();

}}

Transform

PreservingSemantics

class Aint aObject bvoid a()

class B extends

Avoid b()void c(A a)

Syntacticconstraintse.g.fieldswithinaclassmusthavedistinctnames

Semanticconstraintse.g.methodoverloadsmustbepreserved

Freelyrenamingfields/variables/methodsmaychange theprogramsemantics

musthavedistinctnames

musthavedistinctnames

mustnotoverride

methoda()



Inference



}



}

PredictionPhase


LearningPhase



DeGuard Implementation

DeGuard Implementation

www.apk-deguard.com

§ StaticanalysisframeworkforJavaandAndroid

StaticAnalysis

LearningandMAPInference§ Scalableopen-sourceframework

forstructuredprediction§ Open-source:http://nice2predict.org

§ Trainingdata:2Kopen-source,unobfuscated Androidapplications

1. CanDeGuard reverseProGuard?2. CanDeGuard detectthird-partylibraries?3. IsDeGuard usefulformalwareinspection?

EvaluationEvaluation

ProGuard Experiment

SourceCode

ObfuscatedAPK De-obfuscatedAPK

Non-obfuscatedAPK =?

AfterObfuscation

Fields Methods Classes Packages Total

20

40

60

80

100

0

%ofprogramelements

only13%knownnames

Knownnames

CanDeGuard reverseProGuard?

Fields Methods Classes Packages Total

20

40

60

80

100

0

%ofprogramelements

Knownnames

Correctlypredictednames

Mis-predictednames

Packagenamesaredirectlyusedto

predictthird-partylibraries

1.6%knownnames

80.6%correctnames

80%ofthenamesareidenticaltotheoriginalones

i.e.,identicaltotheoriginalnames

CanDeGuard DetectThird-PartyLibraries?

LibraryCode

SourceCode

ObfuscatedAPK

ProGuardobfuscateslibrarypackagenames

De-obfuscatedAPK

?

Precision:93.1%Recall:91%

ProGuard

IsDeGuard UsefulforMalwareInspection?

class d {String a = System.getProperty(..)char[] b;byte [] c;byte[] a(String) {}

}

class Base64 {String NL = System.getProperty(..)char[] ENC;byte [] DEC;byte[] decode(String) {}

}

De-obfuscatingsamplesfromtheAndroidMalwareGenomeProject

MalwareSample De-obfuscatedMalwareSample

Base64Decoder

Revealsstringdecoders

Revealsclassesthathandlesensitivedata(e.g.Location)

Hardtohandleheavily-obfuscatedcode(e.g.reflection)



public DBHelper(Context ctx) {db = getWritableDB();

}


package a.b.c


public a(Context ctx) {b = getWritableDB();

}


Tryonline:www.apk-deguard.com Fields Methods Classes Packages Total

20

40

60

80

100

0

SQLiteHelper

getWritableDB

a

b

name1 name2 weight

SQLiteHelper DBUtils 0.3

SQLiteHelper DBHelper 0.2

name1 name2 weight

getWritableDB db 0.7

getWritableDB instance 0.4

ProbabilisticModels

HighPredictionAccuracy

Moreinfo:http://www.srl.inf.ethz.ch/spas

Summary

Statistical Deobfuscation for Android Applications · Statistical Deobfuscation for Android Applications Benjamin Bichsel Veselin Raychev Petar Tsankov Martin Vechev Department of

Documents