Page 1
#WWDC18
© 2018 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
Jim Grosbach, Senior Manager Platform Technologies Alex Lorenz, Clang Frontend Engineer George Karpenkov, Program Analysis Engineer Ahmed Bougacha, Compiler Backend Engineer
•What’s New in LLVM •Session 409
Page 2
Family of Tools
•Clang •Static Analyzer •Sanitizers •LLDB •GPU shader compilers •
Page 3
Family of Tools
•Clang •Static Analyzer •Sanitizers •LLDB •GPU shader compilers •Swift
Page 4
LLVM Open Source
https://llvm.org
Page 6
Partnership
Apple
Google
Qualcomm
Mozilla
Sony
ARM
Intel
Facebook
Page 7
Partnership
And many more!
Page 8
Participate! https://llvm.org
Page 9
Agenda
•Updates on ARC •New diagnostics in Xcode 10 •Clang Static Analyzer •Increased security •New instruction set extensions
Page 10
Alex Lorenz, Clang Frontend Engineer
•Updates on ARC •
Page 11
ARC object pointers in C structures!
Page 12
ARC forbids Objective-C objects in struct
typedef struct { NSString *name; NSNumber *price; } MenuItem;
Page 13
typedef struct { NSString *name; NSNumber *price; } MenuItem;
Page 14
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
Page 15
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] }; orderMenuItem(item); }x
Page 16
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] };
orderMenuItem(item); }x
Page 17
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] }; // [item.name retain];
orderMenuItem(item); }x
Page 18
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] }; // [item.name retain]; // [item.price retain]; orderMenuItem(item); }x
Page 19
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] }; // [item.name retain]; // [item.price retain]; orderMenuItem(item);
}x
Page 20
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] }; // [item.name retain]; // [item.price retain]; orderMenuItem(item); // [item.name release];
}x
Page 21
// ARC Object Pointers in C Structs!
typedef struct { NSString *name; NSNumber *price; } MenuItem;
void orderFreeFood(NSString *name) { MenuItem item = { name, [NSNumber numberWithInt:0] }; // [item.name retain]; // [item.price retain]; orderMenuItem(item); // [item.name release]; // [item.price release]; }x
Page 22
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem;
Page 23
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = malloc(10 * sizeof(MenuItem)); orderMenuItems(items, 10); free(items); }x
Page 24
ARC pointers are not zero-initialized
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = malloc(10 * sizeof(MenuItem)); orderMenuItems(items, 10); free(items); }x
Page 25
ARC pointers are not zero-initialized
ARC pointers are not cleared before deallocation
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = malloc(10 * sizeof(MenuItem)); orderMenuItems(items, 10); free(items); }x
Page 26
ARC pointers are not zero-initialized
ARC pointers are not cleared before deallocation
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = orderMenuItems(items, 10); free(items); }x
malloc(10 * sizeof(MenuItem));
Page 27
ARC pointers are not zero-initialized
ARC pointers are not cleared before deallocation
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = orderMenuItems(items, 10); free(items); }x
Page 28
ARC pointers are not cleared before deallocation
ARC pointers are not zero-initialized
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = orderMenuItems(items, 10); free(items); }x
Page 29
ARC pointers are not cleared before deallocation
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = orderMenuItems(items, 10); free(items); }x
calloc(10, sizeof(MenuItem));
Page 30
ARC pointers are not cleared before deallocation
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = calloc(10, sizeof(MenuItem)); orderMenuItems(items, 10); // ARC Object Pointer Fields Must be Cleared Before Deallocation for (size_t i = 0; i < 10; ++i) { items[i].name = nil; items[i].price = nil; } free(items); }x
Page 31
// Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct { NSString *name; NSNumber *price; } MenuItem; void testMenuItems() { // Allocate an array of 10 menu items MenuItem *items = calloc(10, sizeof(MenuItem)); orderMenuItems(items, 10); // ARC Object Pointer Fields Must be Cleared Before Deallocation for (size_t i = 0; i < 10; ++i) { items[i].name = nil; items[i].price = nil; } free(items); }x
Page 32
Objective-C Objects in Structs
Page 33
Objective-C Objects in Structs
Structs with Objective-C object fields interoperate across language modes
Page 34
Objective-C Objects in Structs
Structs with Objective-C object fields interoperate across language modes
Unified Objective-C++ ABI between ARC and manual retain/release (MRR)
Page 35
Objective-C Objects in Structs
Structs with Objective-C object fields interoperate across language modes
Unified Objective-C++ ABI between ARC and manual retain/release (MRR)• ABI change for some functions passing or returning structs by value
Page 36
Objective-C Objects in Structs
Structs with Objective-C object fields interoperate across language modes
Unified Objective-C++ ABI between ARC and manual retain/release (MRR)• ABI change for some functions passing or returning structs by value
When a returned or passed struct has:
ARC object pointer fields and no special members (constructors, destructor)
Page 37
Objective-C Objects in Structs
Structs with Objective-C object fields interoperate across language modes
Unified Objective-C++ ABI between ARC and manual retain/release (MRR) • ABI change for some functions passing or returning structs by value
Swift does not support importing structs with ARC object pointer fields
Page 38
•New Diagnostics in Xcode 10
Page 39
Swift and Objective-C Interoperability
Page 40
Swift and Objective-C Interoperability
Swift code can be imported into Objective-C
Page 41
Swift and Objective-C Interoperability
Swift code can be imported into Objective-C
Xcode generates a header that exposes Swift interfaces
Page 42
Importing Swift Closures into Objective-C
Page 43
Importing Swift Closures into Objective-C
@objc protocol Executor { func performOperation(handler: () -> Void) }
Page 44
Importing Swift Closures into Objective-C
@objc protocol Executor { func performOperation(handler: () -> Void) }
Swift: Closure function arguments are non-escaping by default
Page 45
Importing Swift Closures into Objective-C
@objc protocol Executor { func performOperation(handler: () -> Void) }
Swift: Closure function arguments are non-escaping by default• Should not be called after the function returns
Page 46
Importing Swift Closures into Objective-C
@objc protocol Executor { func performOperation(handler: () -> Void) }
Swift: Closure function arguments are non-escaping by default• Should not be called after the function returns
Easy to forget when conforming to the protocol in Objective-C
Page 47
Importing Swift Closures into Objective-C
@objc protocol Executor { func performOperation(handler: () -> Void) }
#import “Executor-Swift.h” @interface DispatchExecutor : NSObject<Executor> - (void) performOperation:(void (^)(void)) handler; @end
Swift: Closure function arguments are non-escaping by default• Should not be called after the function returns
Easy to forget when conforming to the protocol in Objective-C
Page 48
#import “Executor-Swift.h” @interface DispatchExecutor : NSObject<Executor> - (void) performOperation:(void (^)(void)) handler; @end
Page 49
Parameter must be annotated with noescape
// Warning for Missing Noescape Annotations for Method Overrides
#import “Executor-Swift.h” @interface DispatchExecutor : NSObject<Executor> - (void) performOperation:(void (^)(void)) handler; @end
Page 50
// Warning for Missing Noescape Annotations for Method Overrides
#import “Executor-Swift.h” @interface DispatchExecutor : NSObject<Executor> - (void) performOperation:(NS_NOESCAPE void (^)(void)) handler; @end
Page 51
// Warning for Missing Noescape Annotations for Method Overrides
#import “Executor-Swift.h” @interface DispatchExecutor : NSObject<Executor> - (void) performOperation:(NS_NOESCAPE void (^)(void)) handler; @end
@implementation DispatchExecutor - (void) performOperation:(NS_NOESCAPE void (^)(void)) handler { }x @end
Page 52
// Warning for Missing Noescape Annotations for Method Overrides
#import “Executor-Swift.h” @interface DispatchExecutor : NSObject<Executor> - (void) performOperation:(NS_NOESCAPE void (^)(void)) handler; @end
@implementation DispatchExecutor - (void) performOperation:(NS_NOESCAPE void (^)(void)) handler { // Programmer must ensure that handler is not called after performOperation returns }x @end
Page 53
Packing Struct Members with #pragma pack
struct Struct { uint8_t a, b; // 2 byte padding uint32_t c; };
Page 54
Packing Struct Members with #pragma pack
struct Struct { uint8_t a, b; uint32_t c; };
Page 55
Packing Struct Members with #pragma pack
struct PackedStruct { uint8_t a, b; uint32_t c; };
Page 56
Packing Struct Members with #pragma pack
#pragma pack (push, 1) struct PackedStruct { uint8_t a, b; uint32_t c; };
#pragma pack (pop)
Page 57
Packing Struct Members with #pragma pack
#pragma pack (push, 1) struct PackedStruct { uint8_t a, b; uint32_t c; }; // The programmer forgot #pragma pack (pop)
Page 58
Finding Unbalanced #pragma pack Directives
#pragma pack (push, 1) struct PackedStruct { uint8_t a, b; uint32_t c; }; // The programmer forgot #pragma pack (pop)
Page 59
Finding Unbalanced #pragma pack Directives
Unterminated ‘#pragma pack (push, …)’ at end of file#pragma pack (push, 1) struct PackedStruct { uint8_t a, b; uint32_t c; }; // The programmer forgot #pragma pack (pop)
Page 60
Finding Unbalanced #pragma pack Directives
Unterminated ‘#pragma pack (push, …)’ at end of file#pragma pack (push, 1) struct PackedStruct { uint8_t a, b; uint32_t c; }; #pragma pack (pop)
Page 61
Finding Unbalanced #pragma pack Directives
#pragma pack (push, 1) struct PackedStruct { uint8_t a, b; uint32_t c; }; #pragma pack (pop)
Page 62
George Karpenkov, Program Analysis Engineer
•Clang Static Analyzer
Page 63
Clang Static Analyzer Finds edge-case, hard-to-reproduce bugs
Page 64
Static Analyzer Improvements
•Grand Central Dispatch performance anti-pattern
•Autoreleasing variables outliving autorelease pool
•Improved performance and report visualizations
NEW
Page 65
•Grand Central DispatchPerformance Anti-Pattern
Page 66
__block NSString *taskName = nil; dispatch_semaphore_t sema = dispatch_semaphore_create(0); [self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { taskName = task; dispatch_semaphore_signal(sema); }]; dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER); return taskName;
Performance implications: slowdown and hangsGrand Central Dispatch Performance Anti-Pattern
Page 67
__block NSString *taskName = nil; dispatch_semaphore_t sema = dispatch_semaphore_create(0); [self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { taskName = task; dispatch_semaphore_signal(sema); }]; dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER); return taskName;
Performance implications: slowdown and hangsGrand Central Dispatch Performance Anti-Pattern
Page 68
__block NSString *taskName = nil; dispatch_semaphore_t sema = dispatch_semaphore_create(0); [self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { taskName = task; dispatch_semaphore_signal(sema); }]; dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER); return taskName;
Performance implications: slowdown and hangsGrand Central Dispatch Performance Anti-Pattern
Page 69
__block NSString *taskName = nil; dispatch_semaphore_t sema = dispatch_semaphore_create(0); [self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { taskName = task; dispatch_semaphore_signal(sema); }]; dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER); return taskName;
Performance implications: slowdown and hangsGrand Central Dispatch Performance Anti-Pattern
Page 70
Blocks current thread by the execution on a different queue
The task queue usually has lower priority, leads to priority inversion
Spawns useless threads
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
dispatch_semaphore_signal(sema);}];dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
__block NSString *taskName = nil;
return taskName;
:^(NSString *task) { taskName = task;
Performance implications: slowdown and hangsGrand Central Dispatch Performance Anti-Pattern
[self.connection.remoteObjectProxy requestCurrentTaskName
Page 71
Blocks current thread by the execution on a different queue
The task queue usually has lower priority, leads to priority inversion
Spawns useless threads
Waiting on a callback
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
dispatch_semaphore_signal(sema);}];dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
__block NSString *taskName = nil;
return taskName;
:^(NSString *task) { taskName = task;
Performance implications: slowdown and hangsGrand Central Dispatch Performance Anti-Pattern
[self.connection.remoteObjectProxy requestCurrentTaskName
Page 72
^(NSError *error) { NSLog(@“Error: %@“, error); }]; [remoteObjectProxy requestCurrentTaskName
}];
__block NSString *taskName = nil;
return taskName;
:^(NSString *task) { taskName = task;
Grand Central Dispatch Performance Anti-Pattern
id remoteObjectProxy = [self.connection synchronousRemoteObjectProxyWithErrorHandler:
Page 73
If available, use synchronous version of the API
^(NSError *error) { NSLog(@“Error: %@“, error); }]; [remoteObjectProxy requestCurrentTaskName
}];
__block NSString *taskName = nil;
return taskName;
:^(NSString *task) { taskName = task;
Grand Central Dispatch Performance Anti-Pattern
id remoteObjectProxy = [self.connection synchronousRemoteObjectProxyWithErrorHandler:
Page 74
[self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { completionHandler(task); }];
Grand Central Dispatch Performance Anti-Pattern
Page 75
[self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { completionHandler(task); }];
Alternatively, use the API in an asynchronous manner
Grand Central Dispatch Performance Anti-Pattern
Page 76
Detecting Grand Central Dispatch Anti-Pattern Enabling check in build settings
Page 77
•Autoreleasing Variables Outliving Autorelease Pool
Page 78
Autoreleasing Variables and Autoreleasing Pools Using Automatic Reference Counting
__autoreleasing NSError *err = }.
Autoreleasing qualifier: values released on exit from the autorelease pool
code:1 userInfo:nil]; @autoreleasepool {
[NSError errorWithDomain:@"domain"
Page 79
Out parameters are implicitly __autoreleasing
A subsequent read of error parameter will crash
- (void) validateProperty:(NSError **) error { // implicitly __autoreleasing if (error) *error = } }.
code:1 userInfo:nil];
@autoreleasepool {
[NSError errorWithDomain:@"domain"
Page 80
Out Parameters and Autorelease Pools
Out parameters are implicitly __autoreleasing
A subsequent read of error parameter will crash
- (void) validateProperty:(NSError **) error { // implicitly __autoreleasing if (error) *error = } }.
A common source of use-after-free crashes
code:1 userInfo:nil];
@autoreleasepool {
[NSError errorWithDomain:@"domain"
Page 81
Out Parameters and Autorelease Pools
Out parameters are implicitly __autoreleasing
A subsequent read of error parameter will crash
- (void) validateProperty:(NSError **) error { // implicitly __autoreleasing if (error) *error = } }.
A common source of use-after-free crashes
code:1 userInfo:nil];
@autoreleasepool {
[NSError errorWithDomain:@"domain"
Page 82
Out Parameters and Autorelease Pools
Out parameters are implicitly __autoreleasing
A subsequent read of error parameter will crash
- (void) validateProperty:(NSError **) error { // implicitly __autoreleasing if (error) *error = } }.
A common source of use-after-free crashes
code:1 userInfo:nil];
@autoreleasepool {
[NSError errorWithDomain:@"domain"
Page 83
Capturing Autoreleasing Variables in a Block Dangerous when library uses autorelease pools
-enumerateObjectsUsingBlock: runs the block inside the autorelease pool
A subsequent read of the error parameter may crash
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { if (error) *error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; }
Page 84
Capturing Autoreleasing Variables in a Block Dangerous when library uses autorelease pools
-enumerateObjectsUsingBlock: runs the block inside the autorelease pool
A subsequent read of the error parameter may crash
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { if (error) *error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; }
Page 85
Capturing Autoreleasing Variables in a Block Dangerous when library uses autorelease pools
-enumerateObjectsUsingBlock: runs the block inside the autorelease pool
A subsequent read of the error parameter may crash
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { if (error) *error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; }
Page 86
Capturing Autoreleasing Variables in a Block Dangerous when library uses autorelease pools
-enumerateObjectsUsingBlock: runs the block inside the autorelease pool
A subsequent read of the error parameter may crash
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { if (error) *error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; }
Page 87
Xcode 10: Check for __autoreleasing Misuse Common source of use-after-free
Xcode 9: introduced a compiler warning requiring explicit __autoreleasing
Xcode 10: more targeted static analyzer warning
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { if (error) *error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; }
Page 88
Xcode 10: Check for __autoreleasing Misuse Common source of use-after-free
Xcode 9: introduced a compiler warning requiring explicit __autoreleasing
Xcode 10: more targeted static analyzer warning
Write to autoreleasing out parameter inside autorelease pool
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { if (error) *error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; }
Page 89
Capturing Autoreleasing Variables Write to strong variables first
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { __block NSError *localError; [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { localError = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; if (error) { *error = localError; } }
Page 90
Capturing Autoreleasing Variables Write to strong variables first
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { __block NSError *localError; [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { localError = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; if (error) { *error = localError; } }
Page 91
Capturing Autoreleasing Variables Write to strong variables first
- (void) findProblems:(NSArray *)arr error:(NSError *__autoreleasing*) error { __block NSError *localError; [arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) { if ([value isEqualToString:@"problem"]) { localError = [NSError errorWithDomain:@"domain" code:1 userInfo:nil]; } }]; if (error) { *error = localError; } }
Page 92
•Improved Performanceand Report Visualizations
Page 93
Improved Performance and Report Visualizations
Explores your program in a more efficient way
15% average increase in number of found bugs
Within the same analysis time
Page 94
Xcode 9: Unnecessary Long Error Paths
Page 95
Xcode 10: Improved Error Reporting
Page 96
Use Clang Static Analyzer Finds your bugs before your users do
Page 97
Use Clang Static Analyzer Finds your bugs before your users do
Page 98
Ahmed Bougacha, Compiler Backend Engineer
•Increased Security
Page 99
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
Page 100
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
Page 101
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
Page 102
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
Page 103
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
Page 104
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
Page 105
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
NULL
Page 106
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
dlog("Hello")
NULL
Page 107
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
dlog("Hello")
NULL
Page 108
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
dlog("Hello")
NULL
Return Address
Page 109
// The Stack: A Refresher
int dlog(const char *str) { ... }
int main(void) { dlog("Hello"); ... }
dlog("Hello")
NULL
Local Variables
Return Address
Page 110
NULL
Local Variables
Return Address
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; ... }
dlog("Hello")
Page 111
NULL
Return Address
logfile_path
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; ... }
dlog("Hello")
Page 112
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; logfile_path = dlogfile(); ... }
const char *dlogfile() { ... }
NULL
Local Variables
Return Address
dlog("Hello")
Page 113
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; logfile_path = dlogfile(); ... }
const char *dlogfile() { ... }
NULL
Local Variables
Return Address
dlog("Hello")
dlogfile()
Page 114
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; logfile_path = dlogfile(); ... }
const char *dlogfile() { ... }
NULL
Local Variables
Return Address
Local Variables
Return Address
dlog("Hello")
dlogfile()
Page 115
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; logfile_path = dlogfile(); ... }
const char *dlogfile() { ... }
NULL
Local Variables
Return Address
dlog("Hello")
dlogfile()
Page 116
// The Stack: A Refresher
int dlog(const char *str) { char *logfile_path; logfile_path = dlogfile(); ... }
const char *dlogfile() { ... }
NULL
Local Variables
Return Address
dlog("Hello")
Page 117
NULL
Local Variables
Return Address
dlog("Hello")
Page 119
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
dlog("Hello")
Page 120
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
dlog("Hello")
Page 121
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
dlog("Hello")
Page 122
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
dlog("Hello")
Page 123
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
buffer[1] = 'e';
dlog("Hello")
Page 124
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
dlog("Hello")
Page 125
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';dlog("Hello")
Page 126
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
dlog("Hello")
Page 127
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
??? = '\0';
dlog("Hello")
Page 128
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
??? = '\0';
dlog("Hello")💥💥
Page 129
// Stack Buffer Overflows: The Fix
int dlog(const char *str) { char buffer[4]; ... strncpy(buffer, str, sizeof(buffer)); ... }
NULL
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
??? = '\0';
dlog("Hello")💥💥
Page 130
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }.
NULL
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
??? = '\0';
dlog("Hello")💥💥
Page 131
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... }.
NULL
Local Variables
Return Address
dlog("Hello")Canary
Page 132
NULL
Return Address
dlog("Hello")
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... // if (canary is invalid) // abort(); return ...; }.
Page 133
NULL
Return Address
dlog("Hello")
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... // if (canary is invalid) // abort(); return ...; }.
Page 134
NULL
Return Address
dlog("Hello")
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
??? = '\0';
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... // if (canary is invalid) // abort(); return ...; }.
Page 135
NULL
Return Address
dlog("Hello")
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
buffer[3] = 'l';
??? = 'o';
??? = '\0';
❌
❌
// Stack Buffer Overflows
int dlog(const char *str) { char buffer[4]; ... strcpy(buffer, str); ... // if (canary is invalid) // abort(); return ...; }.
Page 137
Stack Protector
Detects stack buffer overflows
Page 138
Stack Protector
Detects stack buffer overflows
Already enabled by default
Page 139
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
NULL
Local Variables
Return Address
Canary
Page 140
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
NULL
Local Variables
Return Address
Canary
Page 141
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
NULL
Local Variables
Return Address
Canary
Page 142
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
NULL
Local Variables
Return Address
Canary
Page 143
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
NULL
Local Variables
Return Address
dlog("Hello", 15000)
Canary
Page 144
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
NULL
Local Variables
Return Address
dlog("Hello", 15000)
Canary
Page 145
// Stack Allocation: Variable Length Arrays
int dlog(const char *str, size_t len) { char buffer[len]; ... }.
dlog("Hello", 15000)
Page 146
// Stack Allocation: Pages
int dlog(const char *str, size_t len) { char buffer[len]; ... }
dlog("Hello", 15000)Page
Page
Page 147
// Stack Allocation: Pages
int dlog(const char *str, size_t len) { char buffer[len]; ... }
dlog("Hello", 15000)Page
Page
buffer[10000] = 0;
Page 148
// Stack Allocation: Pages
int dlog(const char *str, size_t len) { char buffer[len]; ... }
dlog("Hello", 15000)Page
Page
buffer[10000] = 0;
Page 149
// "Stack Clash"
int dlog(const char *str, size_t len) { char buffer[len]; ... }
dlog("Hello", /*huge!*/)Page
Page
Page 157
buffer[0] = 'H';
Page 158
buffer[0] = 'H';
buffer[1] = 'e';
Page 159
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
Page 160
buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
Page 161
💥buffer[0] = 'H';
buffer[1] = 'e';
buffer[2] = 'l';
💥💥
Page 164
// Stack Checking
int dlog(const char *str, size_t len) { // if (buffer[len] not in stack) // abort(); char buffer[len]; ... }
Heap
Page 165
// Stack Checking
int dlog(const char *str, size_t len) { // if (buffer[len] not in stack) // abort(); char buffer[len]; ... }
Heap
Page 166
// Stack Checking
int dlog(const char *str, size_t len) { // if (buffer[len] not in stack) // abort(); char buffer[len]; ... }
Heap
Page 167
// Stack Checking
int dlog(const char *str, size_t len) { // if (buffer[len] not in stack) // abort(); char buffer[len]; ... }
Heap
Page 168
// Stack Checking
int dlog(const char *str, size_t len) { // if (buffer[len] not in stack) // abort(); char buffer[len]; ... }
Heap
❌
Page 169
Stack CheckingNEW
Page 170
Stack Checking
Detects “Stack Clash”
NEW
Page 171
Stack Checking
Detects “Stack Clash”
Enabled by default in Xcode 10
NEW
Page 172
•New Instruction Set Extensions
Page 175
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 176
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 177
AVX-512 512-bit vectors
Page 178
AVX-512 512-bit vectors
X86-64 (SSSE3)
New Macs(AVX)
iMac Pro(AVX-512)
Page 179
AVX-512 512-bit vectors
X86-64 (SSSE3)
New Macs(AVX)
iMac Pro(AVX-512)
128
Page 180
AVX-512 512-bit vectors
X86-64 (SSSE3)
New Macs(AVX)
iMac Pro(AVX-512)
256
128
Page 181
AVX-512 512-bit vectors
X86-64 (SSSE3)
New Macs(AVX)
iMac Pro(AVX-512) 512
256
128
Page 182
AVX-512 512-bit vectors
X86-64 (SSSE3)
New Macs(AVX)
iMac Pro(AVX-512) 323232323232323232
32
32
32
32
32
32
32
32
32
32
32
32
32
32
32
32
32
32
32
Page 183
AVX-512 More vector registers
Page 184
AVX-512 More vector registers
SSSE3 AVX-512
Page 185
AVX-512 More scalar registers too
SSSE3 AVX-512
Page 186
// Specializing a Function for AVX-512
void compute() { // expensive numerical computation ... }
Page 187
// Specializing a Function for AVX-512
void compute_generic() { // expensive numerical computation ... }
Page 188
// Specializing a Function for AVX-512
void compute_generic() { // expensive numerical computation ... }
__attribute__((__target__("arch=skylake-avx512"))) void compute_avx512() { // expensive numerical computation tuned for AVX-512 ... }
Page 189
// Specializing a Function for AVX-512
void compute_generic() { // expensive numerical computation ... }
__attribute__((__target__("arch=skylake-avx512"))) void compute_avx512() { // expensive numerical computation tuned for AVX-512 ... }
Page 190
// Specializing a Function for AVX-512
void compute_generic() { // expensive numerical computation ... }
__attribute__((__target__("arch=skylake-avx512"))) void compute_avx512() { // expensive numerical computation tuned for AVX-512 ... }
Page 191
// Specializing a Function for AVX-512
#include <simd/simd.h>
void compute_generic() { simd_float4 vector; vector = a + b; ... }
__attribute__((__target__("arch=skylake-avx512"))) void compute_avx512() { simd_float4 vector; vector = a + b; ... }
Page 192
// Specializing a Function for AVX-512
#include <simd/simd.h>
void compute_generic() { simd_float16 vector; vector = a + b; ... }
__attribute__((__target__("arch=skylake-avx512"))) void compute_avx512() { simd_float16 vector; vector = a + b; ... }
Page 193
// Specializing a Function for AVX-512
#include <immintrin.h>
void compute_generic() { // expensive numerical computation ... }
__attribute__((__target__("arch=skylake-avx512"))) void compute_avx512() { __m512 vector; ... }
Page 194
NEWAVX-512 Enabling in Xcode
Page 195
AVX-512 Considerations
Page 196
AVX-512 Considerations
Only pass large vectors to/from AVX-512 functions
Page 197
AVX-512 Considerations
Only pass large vectors to/from AVX-512 functions
Align AVX-512 vectors properly
Page 198
AVX-512 Considerations
Only pass large vectors to/from AVX-512 functions
Align AVX-512 vectors properly
Use Accelerate.framework!
Page 199
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 200
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 201
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 202
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 206
Atomics
Thread #1
Page 207
Main Memory
Atomics
Thread #1
Page 208
Main Memory
Atomics
Thread #1
shared_var
_Atomic int shared_var;
Page 209
Main Memory
Atomics
Thread #1
shared_var
++shared_var;
Page 210
Main Memory
AtomicsARM v8
Thread #1
shared_var
Page 211
Main Memory
AtomicsARM v8
Thread #1
shared_var
Page 212
Main Memory
AtomicsARM v8
Thread #1
shared_var
tmp = shared_var;
Page 213
Main Memory
AtomicsARM v8
Thread #1
shared_var
tmp = shared_var;
Page 214
Main Memory
AtomicsARM v8
Thread #1
shared_var
tmp = shared_var;tmp += 1;
Page 215
Main Memory
AtomicsARM v8
Thread #1
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp;
Page 216
Main Memory
AtomicsARM v8
Thread #1
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp;
Page 217
Main Memory
AtomicsARM v8
Thread #1 Thread #2
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp; shared_var2
Page 218
Main Memory
AtomicsARM v8
Thread #1 Thread #2
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp;
tmp = shared_var2;tmp += 1;
shared_var2 = tmp;shared_var2
Page 219
Main Memory
AtomicsARM v8
Thread #1 Thread #2
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp;
tmp = shared_var2;tmp += 1;
shared_var2 = tmp;shared_var2
Page 220
Main Memory
AtomicsARM v8
Thread #1 Thread #2
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp;
tmp = shared_var2;tmp += 1;
shared_var2 = tmp;shared_var2
Page 221
Main Memory
AtomicsARM v8
Thread #1 Thread #2
shared_var
tmp = shared_var;tmp += 1;shared_var = tmp;
tmp = shared_var2;tmp += 1;
shared_var2 = tmp;shared_var2
Page 222
Main MemoryThread #1 Thread #2
++shared_var; ++shared_var2;
ARM v8.1 Atomics NEW
shared_var shared_var2
Page 223
ARM v8.1 Atomics Considerations
Page 224
Only useful for raw C11/C++11 atomics
ARM v8.1 Atomics Considerations
Page 225
Only useful for raw C11/C++11 atomics
Use GCD, PThread, os_unfair_lock, …!
ARM v8.1 Atomics Considerations
Page 226
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 227
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 228
16-Bit Floating Point ARM v8.2
Page 229
16-Bit Floating Point ARM v8.2
Double Precision (Apple A10)
Single Precision(Apple A10)
Half Precision(Apple A11)
Page 230
16-Bit Floating Point ARM v8.2
Double Precision (Apple A10)
Single Precision(Apple A10)
Half Precision(Apple A11)
64
Page 231
16-Bit Floating Point ARM v8.2
Double Precision (Apple A10)
Single Precision(Apple A10)
Half Precision(Apple A11)
32
64
Page 232
16-Bit Floating Point ARM v8.2
Double Precision (Apple A10)
Single Precision(Apple A10)
Half Precision(Apple A11) 16
32
64
Page 233
16-Bit Floating Point ARM v8.2
Double Precision (Apple A10)
Single Precision(Apple A10)
Half Precision(Apple A11) 1616161616
32
16
32
16
32
64
16
32
64
Page 234
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 235
AVX-512ARMv8.1 Atomics
ARMv8.2 16-Bit Float
Page 236
Detecting Instruction Set Extensions
Page 237
Detecting Instruction Set Extensions
New instructions are not available everywhere
Page 238
Detecting Instruction Set Extensions
New instructions are not available everywhere
Detect available extensions using sysctlbyname
Page 239
Detecting Instruction Set Extensions
New instructions are not available everywhere
Detect available extensions using sysctlbyname
Or just let system frameworks handle this for you!
Page 240
New Instruction Set Extensions Summary
Page 241
Supported On
Type__target__
march=Clang Flag sysctlbyname
New Instruction Set Extensions Summary
Page 242
Supported On
Type__target__
march=Clang Flag sysctlbyname
AVX-512 Vectors __m512 skylake-avx512 -march=skylake-avx512 hw.optional.avx512{f,bw,dq,vl,cd}
New Instruction Set Extensions Summary
Page 243
Supported On
Type__target__
march=Clang Flag sysctlbyname
AVX-512 Vectors __m512 skylake-avx512 -march=skylake-avx512 hw.optional.avx512{f,bw,dq,vl,cd}
ARM v8.1 Atomics _Atomic monsoon -mcpu=monsoon hw.optional.armv8_1_atomics
New Instruction Set Extensions Summary
Page 244
Supported On
Type__target__
march=Clang Flag sysctlbyname
AVX-512 Vectors __m512 skylake-avx512 -march=skylake-avx512 hw.optional.avx512{f,bw,dq,vl,cd}
ARM v8.1 Atomics _Atomic monsoon -mcpu=monsoon hw.optional.armv8_1_atomics
ARM v8.2 Float 16 _Float16 monsoon -mcpu=monsoon hw.optional.neon_fp16*
New Instruction Set Extensions Summary
* Available in a future Beta
Page 245
ARC Object Pointers in C Structs
Stack CheckingBetter Static Analyzer Performance
GCD Anti-pattern Analyzer Check
Autoreleasing Misuse Analyzer Check
AVX-512
ARM v8.1 Atomics
ARM v8.2 16-Bit Float
Suspicious Pragma Pack
Missing Noescape Annotation
Page 246
ARC Object Pointers in C Structs
Stack CheckingBetter Static Analyzer Performance
GCD Anti-pattern Analyzer Check
Autoreleasing Misuse Analyzer Check
C++17 std::variant
C++17 std::optional
C++17 std::any
AVX-512
ARM v8.1 Atomics
ARM v8.2 16-Bit Float
Suspicious Pragma Pack
Missing Noescape Annotation
Faster Compiles for Huge/Generated Functions
Page 247
More Informationhttps://developer.apple.com/wwdc18/409
LLVM Compiler, Objective-C, and C++ Lab Technology Lab 10 Thursday 3:00PM