IWCG Privacy Update - World Wide Web Consortium · A malicious 2D web page - without access to camera data - recognizes that it is on an optically see-through device (e.g. ... Stationary

IWCG Privacy Update26.10.2018TPAC, Lyon

Agenda15m - High-level overview of documented threat vectors and potential mitigations

… based on issues and explainer at Privacy & Security repo

https://github.com/immersive-web/privacy-and-security

15m - Questions and Discussion

What is WebXR?API that allows sites to build virtual and augmented reality experiences.

Provides an API for determining device orientation for rendering purposes, and exposes new types of real-world data for augmented reality.

For VR, allows scenes to be rendered in a headset (+smartphone +desktop)

For AR, allows rendering of virtual objects into the real world

● On screen-based displays, manages camera rendering…● … and supports non-screen displays such as transparent HMDs

Privacy ApproachThe privacy & security repo documents patterns for web-based XR threat vectors.

Specifically

- What are the threat vectors if a site has access to a data type- What are considerations and potential mitigations

Focus of this presentation: Mitigations unique to the threat vector / data type.

E.g. User consent is always a potential mitigation

Threat vectorsConsidered thus far..

● Perception of camera access● Access to real-world geometry● Object identification● Permissions● Pose (device orientation) and frame of reference data

Perception of Camera Access

Why does it matter?In AR, on screen-based devices the user may think the site has access to the camera… even if the site doesn’t.

Perception of Camera AccessThreat Vector #1… without actual camera access

1. A malicious site could create an AR session where the user agent displays the camera (and nothing else);2. A smartphone user could visit that site, be presented with a front-camera view, potentially capturing a compromising situation;3. The user would likely close the camera view, quickly;4. The site could then say “Thank you for that picture! We’ll add to the public gallery. Please pay our $10 membership fee to

control who can see the picture.”

Threat Vector #2… without actual camera access

1. The user is using an optically see-through device;2. A malicious 2D web page - without access to camera data - recognizes that it is on an optically see-through device (e.g.

detecting through CSS that it is on an additive display);3. The page renders a camcorder viewfinder with a blinking REC indicator in the corner (e.g. on an additive display, the viewfinder

could be filled with black pixels, and would thus show the real world). This would make it appear that the web page can record real world imagery, even though it can’t.

Mitigation: Clear communication to the user about site access to camera data

Real-World Geometry Data

Why does it matter?New web APIs allow sites to access 3D geometry that represents the user’s physical environment.

Threat Vectors● Personally Identifying Information

○ Face analysis○ Embossed credit card #

● Fingerprinting○ Room geometry

● Location○ If near a known location (e.g. geometry of eiffel tower)○ Independent of GPS sensors (and thus possibly without permission for geolocation)

● Physical security○ Map out inside of private, secure area; get geometry of a door key

Threat Vectors● Profiling

○ Income estimation (size of house, valuable items)○ User height (from ground plane)○ Determine previously visited places based on what geometry is available

○ Even bounding boxes can be used for profiling, context

● Historical geometry / Session Data Privacy○ Some AR systems have geometry from before the user’s current session

■ May allow analysis of where the user has previously been (e.g. historical path analysis)○ Others merge geometry from the session into larger planes

Potential MitigationsThrottling & Precision: Not clear what value this will bring, because

● Lower precision hurts the user experience● Multiple sessions can merge inaccurate data to infer ground truth● Low-resolution data still has threat vectors

○ But: Users can visualize low-resolution data; that may be a useful policy tool

Filtering: User agent can limit geometry access:

● E.g. Only allow access to what is seen in the current session● But: May need AR sub-system to cooperate (e.g. geometry behind wall)

Clearing data: Where supported can prevent historical geometry being shared

Object Identification

Why does it matter?It’s useful in augmented reality for the site to be able to identify 2D images or 3D objects.

But: The user may not know what’s being looked for.

Bottle of water

Threat Vectors (Object Identification)● Location

○ Storefronts, QR codes, street signs, configurations of objects

● Profiling ○ Income estimation (particularly dangerous if location is known)

● Compromising info○ E.g. Embarrassing imagery or objects

● User Safety / Obscuring○ Put user in a dangerous situation, e.g. covering a stop sign or a hole in the ground○ Put others in a dangerous situation, e.g. make it look like someone else holds a gun○ Embarrassing situations, e.g. insert inflammatory images during a negotiation

Object IdentificationPossible Mitigations

● Throttling○ Limit # of objects or images the site can identify

● User visibility○ Allow the user to see what the site is looking for

● No access to object position○ Prevent site from actually knowing where in the user’s space an object was located○ However, even the existence of an object may be enough for a bad actor

● Composition rules: UA prevents obscuring important objects○ E.g. a “safe field” where the user always sees the real world

Permissions

PermissionsAR uses a lot of sensors and data! Considerations for web permissions:

● Permission fatigue● Ambiguity and complexity: What is the site doing exactly?● Persistent or long-running permissions beyond scope of session

Possible mitigations

● Time-limited permissions● Session-scoped ‘modes’ (e.g. AR Mode)

○ Flexibility: Could either ask for consent, or give notification, or both depending on data access○ Clear scope: User agent has more UX flexibility to inform the user clearly○ Enforcement: Only the data permitted is allowed, and only during the session

Pose and Frame of Reference Data

(This is still a WIP…)

Access to Pose and Frame of Reference DataSites can access:

● Pose of viewer or device: Position and Orientation○ Note: Position is real-world scale, but is not a real-world location

● Floor height, space bounds

On a variety of devices and in various contexts

● Immersive (HMDs)● Inline

○ On phone○ On desktop/laptop○ In HMD while 2D browsing

Threat Vectors● Gaze Tracking

○ What is user looking at?

● Input sniffing○ When: Input ⇒ Device Orientation / Motion ⇒ Pose○ This has been demonstrated to read PINs on mobile devices at 20Hz○ May be possible on desktop/laptop (keyboard) or HMDs (gaze inputs, virtual keyboards)

● Profiling○ Using: Bounds, Floor Height, Action Analysis (e.g. walking)

● Fingerprinting○ Using: Bounds, Floor Height, Gait Analysis

● Location○ May be possible to identify specific location based on trajectory or path○ Example: Recognizing pattern of major roadways, recognizing pathways of a campus

WebXR Modes, Implied ThreatsMode (>> Submode) Available Data Implied Threat Vectors

frameOfRef = NULL None None

Stationary >> position-disabled Orientation Input sniffing

Stationary >> eye-level Orientation, Position Input sniffing, Location, Profiling, Gaze, Fingerprinting

Stationary >> floor-level Orientation, Position, Floor Level Input sniffing, Location, Profiling, Gaze, Fingerprinting

Bounded Orientation, Position, Region bounds, Floor Level

Input sniffing, Location, Profiling, Gaze, Fingerprinting

Unbounded Orientation, Position Input sniffing, Location, Profiling, Gaze, Fingerprinting

Potential MitigationsInput Sniffing

● Focused & Visible● Restrict pose data to:

○ Same origin only (i.e. pose data only provided to same-origin elements)○ Single-origin only (i.e. only one origin can have pose data at a time)

● Secure context● Feature policy● User consent

… more generally...Potential mitigations similar to Generic Sensor API

● Focused & Visible● Same origin only● Secure context only● Feature policy● User consent (as appropriate)

Potential MitigationsLocation

● Limit position ranges based on frame of reference (e.g. bounded, stationary)● User consent (e.g. unbounded)

Non-solutions

● Data throttling● Errors or rounding● Inline vs. exclusive

Potential MitigationsGaze Tracking

● No position (orientation only)● Only give pose when user is looking at inline element● … or only when user is looking at page (when inline)● Blur when a frame is not focused

Potential MitigationsFingerprinting

● Data errors / rounding (for bounds)○ E.g. Restrict bounds to rectangle only

● Fixed position (for gait)

Profiling

● Restrict bounds data to rectangle○ Harder to determine room type

Join the conversation!

https://github.com/immersive-web/privacy-and-security

Q&A

QuestionsAny existing privacy frameworks we should be thinking about?

Are there updates on permissions that apply?

Are there guidelines on how to make user prompts informative and useful?

…?