CSE 595, Winter 2000 1 January 27: Technology zAdministrivia yNext week’s readings on web — please check soon to make sure you can download them ygrading.

CSE 595, Winter 2000 1

January 27: Technology

Administrivia Next week’s readings on web — please check soon to

make sure you can download them grading — changing weights for reviews and class

discussion

Constraints: overview Cassowary: demo and lecture DeltaBlue (short presentation) More paper discussion:

Brad Myers, “UI Toolkits” QT “MOBILE”

CSE 595, Winter 2000 2

Quote of the Week

“I wish to God these computations could be performed by steam!”

-- Charles Babbage

CSE 595, Winter 2000 3

Constraints for User Interface Applications Lecture Outline - Constraints portion

what are constraints constraint hierarchies (theory) Cassowary (simplex-based solver) DeltaBlue (local propagation solver)

Online information: www.cs.washington.edu/research/constraints(also see the link there to the Constraints Archive)

CSE 595, Winter 2000 4

What Are Constraints?

Relations that should hold (in the mathematical sense of “relation”)

Can be used for establishing, verifying, maintaining, and reasoning about such relations

Clear declarative semanticsData invariants & assertionsEmphasize result, not process

CSE 595, Winter 2000 5

Domains

Numeric or non-numeric Discrete or continuous Finite or infinite

Real numbers: numeric, continuous, infinite

Integers: numeric, discrete, infinite Characters: non-numeric, discrete, finite Bitmaps: non-numeric, discrete, infinite

CSE 595, Winter 2000 6

UI Applications of Constraints

• screen and window layout (e.g. w1 is above w2)

• consistency between application data and views, consistency among multiple views

• drawing editor• web page layout• animation

CSE 595, Winter 2000 7

Constraint Graphs

Represent the system as a bipartite graph: constraint nodes variable nodes there are edges from each constraint to the

variables it constrains(papers aren’t always consistent about this:

some represent the constraint system as a graph with nodes for the variables and hyperedges for the constraints)

CSE 595, Winter 2000 8

One-way Constraints Example

CSE 595, Winter 2000 9

One-way Constraint Algorithms

Simplest, cleanest version: constraint graph is a directed acyclic graph

Solvers: Natural order evaluation (eager) EVAL/Vite (lazy)

CSE 595, Winter 2000 10

One-way Constraints Scorecard

The computation model is easy to understand. The algorithms are easy to implement. The algorithms are very fast. But they do not handle multi-directional

constraints, cycles, conflicts, and over- or under-constrained problems.

CSE 595, Winter 2000 11

Relaxing the Acyclic Restriction for One-way Constraints how to handle cycles:

ignore, once around, iterate to fixed point

how to handle conflicts ignore, error, compare results

CSE 595, Winter 2000 12

One-way Constraint Systems

Microsoft Excel

EVAL/Vite

GARNET, Amulet

Rendezvous

Visio (if you look at it right)

CSE 595, Winter 2000 13

Moving beyond one-way constraint systems:

Problem: It’s Easier To State Constraints Than To Solve Them

Simple constraints can express very difficult problems:

xn + yn = zn, n > 2 Reasonable constraint systems are restricted Restrictions allow the constraints to be solved

efficiently and still define useful computations

CSE 595, Winter 2000 14

A Problem for Interactive Systems: Change

What should change to keep the constraints satisfied?

constraint: a+b=ccurrently a=3, b=4, c=7user changes c to 9. What should happen?

CSE 595, Winter 2000 15

Constraint Hierarchies

Ordered sequence of sets of constraints Level 0 must be solved; other levels are preferences Level i dominates level i + 1 Used to control perturbation and add defaults

0 Required Hard

1 Strong

2 Medium Soft

3 Weak

4 Very Weak

CSE 595, Winter 2000 16

Edit and Stay Constraints

Used in defining constraints for interactive systems

Semantically, these are just constraints of the form v=c for a variable v and a constant c (at some level of preference)

Typically the strength of an edit will be strong, and the strength of a stay will be very weak (weaker than all the other constraints)

CSE 595, Winter 2000 17

Edit and Stay Constraints (2)

For edit constraints, c is supplied externally (e.g. mouse position)

For stay constraints, c is the value of v at the previous step

In an interactive system, we will need to solve very similar collections of constraints many times, each collection differing only in the value of the constants c in the edit and stay constraints

CSE 595, Winter 2000 18

Constraint Systems - Some Principles

Some Principles for Constraint Systems for Interactive Graphical Applications

• Have a declarative theory, independent of the constraint satisfaction algorithm.

• Deal with state and change directly.• Handle overconstrained and underconstrained

systems.• Semantics of figure should be independent of

the order in which it was constructed or manipulated.

CSE 595, Winter 2000 19

Constraint Satisfaction Algorithms

• One-way solvers (natural order evaluation, lazy evaluation)

• local propagation solvers (DeltaBlue, SkyBlue, Quickplan)

• local propagation for inequalities (Indigo)• Simplex-based solvers (Cassowary, QOCA)• Fourier elimination based solvers • hybrid algorithms (DETAIL, Ultraviolet)• CSP algorithms (for constraints over finite

domains)• iterative numeric techniques

CSE 595, Winter 2000 20

The Cassowary

CSE 595, Winter 2000 21

The Quokka

(Actually, the solver is spelled “QOCA” …)

CSE 595, Winter 2000 22

The Fourier

CSE 595, Winter 2000 23

Scheme Constraints Window Manager

Example: constraints to tile windows

CSE 595, Winter 2000 24

SCWM - Button Bar

CSE 595, Winter 2000 25

SCWM - Constraints Investigator

Xterm A is constrained to be to the left of Xterm B and above Xterm C. C has a min width, and the lower right window has an anchored right hand side.

CSE 595, Winter 2000 26

Cassowary

Average rating of paper: 4.3 (low 2, high 6)

(could there be any non-technical factors that influence the ratings?)

switch to latex slides here

CSE 595, Winter 2000 27

Hierarchical Local Propagation Constraint Satisfaction Algorithms

DeltaBlue SkyBlue QuickPlan

Propagation refers to flow of data Local refers to search strategy - local choices (Constraint hierarchies require global solutions) Incremental for efficiency

CSE 595, Winter 2000 28

Constraint Methods

c = a+bc := a+ba := c-bb := c-a

CSE 595, Winter 2000 29

Read-Only Variables

c = a? + bc := a+bb := c-a

Note that there isn’t a method for setting a

CSE 595, Winter 2000 30

Semantics of Read-Only Variables (intuition)

constraint graph can’t have cycles through read-only variables

divide into writeable subgraphs (i.e. split graph at read-only variables)

do topological sortsolve subgraphs independently

CSE 595, Winter 2000 31

Read-Only Variable Example

required x=y?strong x=10weak y=20

solution is x=y=20

CSE 595, Winter 2000 32

DeltaBlue

Uses cached information (the “Walkabout strength”) to make local choices towards a globally optimal locally-predicate-better solution.

The Walkabout Strength is the weakest constraint on a reversible path.

WS = min(constraint, WS of inputs thatcould be outputs)

CSE 595, Winter 2000 33

Simple DeltaBlue Example

required 1.8 * C = Temprequired Temp + 32 = Fweak C stayweak F stay

add a constraint:strong F edit

CSE 595, Winter 2000 34

Notes on Brad Myers paper

“The State of the Art in User Interface Software Tools”

… as of 1995. (Unfortunately, I couldn’t find any comparable recent surveys.)

ratings: average 4.6 low 2 high 6

CSE 595, Winter 2000 35

User Interface Tools

Terminology:user interface management systems

(sometimes used for tools that handle the sequencing of operations)

toolkitsuser interface development environmentsinterface buildersapplication frameworks

CSE 595, Winter 2000 36

Why Building UI’s Is Hard

large portion of total code (~50%)support for rapid prototyping and iterative

designmultiprocessing (e.g. interrupts, multiple

tasks)real-time programmingrobustness in face of errorstestabilitymodularizationtool complexity

CSE 595, Winter 2000 37

Software Engineering Perspective

Requirements are classic:want reusabilitywant to miminize dependencies between

application and the interfaceneed tight coupling to allow necessary

information to flowneed good performance

CSE 595, Winter 2000 38

The Seeheim User Interface Model

PresentationDialog Control

Application Interface

ModelUser

CSE 595, Winter 2000 39

Problems with Seeheim Model

Coupling of application and presentation is too restricted

Example: want an interface to a chess game. When you move the mouse over a piece, the legal moves should be highlighted.

CSE 595, Winter 2000 40

Input/Output Models

textual (e.g. standard in, standard out in unix)

early bitmap display systems -- allowed direct access to display

windowing systems I/O primitives organized around windows

CSE 595, Winter 2000 41

Input Models

Generally, the windowing system represents input as a stream of events.

Each event record typically contains: type and value of event (e.g. which key was

pressed) window that the event was directed to timestamp x and y mouse position

CSE 595, Winter 2000 42

Window Choices

overlapping vs. tiled windowsiconificationkeyboard focus (click to type vs. move to

type)widget set

CSE 595, Winter 2000 43

Toolkits

set of procedures or class definitions that can be used by application programs

virtues of object-oriented approach natural way to think about widgets ease of extension using subclassing cross-platform support

connecting widgets with application: method invocations (application widget) callbacks (widget application ) constraints (one or both ways)

CSE 595, Winter 2000 44

Higher-Level Tools

special purpose specification languages (e.g. for dialogs)

state transition networkscontext free grammarsevent languagesdeclarative languages (e.g. for layout),

including constraints

CSE 595, Winter 2000 45

QT Toolkit

cross-platform C++ application frameworkwidget class hierarchysignal/slot communication mechanism

class can define member functions to be slots class can specify that its instances can emit signals can connect signal of one object to a slot of

another object that emits a signal doesn’t need to know

which member functions will be invoked as a result mechanism is type-safe

CSE 595, Winter 2000 46

QT Layout Manager Classes

Box layout (horizontal or vertical stacking)Grid layout (m n grid)(layout managers seem like a good

application for constraints …)