Harshal Lehri - Information and Computer Sciencematuszek/cis554-2016/Talks/Factor.pdfHarshal Lehri. What’s different about Factor? Factor is a concatenative programming language

Harshal Lehri

What’s different about Factor?

● Factor is a concatenative programming language - A program can be viewed as a series of functions applied on data

● Factor is a stack oriented program - Data in Factor is stored on a stack - functions pop data from the stack and the output data is pushed on the same stack

● Factor uses an Image-based persistence model - the program code and data are saved in an image file , similar to core dumps

● Built in Foreign Function Interface that allows it to talk to declarative programs like C, Objective-C and Fortran

A Practical stack oriented language

● Factor supports popular functional programming concepts like currying and higher order functions( like map, reduce, filter)

● Factor supports object-oriented programming - via tuples and generic words

● Factor is dynamically typed and has a type system● Factor supports powerful meta programming capabilities using parsing

words● Garbage collection, optimizing compiler , nice REPL, large standard library

and more...

Concatenative Programming

● A program can be written as a chain of functions applied one after another, each modifying data before passing it to the next one

● Unix pipes are a good example of concatenative programming● Unix pipes use a Queue to store intermediate results while Factor uses a

Stack

Concatenative Programming- An Example

● Consider a Unix command like cat file1.txt | grep “filter-text” | awk {print $1}● The program reads contents of file, filters lines containing the word

“filter-text” and prints the word before the first whitespace in the line

cat file1.txtFile data

grep “filter-text”

Filtered Lines awk {print $1}

Words and Vocabularies

● Functions in Factor are encouraged to be small and read from left to right . They are known as words

● A set of words form a module and are called vocabularies● Factor programs are a sequence of words (including ones that generate

data into the stack)● Words are usually 1-3 lines in length and larger ones can be factored out

into smaller words applied after one another● Words are separated only by spaces - this makes tokenization simple

A simple Factor program - Finding Factorial of 10

● Suppose we want to calculate the factorial of 10 in Factor. We can do that as follows

10 [1,b] 1 [ * ] reduce

push 10 on the data stack

Generate a lazy sequence from 1 to number on top of the stack

Apply reduce on the sequence on top of the stack

push 1 on the stack

A “quotation” or a lambda function, here multiplication

The reduce function that takes a sequence, base number and a function, all present on the stack

Factorial of 10 - View from the REPL

Factorial of 10 - Key Takeaways

● Literals like 10, “hello”, 0.19 are treated as words themselves, they have the effect of adding the data to the stack

● Words use the postfix notation i.e data is pushed to the stack before the operator. Hence there is no notion of precedence required

● Words in factor can use any character, number or punctuation - only whitespace is used for tokenization. Hence we can have words like [a,b]

● [ * ] is an anonymous function or quotation. Note the whitespace in between [ and *. [ is a special word called parsing word

Writing a more general Program in Factor

● The previous program did not look much like a function - it worked for only one input

● The aim of the next program will be to create a general function as well as introduce the notions of parsing-word, stack-effects and combinators

Parsing-Words

● A parsing word is a special word that is part of the meta-programming of Factor

● Unlike regular words, parsing-words do not have an immediate effect on the stack - rather they interact with Factors parser to influence how successive words are to be parsed

● A Parsing-Word is defined using the SYNTAX: … ; word (note SYNTAX: itself can be considered a parsing word)

Some common Parsing Words

● [ ... ] : These parser words are used to define quotations or anonymous functions. For example [ * ]

● { ... } : These are parser words used to define arrays. For example { 1 2 3 4 5 }

● H{ ... } : These are parser words used to define hash maps. For example H{ { “A” “B” } { “C” “D” } }

● : … ; : These are parser words used to define functions● USE: … ; : Parser word to import packages● TUPLE: … ; : Parser word to define a class

Defining Functions in Factor

● A function definition in Factor is as follows

: name ( x1 x2 …. xn -- y1 y2 ….ym ) body ;Parsing word begin

Parsing word end

Function Name

The “stack effect” of the function

Body of the function

Stack Effects

● Stack effects is an important feature in Factor. They behave like a pre-condition and post-condition for that function

● The notation ( x1 x2 … xn -- y1 y2 … ym ) means that the function processes x1 x2 … xn items on the stack and output y1 y2 .. ym on the stack

● The variable names themselves don’t mean anything, the names are chosen to allow the user to quickly identify the behaviour of the function

● To use the names x1,x2 … xn in the body , the :: parsing-word is used when declaring functions instead of :

Stack Effects - Examples

● dup ( x -- x x ) : Duplicate the top element of the stack● drop ( x -- ) : Drop the top element of the stack● swap ( x y -- y x ) : Swap the top two elements of the stack● nip ( x y -- y ) : Remove second element from top on the stack● rot ( x y z -- y z x ) : Rotate left top three elements of the stack● -rot ( x y z -- z x y ) : Rotate right top three elements of the stack● product ( {x1,...,xn} -- x1*...*xn ) : Product of an array

Dataflow Combinators

● Stack based programs require the user to run a mental stack machine, which becomes trickier when applications shuffle the stack (like swap)

● Dataflow Combinators simplify this by encapsulating common, easy to understand dataflows

○ cleave combinators like bi, tri that apply multiple functions on a data value, for example the program 10 { [ 3 + ] [ 2 - ] } cleave results in the top of the stack having value 13 8

○ spread combinators like bi*, tri* takes a series of objects and a equal number of quotations and apply the quotation on the appropriate object

○ napply combinators like bi@, tri@ take a quotation and an integer n and apply the quotation on the top n objects on the stack

Dataflow Combinators - Examples

● 10 [ 3 + ] [ 2 - ] bi results in the data stack having 13 and 8 as its top elements. Equivalent to 10 { [ 3 + ] [ 2 - ] } cleave

● 10 12 [3 + ] [ 2 - ] bi* results in the data stack having 13 and 10 as its top elements. Equivalent to 10 12 { [ 3 + ] [ 2 - ] } spread

● 10 12 [ 2 + ] bi@ results in the data stack having 12 and 14 as its top elements. Equivalent to 10 12 [ 2 + ] 2 napply

Currying in Factor

● Factor allows function currying using the curry word● The curry word accepts a 10 [ + ] curry generates a new word on [ 10 + ] by

absorbing the top element of the stack into the quotation● The curry word requires two arguments an argument and a quotation, and

absorbs the argument into the quotation● For example if the top of the data stack looks like [ 2 + ] [ 3 + ] [ bi ]. Using

the curry word, the new stack looks like [ 2 + ] [ ~quotation~ bi ]. Note that it only absorbed one argument at a time

Primality testing program in Factor

: prime? ( n -- ? ) [ sqrt [2,b] ] [ [ multiple? ] curry ] bi any? not ;

Word name Stack effect for the word, this states that the word takes the top element of the stack and returns a boolean (The symbols themselves don’t mean anything)

Quotation of generating numbers from 2 to √n

Checks if one number is a multiple of another

A cleave combinator

Negate the final result

Checks if there is a number in a sequence that satisfies a given condition

Recursion in Factor

: fib ( n -- fib(n) ) dup 1 > [ [ 1 - fib ] [ 2 - fib ] bi + ] when ;

● Factor performs tail-call optimization

Object System in Factor

● Factor is purely object-oriented programming language - every value is an object and basic operations are method calls

● Unlike traditional object-oriented programs, methods do not belong to a particular object or class, rather a method is defined as a generic word with its implementation being different for different classes

Defining new Classes - The Tuple: parsing word

● A class can be defined using the Tuple: parsing word as TUPLE : class-name field1 field2 …. ; . For example :

TUPLE: circle radius ;

TUPLE: rectangle length width ;

● Each instance variable has an associated getter and setter. For example the circle class has a radius>> getter and >>radius setter. Fields can be made read-only and by defining them like { radius read-only }

Creating Objects of a Class

● An object of the class can be created using the new word like

rectangle new 10 >>length 10 >>width

● …. or using the boa(By-order-of-arguments) constructor

10 10 rectangle boa

● The boa constructor is common enough to have its own parsing defined as C: <rectangle> rectangle, which can be used as 10 10 <rectangle>

Methods for classes - Generic words

● The GENERIC: parsing word allows a user to define methods common to different classes. For example an area method for the two defined shapes can be written as

GENERIC: area ( shape -- area )

M: circle area radius>> dup * pi * ;

M: rectangle area [ length>> ] [ width>> ] bi * ;

Derived Classes

● Primitive classes like Strings, Numbers and words cannot be subclassed● Tuple classes offer single inheritance - rooted at a class called tuple● More classes can be defined using set operations on Tuple classes like

UNION: and INTERSECTION: ○ Mixins are special Union class where some class can be added to a mixin that share a

method

● Protocols consist of a set of generic words used by a Mixin. This is equivalent to an interface in other object oriented languages

Unit Tests and Documentation

● Unit Tests can be written using tools.test vocabulary. For example,

[ t ] [ 2 prime? ] unit-test

● Similar to unit-tests, there are different words like must-fail, must-infer etc● The help.markup and help.syntax vocabularies help in documentation

Documentation in Factor

HELP: prime?{ $values { "n" fixnum } { "?" boolean }}{ $description "Tests if n is prime. n is assumed to be a positive integer." } ;

I/O in Factor

● Factor uses asynchronous input/output facilities, similar to NIO on the JVM or Node.js I/O system

● I/O is represented as lazy streams around which words are designed to read or write to a stream

: read-lines ( path -- ) utf8 file-lines [ 1 safe-head write nl ] each ;

Encoding of file Reads lines of a

file as a lazy sequence

read 1 element from the sequence and print it

Apply quotation on each file line

Multithreading

● Factor is single threaded - so it emulates multithreading using coroutines - cooperative threads that periodically yield the core they are running on

● Factor allows for spawning of processes on multiple cores that communicate through channels

● Threads are created using spawn, with stop, suspend, sleep and resume words

● Channels use to and from similar to Akka’s send and receive ( send and receive in Factor is used for marking quotations to be called when receiving or sending a messsage)

Other Features

● Factor allows using local variables using the :: parsing word by binding stack parameters to variables

● Factor allows using dynamic variables using the SYMBOL: parsing word along with get and set words to manipulate it. These variables can be bound to a particular scope using [ ]with-scope parsing words

● Factor has a built in HTTP server library ● Factor provides nice metaprogramming features in terms of macros,

parsing words and functors ● Excellent REPL

Current Limitations

● Factors community is small. It is tricky to find information about Factor on the internet

● It’s not always simple to think in terms of a single stack - many stack manipulating operations are tricky to understand and traditional programs have different implementation

● Factor is single threaded - so programs must be designed in an event driven manner

Some good resources on Factor

● Factor REPL can be downloaded from http://factorcode.org/● Slav Pestov’s paper on Factor http://factorcode.org/littledan/dls.pdf● Andrea Ferreti’s excellent tutorial about basics of Factor

https://andreaferretti.github.io/factor-tutorial/● Concatenative programming websites Wiki on Factor

https://concatenative.org/wiki/view/Factor/Learning● Some good examples on http://planet.factorcode.org/ and

http://progopedia.com/language/factor/● Slav’s Google talk on https://www.youtube.com/watch?v=f_0QlhYlS8g