Building extendable sensor infrastructures for pervasive computing environments

1

Building extendable sensor infrastructures for

pervasive computing environments

Martin Jonsson, Johan Mattsson

The FUSE Research Group, KTH Center for Wireless Systems, KTH

DSV, Electrum 230, SE-164 40 Kista, Sweden

[martinj, johanm]@dsv.su.se

Abstract. Experiences from work with sensors in pervasive computing environments are

described and summarized. Problems regarding reusability, scalability and ease of

deployment of sensor systems are identified. An infrastructure is presented consisting of a

sensor platform to be embedded within the rooms of a building as well as a system for

distribution of sensor data. The infrastructure supports development of sensor-based

applications on a low level and eases reusability of sensors and sharing of sensors between

applications, as well as supports discovery, transport and refinement of sensor data.

Introduction Providing information from sensors to applications can be very useful in several ways. The

behavior of an application can be adapted to make the interaction more efficient or to increase

the ease of use. You can also imagine entirely new types of applications that are designed

specifically to make use of some certain sensor or context information. Under the notion of

context aware computing there has been several attempts both to examine how context

information can be captured in different ways, and how to design applications that use it [1-2].

Within the area of ubiquitous or pervasive computing several systems have been presented

that incorporate input from different kinds of sensors [3-5]. A special case of pervasive

computing environments is so called intelligent buildings where sensing capabilities are a part

of the architecture of the building [6].

2

The increasing use of context information in applications makes it interesting to provide

different kind of support that makes it easier for system developers to add sensor components

to their systems. The support can be given on different system levels from hardware gadgets

to higher-level support such as transport, abstraction and interpretation of sensor data. To

design such support systems it is however necessary to understand how the sensor information

can be used in different applications, and what requirements the different usages put on such a

support system.

Experiences of sensor usage in ubiquitous computing environments In the FUSE research group, a number of applications have been developed within the area of

pervasive computing. Through this work useful experience regarding the use of sensors has

been gained. Some of the different projects are here described briefly:

Meeting support with active documents

The meeting support system “fuseONE” [8] targets the problem of document management in

project meetings. The system contains Active Documents that can use information concerning

who are in the same room to identify the occurrence of a meeting. When the meeting starts the

document presents itself on a public display in the room. The system also supports sending

documents between computers in the same room. A context sensitive desktop shows the

possible receivers of documents present in the room.

This system relies heavily on information about the current locations of people. The

location sensors were so called iButtons [9], small memory circuits containing an identifier of

its owner. Whenever the users entered a meeting room, they pressed these buttons against one

of the readers scattered around the room, thereby actively revealing their presence in the

room. The system that provides the context information to the applications is called Context

Shadow [10] This system provides means to find services that are relevant to a person’s

current context. The Context Shadow system can use any type of location sensor to set up a

connection between a person and a location.

The VAMPIRE storyteller

In the VAMPIRE project the aim was to create a storytelling agent for kids. The storyteller is

implemented as a physical humanoid character, placed in an exhibition area. When a person

3

approaches the agent, it responds by turning its head towards the person. As soon as someone

sits down on a mat in front of the agent, it will start to display a scary story using text on its

belly. The kids can interact with the story by placing different objects in the agent’s hand.

These objects will then appear in the story. Every now and then, the story halts, and the agent

speaks out a question that the audience can respond to verbally. This would also affect the

content of the story. In this project several sensors was used:

• Infrared motion sensors to detect approaching people

• Pressure sensors to detect whether someone was sitting on the mat

• A RFID reader to detect what objects were placed in the hand of the agent

• Speech recognition to understand the answers to the questions

To collect the motion and pressure sensors, we used a Basic Stamp circuit connected to the

serial port of a computer. The RFID reader circuit was also connected to the serial port.

Finally, the Speech recognition used a microphone that was connected to a computer running

speech recognition software. Several computers were involved in the system and a tuple space

was used to share sensor data among the different components.

A system for non-intrusive messaging

In meeting situations, incoming messages often have a disturbing effect on the participants. In

an attempt to tackle this problem a prototype was created where the presentation of the

messages could be varied in different ways, by using different modalities, personal and public

displays, peripheral and ambiguous renderings etc. Private messages where e.g. sent to private

displays embedded at the position where a person where seated. The displays were built using

network connected IPAQs embedded in the table, with iButton readers connected to the serial

pins in the cradle connector. The iButton readers were used to personalize the display

services. There was also a knob in the middle of the table that the participants could use to

select an acceptable “disturbance level” for the meeting. The knob consists of a potentiometer

connected to a TINI-board [12], which is a network-enabled microcomputer.

4

Figure 1. To the left: The Vampire storyteller. To the right: The non-intrusive

messaging environment.

Lessons learned From the projects described above and other work, we have gained useful experiences both

regarding what context information that is useful in different settings, as well as experiences

regarding the implementation of sensor based systems.

Regarding implementation issues, each of the projects used its own specific

implementation of sensors and a lot of effort was put into low-level implementations. The

degree of specialization of these implementations makes it hard to reuse parts of the system

for other purposes, as well as it makes it hard to modify the system according to future

changes in the application. In the fuseONE and especially in the VAMPIRE project, each

sensor implementation occupied a laptop-pc making it a very expensive and clumsy solution.

We believe that the implementation process could be supported in several ways.

Simplifications regarding low-level installation of sensors would decrease development time a

great deal. General components could be used that would only require smaller modifications

to support the sensors being used. Some kind of general tools could also be used for the

process of subscribing for or fetching sensor data. Ideally, it should be possible to fetch the

sensor data using several different protocols to ease the integration with different applications.

Experiences from the applications above as well as from the work of others also make it

possible for us to draw some conclusions regarding the usage of different kinds of sensor

information in applications.

• Some context information, like the location of people, is very common in different

applications, thus sharing sensors among applications might be a good idea.

• Certain context information could be fetched using different types of sensors, such as

the identity of people and objects, the level of activity in a room etc, making it

5

possible to use higher-level abstractions to hide the details of the underlying sensor

implementation.

• Different applications may want the sensor data from one sensor in different formats,

or on different levels of abstraction.

• Many sensors are similar in terms of implementation, such as simple AD converter

based sensors like light and pressure sensors, or switch like sensors like motion

sensors and step sensors. This would suggest that general templates could be defined

that only requires minor changes to support different kinds of sensors.

Analyzing our own and others usage of sensors in pervasive computing applications we

can sort the majority of the sensors into four groups according to some specific properties.

• Identifiers: Sensors that can acquire an ID or other limited information from a

“foreign” object. Sensors of this kind include RFID readers, barcode readers, iButton

readers etc. Identifiers could for example be used to identify persons and objects that

arrive in a room or to various partially defined locations of the room. Some of the

identifiers are able to carry data and others just an ID. The identifiers carrying data

might hold specific information regarding their purpose and use.

• Passive environment sensors: Sensors that measure something continuously. These

sensors are passive in the sense that they don’t react on changes of the data they

measure. The passive sensors are mainly used when changes in the environment don’t

require immediate response, and when the fetching of sensor data is trigged by events

that don’t depend on the sensor value. Sensors of this kind could be temperature

givers, linear photoresistors, sound level microphones, GPS receivers etc.

• Active environment sensors: Sensors that actively indicates changes in the

environment. Sensors of this kind include different kinds of alarms, motion detectors

etc.

• Streaming sensors: Sensors that capture series of data where not only the latest piece

of data is interesting. This group of sensors include video cameras, microphones etc.

In the analysis, two other groups of gadgets also appear that are not technically sensors, but

show some similarities to sensors especially regarding implementation:

6

• Interactors: Tangible interaction devices that can give input to local applications.

This could be push buttons, knobs, sliders etc. The interactors can often be

implemented using the same components as the other sensors mentioned.

• Actuators: Simple output devices that can be used to provide different kind of

feedback and status information, often related to the sensors mentioned above. This

could be LEDs, simple speakers etc.

The problems listed above, together with the analysis of sensors and sensor usage has been

used as a requirement specification for a system that supports the development of sensor

based applications.

A multi purpose sensor infrastructure In order to deal with the problems described above we propose an infrastructure supporting

the development of sensor-based applications, and especially the development of so called

intelligent buildings. The architecture we have considered includes an embedded computer

platform with support for a number of different sensors as well as higher-level support for

distribution of sensor data to applications. The system has been implemented in a reference

implementation. The reference application is an extension of the knob service presented above

and does not cover all features of the infrastructure described below, but only those required

for that specific application.

Since a sensor could be anything between a video camera and a push button, it is very hard

to create a support system that can handle all kinds of sensors. We have therefore restricted

the supported sensors to typical instances of some of the different groups of sensors identified

above, where each group puts different requirements on the system:

Identifiers: To support these sensors the system needs to be able to listen for new objects that

arrive, ask for currently present objects and read data and IDs from these objects.

Active environment sensors: For these sensors the system has to be able to generate some

sort of events when a sensor value is changed or a threshold is reached. The event should

notify or trigger listening applications, so that they can react on the changes immediately.

Passive environment sensors: The system must provide means for applications to fetch the

values from these sensors with small latencies. It should also be possible to change a passive

sensor to an active sensor, so that it will generate events when the values reach certain

thresholds. The system must therefore provide means to set these thresholds.

7

Interactors: We choose to include simpler forms of such interaction devices among the

supported sensors, with the constraint that it should be possible to model the interactor in

terms of the sensor types above.

Sensor platform architecture

The proposed infrastructure consists of two parts: a sensor platform and a software

infrastructure for distribution of sensor data. The sensor platform consists of a small, network

enabled and programmable micro-controller device where a number of sensors can be

attached in parallel. The micro-controller contains firmware and drivers for the sensors as

well as support for network communication.

The idea is that these platforms could be integrated in a limited physical space such as the

rooms of a building. A number of different sensors could then be connected to the platform

according to the current needs.

The platform could later easily be modified to cover future modifications and additions of

applications in that locale. To make the sensor platform reusable and easy to adapt to new

sensors we propose that the controller should support a modular high level programming

language. The high level programming language increases the possibilities to create

abstractions and different kinds of sensor templates. In our reference implementation we have

been using the TINI board from Dallas Semiconductors, which is a micro controller

supporting Java Me.

Since the computing power of micro-controllers is limited, we propose that manipulation

and distribution of sensor data is performed on an external sensor server, running on a more

powerful computer somewhere on the network.

For the software on the micro-controller we propose a layered architecture as following:

Sensor connection library: Software that enables communication with the connected sensors

in a unified way. The API contains a set of routines needed to communicate over any of the

available communication ports. The ports or communication channels could be e.g.:

• Instructions to read and write to general I/O ports on a micro controller

• RS-232 comm. port

• Dallas OneWire network

• IR or IRDA ports

• RF or bluetooth

In the reference implementation a potentiometer was connected to a OneWire AD-converter

via a OneWire network. (OneWire is a technology from Dallas Semiconductors).

8

Sensor drivers: Native implementations to generalize the most basic instruction set for some

standard sensors or peripherals. Examples of the operations that have to be served by the

driver are read and write data operations, initialization of sensors and forwarding of interrupts.

Sensor address layer: Initializes the sensors and associates specific hardware addresses with

access IDs and symbolic descriptions that makes it possible to access the sensors without

digging into low-level details of how they are connected.

Sensor access layer: provides a middleware API to the sensors and implements strategies for

sharing the raw sensor data. The access layer provides methods to request sensor data and

keeps track of event- or poll-based subscriptions for sensor data.

Sensor communication server: Controls the communication channels for distribution of raw

sensor data. Administrates the connections to one or several context servers. This layer might

also convert the sensor data to fit the communication channel i.e. translations back and forth

to byte-packets for each transmission of data.

Figure 2. A schematic view of the sensor infrastructure

Context distribution infrastructure

The distribution of sensor data to applications is handled by a system called Context Shadow

[10]. The Context Shadow system is based on a blackboard architecture where certain entities

are represented as context servers. These servers serve as repositories for context information

related to that entity, and are implemented as tuple spaces. Typical entities that are provided

with a context server are persons and locations, but could also include entities like projects. A

context server contains two types of data: context information concerning the entity that the

server represents and links to other context servers. The context data from the sensor

platforms described above are posted to a context server representing the location where they

reside. The links between the context servers describes relations between the entities they

represent, e.g. a link between a person and a place indicates that the person is currently

9

present in that room. The links are established dynamically by using sensors that when

identifying e.g. a persons or a places also gets a reference to the related context server.

The Context Shadow system provides applications with a query interface with which they

can search the contents of a context server. The queries can also propagate to other context

servers through links found in the original server. In this way an application that is connected

to a personal context server, can get sensor information from the context server related to the

room that the person is currently in. Using the query interface the applications can either ask

for sensor data of different types like temperature or humidity, or it can ask to receive data

from a sensor with a specific ID. A successful query will return a tuple containing either the

actual context data if the sensor is a passive environment sensor, or a listener, that reports

events from an active environment sensor, if the sensor is an active sensor type.

Any application should be able to use any sensor data posted into a repository. This raises

questions on the format of the data. Different services might want the data in different formats

or at different levels of abstraction. Introducing Context Refiners solves this problem. A

Context Refiner reads data from the context server, transforms it and then adds the new data

back to the context server. A new Context Refiner module is easily added to the context

server and does not even have to reside in the same computer as the context server.

Related work The Context Toolkit system from Georgia Tech [1] is an example of a higher-level support

system with which you can incorporate sensor data in your applications. The system is built

with a widget approach making it possible for applications to incorporate context data in their

application about the same way as you incorporate GUI components. The Context Toolkit

system has several similarities with the context distribution infrastructure described above.

One major difference however is that the Toolkit system is more focused on designing

standalone context aware applications than it provides a shared infrastructure of context data,

which is the aim for the system proposed in this paper. The system does also not cover the

support for the low level implementation of sensors.

Other projects have however tackled the problem of providing multi purpose sensor

platforms. Smart-its [15] is a modular sensor platform that uses wireless communication to

allow multiple smart-its to communicate to with each other. The sensor platform hosts a set of

common sensors: a 2-axis accelerometer, pressure sensor, light sensor, temperature,

microphone and a ball switch / motion-detector. There are also actuators, a LED and a piezo

speaker. The platform can be extended with additional sensors using a set of I/O ports, serial

10

and I2C connectors. A library that makes the sensors and communication useable without

needing to know all implementation details is also provided for the sensor board.

The Berkly Sensor Motes [7] provides a platform similar to the smart-its with a slightly

stronger focus on the wireless communication and the software platform they call TinyOS

which is used on their embedded sensor platforms. In an example application a number of

sensor platforms were connected to motion sensors. The sensor data were then captured by an

application that provided information regarding free conference rooms.

Both the platforms described above have put a lot of effort into issues of power

consumption as well as minimizing the physical size of the platforms. There is also an

ambition to make the platforms mobile by using wireless communication. These ambitions

however put a lot of requirements on the platform, constraining it in many ways. Since the

system in this paper is focused on stationary implementations of the platforms we can chose

to use a bigger size platform with better processing and memory capabilities, making the

platform easier to program, and thus to modify for different purposes, since you can use

higher level programming languages.

Conclusions We have, from our own experiences identified a number of obstacles when creating

applications that includes different kinds of sensors. From these experiences, we have created

a sensor information infrastructure that supports the development of sensor-based services.

Support is given at sensor level by providing driver templates for common sensors, a

uniform addressing layer and a communication API. On a higher level, support is given for

distribution of context information to applications, refinement and transformation of context

information and to allow for applications to automatically detect sensors in new

environments.

The sensor infrastructure is extendable in the sense that new sensors easily can be added to

existing sensor platforms, and also due to that the distribution infrastructure easily

incorporates newly added sensor platforms and can handle an infinite number of platforms.

The infrastructure has been implemented in a reference implementation where parts of the

functionality has been implemented and tested.

References 1. Dey, A.K., Understanding and Using Context, Personal and Ubiquitous Computing, Special

issue on Situated Interaction and Ubiquitous Computing, 5(1), (2001)

11

2. Schilit, B., Adams, N., Want, R.: Context-aware computing applications. In: First

International Workshop on Mobile Computing Systems and Applications, (1994) 85-90.

3. Pham, T., Schneider, G., Goose, S.: Exploiting Location-Based Composite Devices to

Support and Facilitate Situated Ubiquitous Computing. In Proceedings of the 2nd

International Symposium on Handheld and Ubiquitous Computing (HUC2K), Bristol, UK,

September 25-27, 2000. pp. 143-156.

4. Starner T, Kirsch D, Assefa S: The Locust Swarm: An environmentally-powered,

networkless location and messaging system. The First International Symposium on Wearable

Computers, ISWC’97, Boston, USA, October 1997.

5. José, R., Davies, N.: Scalable and Flexible Location-Based Services for Ubiquitous

Information Access. In Proceedings of the 1st International Symposium on Handheld and

Ubiquitous Computing (HUC’99), Karlsruhe, Germany, September 1999. 52-56.

6. Essa, I., Ubiquitous Sensing for Smart and Aware Environments: Technologies towards the

building of an Aware Home, Position Paper for the DARPA/NSF/NIST Workshop on Smart

Environments, July 1999.

7. Conner, S., Krishnamurthy, L. and Want, R.: Making Everyday Life Easier Using Dense

Sensor Networks. In Proceedings of ACM Ubicomp, Atlanta Georgia, Oct 2001.

8. Werle, P., Kilander, F., Jonsson, M., Lönnqvist, P., Jansson, C., A Ubiquitous Service

Environment with Active Documents for teamwork support. In Proceedings of ACM

Ubicomp, Atlanta Georgia, Oct 2001.

9. iButton is a product of Dallas Semiconductor Corp <http://www.ibutton.com>

10. Jonsson, M.: Context Shadow: An Infrastructure for Context Aware Computing. AIMS

2002 Workshop in conjunction with the ECAI2002 Conference, Lyon, France, July 2002.

11. Wyckoff, P: Tspaces. IBM Systems J.,Vol.37,No.3, Aug.1998,pp.454-474,

http://www.almaden.ibm.com/cs/TSpaces.

12. TINI Board is a product of Dallas Semiconductor Corp <http://www.ibutton.com/TINI>

13. Johanson, B. and Fox, A., The Event Heap: A Coordination Infrastructure for Interactive

Workspaces. 4th IEEE Workshop on Mobile Computing Systems & Applications, June 2002.

14. Simple Object Access Protocol (SOAP) 1.1, W3C Note 08 May 2000,

http://www.w3.org/TR/2000/NOTE-SOAP-20000508/

15. Holmquist, L.E. Mattern, F. Schiele, B. Alahuhta, P. Beigl M. and Gellersen, H.W. Smart-

Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts,

Proc. of UBICOMP 2001, Atlanta, GA, USA, Sept. 2001.