Citadel Push Email guide v1

Citadel Push Email Install guideRevision 1Sunday, 10 June 2007

Citadel Push Email Install Guide | http://bionicmessage.net/citadel/

http://bionicmessage.net/citadel/

© Mathew McBride 2007Licensed under the Creative Commons Creative Commons Attribution-Share Alike 2.5 Australia License

Components of the Citadel Push Email solution have been developed under my free time under a view of a starting a business should demand warrant; Due to this I do not accept any direct donations, however, I am willing to work under contract should you have any needs that need to be fulfilled. http://bionicmessage.netThe Citadel Email connector for Funambol was coded in Australia from local and imported code :)

Questions, Comments and HelpPlease email [email protected] with any questions and comments relating to the Citadel Email connector.

Additional LiteratureDocumention for various aspects of Funambol 5 is available from http://forge.objectweb.org/project/showfiles.php?group_id=96&release_id=1981 Documention for the PocketPC client is available from http://forge.objectweb.org/project/download.php?group_id=96&file_id=8210The Citadel Email Connector follows a similar design to the Funambol GroupDAV Connector ( http://bionicmessage.net/funambol/ ), and documentation for it is helpful here too. In particular, the install process for the GroupDAV connector has been documented at the OpenGroupware Documentation Project - http://docs.opengroupware.org/search?SearchableText=Funambol



http://docs.opengroupware.org/search?SearchableText=Funambol

http://bionicmessage.net/funambol/

http://forge.objectweb.org/project/download.php?group_id=96&file_id=8210

http://forge.objectweb.org/project/showfiles.php?group_id=96&release_id=1981

mailto:[email protected]

http://bionicmessage.net/

Table of ContentsCitadel Push Email Install guide..........................................................................................................1

Questions, Comments and Help.......................................................................................................2Requirements...................................................................................................................................4Requirement One – Java Development Kit.....................................................................................5Requirement Two – Database..........................................................................................................6

HSQLDB.....................................................................................................................................6PostgreSQL.................................................................................................................................6

Requirement Three - Installing Tomcat 5........................................................................................7Installing Funambol.........................................................................................................................8

Environment Variables................................................................................................................8Placing the right files..................................................................................................................8Editing the configuration file......................................................................................................8Installing everything...................................................................................................................9Running it....................................................................................................................................9

Installing and Launching the Funambol Administration Tool.......................................................10Creating an email connector instance.......................................................................................10If there are problems.................................................................................................................10Changing the administrator password.......................................................................................10

Setting up and syncing the Funambol client on PocketPC: Screenshot guide...............................11Configuring SMS push using the Clickatell SMS gateway...........................................................15

Clickatell basics........................................................................................................................15Creating the BMClickatell configuration file...........................................................................16Configuring Funambol to use BMClickatell.............................................................................16

Troubleshooting ............................................................................................................................18Some messages show up but others don't.................................................................................18Logging and turning up the log level........................................................................................18Store logs...................................................................................................................................18Tomcat logs...............................................................................................................................18Recovery syncs.........................................................................................................................18

Appendix: Funambol J2ME client.................................................................................................19



RequirementsTo set up push email with the Citadel server, the following requirements need to be satisified:

● An operating system with a Java Development Kit● Some command line knowledge of your operating system● A Funambol 5 server1

● An account with the SMS gateway at Clickatell.com ○ OR directly-accessible (non-NAT) internet addresses assigned to each of your phones

from your mobile carrier. This is the only route applicable to those in North America● The Citadel email connector from bionicmessage.net● Windows Mobile 5/6 PocketPC devices

○ One could use Windows Mobile “Smartphone” devices but the developer has not tested them in production

Citadel Push Email builds on the Funambol Push Email solution, and has a goal to be more friendly in a typical Citadel environment than the carrier or enterprise-oriented solution that Funambol distributes. This involves using our own email “module” in Funambol. This guide will describe how to install a component that uses Clickatells' SMS gateway to send a notificiation SMS2. While paying for an SMS allowance for each user is an additional cost, the alternative is to ask your carrier to provide public internet addresses for each of your phones (an alternative could be a private APN – at large cost). After multiple tests, however, using TCP/IP notification is not viable battery wise. While the use of a Windows Mobile 5/6 PocketPC device is recommended, Funambol has clients for other platforms, such as Blackberry and J2ME. The J2ME client has been tested, however, the developers are yet to get SMS push working with it.

1 Funambol 6 stable has been released at the time of writing; however, until Funambol 6 can be stability proven with the connector stay with 5 instead.

2 Despite the notification component being coded to use Clickatell, modifying it to use another SMS gateway with a HTTP interface would be trivial. The only requirement is the ability to specify the UDH (message header) and message contents in binary.



Requirement One – Java Development KitA Java 5 Development Kit is required to run Funambol. For most installations, the development kits that Sun distributes will suffice, however, alternatives can be found from several vendors, such as BEA (JRockit) and IBM (J9). Both Sun's Java 5 JDK and IBM's J9 have been tested.Java 5 Release 12 is the latest JDK release from Sun, at the time of writing, in the Java 5 series and can be downloaded from http://java.sun.com/javase/downloads/index_jdk5.jsp (Java 6 JDK's are available but the developer has not deployed any installations on one)IBMs' J9 JDK is similar to the version distributed by Sun, but uses IBM's own virtual machine. More importantly, this is the only reasonable JDK available to Linux/PPC users, and also happens to be what the developer is running on his production setup. IBM's developer kits are available free of charge, but registration is required: http://www.ibm.com/developerworks/java/jdk/linux/download.html Linux/PPC users will require the 32/64-bit iSeries kits.Assuming you download a self-extracting archive, simply chmod +x the archive and run in the place you wish to keep the developer kit in.



http://www.ibm.com/developerworks/java/jdk/linux/download.html

http://java.sun.com/javase/downloads/index_jdk5.jsp

Requirement Two – DatabaseFunambol requires a backend database to store certain configuration settings. This can be any JDBC database, however, HSQLDB and PostgreSQL are the two preferred backends. MySQL may also be used, however, it has now been obsoleted by Funambol.

HSQLDBHSQLDB is a native Java SQL-compliant database, and is more conveinient where PostgreSQL is not in place on the system. HSQLDB may be downloaded from SourceForge: http://sourceforge.net/project/showfiles.php?group_id=23316The HSQLDB zip file contains many utilities, however, all we are interested in is hsqldb.jar, which can be found in lib/I suggest you copy hsqldb.jar to a stand alone location and use this startup shell script:#!/bin/sh

java -classpath hsqldb.jar org.hsqldb.Server -database.0 fnbl \

-dbname.0 fnbl

This starts a HSQLDB server with one database – 'fnbl'

PostgreSQLA JDBC driver .jar for PostgreSQL can be downloaded from http://jdbc.postgresql.org/ .Ensure you have a database set up for Funambol before we install it.



http://jdbc.postgresql.org/

http://sourceforge.net/project/showfiles.php?group_id=23316

Requirement Three - Installing Tomcat 5Tomcat 5 can be downloaded from tomcat.apache.org at http://tomcat.apache.org/download-55.cgi .Note that you need to download 5.0.30-beta not 5.5.Tomcat should be able to run 'out of the box', however, as per Funambol's recommendation3, Tomcat should be configured to listen on port 50001 instead of the default 8080. To do this, open conf/server.xml in a text editor and find: 

<Connector port="8080"

Change 8080 to 50001, save the file, and there should be no need to touch Tomcat from on then.

3 Funambol's J2ME client only listens for SMS notifications on port 50001, the reason being that SMS notifications on other ports are usually intercepted by the phone software itself (i.e Symbian S60 where there already is a SyncML client), and is therefore discouraged by the WAP specs.



http://tomcat.apache.org/download-55.cgi

Installing FunambolFunambol 5.0, the version of Funambol recommended for the Citadel Email Connector can be downloaded from ObjectWeb at http://forge.objectweb.org/project/download.php?group_id=96&file_id=6954 .

Environment VariablesAfter extracting the zip archive, the actual Funambol server files will be located in Funambol/ds-server . Before we start configuring the Funambol server, I recommend creating a bash script with the environment variables that can be used with the “source” command:#!/bin/sh

export JAVA_HOME=<PATH_TO_JDK>

export J2EE_HOME=<PATH_TO_TOMCAT>

Save this file to a permanent location.

Placing the right filesThe Citadel Email Connector comes in two packages – the Funambol module itself (citadel.s4j) and a package of libraries used by the connector (email_libs). citadel.s4j should be placed in ds-server/modulesThe contents of email_libs (the .jar files) need to be placed in ds-sever/default/libs.

Editing the configuration fileThe installer configuration file for Funambol is ds-server/install.propertiesThe first line important to us is “dbms=”. As the configuration file explains, this needs to be set to the database you have chosen.Example configuration blocks for each database are listed. Here is an example for HSQLDB from a running sustem:jdbc.classpath=/Library/Funambol/database/hsqldb.jar

jdbc.driver=org.hsqldb.jdbcDriver

jdbc.url=jdbc:hsqldb:hsql://localhost/fnbl

jdbc.user=sa

jdbc.password=

The next important line is “modules-to-install=”. If you only want to use the Citadel email connector, all you need is:modules-to-install=foundation-3.0.9,citadel




Installing everythingTo install everything, ensure that you have placed the environment variables into the current environment by using source on the shell script you have created:source ./environment.sh

Then launch the installer:bin/install.sh tomcat50

When asked, answer “yes” to replace the database for each module, as this is a new install.

Running itRun bin/start.sh to start Funambol. Hopefully the first launch should go without incident



Installing and Launching the Funambol Administration ToolThe Funambol Administration Tool is a desktop Java application; it needs to be run on an X11 environment/Windows/Mac etc., and doesn't need to be on the server. A cross platform tarfile can be downloaded from http://forge.objectweb.org/project/download.php?group_id=96&file_id=6957 .After extracting the admin tool, it can be run by launching bin/funamboladmin. JAVA_HOME will need to be in your environment variables, simply export in a similar way to the environment bash script.

Once the administration tool is launched, go File->Login. Remember to change the port from 8080 if you have done, and then, if all is well, login.

Creating an email connector instanceThe first point of attack is to head down to the Modules section in the tree, where you should see citadel as a node, and navigating down that, Citadel Email. Double click on the Citadel Email node to launch a configuration applet. Before we launch into connector specifics, you need to create a “store” directory on the server, where the connector will store object caches and logfiles for each client using it. This needs to be read-writable and creatable by the Funambol server, but should not be readable by other users than the one running Funambol. Then we have the configuration options for the connector itself.

SyncML Source is the name that clients will “ask” Funambol for when syncing.Sent Items, Drafts and Trash folders can be synced by the connector, but in the case of the first two, support for doing so was temporarily dropped by Funambol clients due to differences in how various mobile email clients handle them. In the case of Trash, you can sync it, but for most people, its not worth the wasted sync time.Client timezone is the timezone offset from GMT, i.e +1000, that emails synced to clients will carry. For the Windows Mobile client, this needs to be correct as its email client won't translate the times for us.Click Apply to create the connector instance. If all goes well, there should be no problems.

If there are problemsJust about all connector creation problems are due to the contents of email_libs not being extracted properly into ds-server/default/libs. Check that they are, and run the Funambol installer again, this time, you can answer “No” to database replacement questions as the database is already there.




Changing the administrator passwordYou probably want to change the administrator password from its default. To do this, head to the “Users” node, double click it to bring up the user configuration applet. Hit the Search button to bring up a user list, select admin and hit the Edit button. You can enter a new password here, but Funambol will also demand a first and last name and email address for the administrator, though.



Setting up and syncing the Funambol client on PocketPC: Screenshot guideFunambol 6 clients for the PocketPC can be downloaded from http://forge.objectweb.org/project/showfiles.php?group_id=96&release_id=1983 .You can download Funambol 6 in either a .cab archive for on the device install or a Windows installer that will conduct the process over ActiveSync. After Funambol has been installed on the device, launch it.


Figure 1: Funambol screen on first install. The soft key menu has multiple options, to start off with, go to Account.

Figure 2: Funambol accounts screen. Enter in the Funambol URL for your server, which is in the format of:http://host:port/funambol/dsEnter in your Citadel user name and password.Hit Save and head to Sync Settings from the menu.


http://forge.objectweb.org/project/showfiles.php?group_id=96&release_id=1983


Figure 3: The sync settings screen allows you to choose the data on the device to sync. For now untick the defaults (Calendar, Tasks) and tick email. Tap Details next to Email to continue

Untick everything, tick email

Figure 4: This screen allows you to customize the emails you receive, and how much text data to accept per email. The Citadel Email connector is capable of passing along attachments – if you wish to receive them. Tap Advanced Settings to go to the next step

Figure 5: The advanced email settings screen holds the settings for the server sync source that you entered in the admin tool earlier, the email display name and email address to use. (Note: theres no guarantee that the identity settings here will be honoured – as with any native Citadel client you should do this by editing you're vCard. )Continue to tap Save until you return to the original screen until you return to Figure 1



Figure 6: Select Notifications from the menu and make sure notifications aren't suppressed (so you'll receive mail notifications – sometimes) and you can also change the sync start behavior. Tap Save to return to the initial screen.

Untick

Figure 7: To start the first sync either tap Sync All at the bottom or double tap the Email part.

Tap until sync

starts

Tap to sync all activated

options



Figure 8: On first sync, or a “recovery” sync, there will be a message asking if you wish to do a full sync. As it says, sit back, tap Yes, as the first sync might take a while depending on network type and server power.

Figure 9: Once the sync starts, the sync process will start in the background, as denoted by the Funambol icon in the icon bar up to. Feel free to carry on with other tasks on your device as you wish.

Figure 10&11: Once sync is finished, the new message count will show on the Today screen.Citadel mail can be accessed via Windows Mobile's Messaging application. On Windows Mobile 5 (left), the Funambol box can be accessed by the 'show' menu. On Windows Mobile 6 (right), the messaging application will present Funambol in the mailbox list at the start.Note: For performance reasons the Citadel email connector currently does not replicate the “seen” status on the remote device, but any messages read on the device will be marked seen on the next sync.

From there your Citadel/Funambol mail can be accessed as a normal mailbox in the messaging application, and you can read/send/delete messages as normal.

Enjoy!


Configuring SMS push using the Clickatell SMS gatewayTo have your device start an email sync when new email is received in your mailbox, i.e “push email”, you need to either set up Funambol for SMS push, or persuade your mobile carrier to give you a public internet IP address for your device.Even if your carrier can give you a public IP address, using this method of push notification is not recommended by us. Doing so requires having your device on with backlight off all day – not a good idea on modern PDA devices. My 'horror situation' testing has confirmed this – see http://tinyurl.com/28j5rl . Unfortunately, for those in North America, you have very little choice at the moment apart from TCP, for a list of carriers confirmed to offer public IP addresses usable by Funambol, see the Funambol wiki:https://wiki.objectweb.org/sync4j/Wiki.jsp?page=PushTechnology

Configuring Citadel to notify FunambolCitadel includes an inbuilt module to 'ping' Funambol when a user receives mail in their inbox. This can be configured as part of the system configuration. A shot from webcit is shown below.

The configuration options for Push Email accept four options: the Funambol server host, its port, the email sync source your devices are configured to use, and the authentication details for Funambol's administrator account in Base64. This is in the unencrypted format user:pass, so for the default Funambol administration password this will be “admin:sa” (remember that you we're told to change

the password!). To transform this to Base64, on most systems you can use OpenSSL:openssl enc -base64

WARNING: OpenSSL will try and add a new line character (“o” at the end of the generated Base64 before the “==”) when entering via stdin. To encrypt the password properly, you need to enter the admin:pass bit and hit Ctrl-D twice afterwards. You can then cut and paste this into webcit or the Citadel text client.

Clickatell basicsClickatell is a SMS gateway which can be used to send SMS'es, in particularly, exotic things like WAP push, over the Internet. Most importantly, SMS gateways like Clickatell charge rates many times lower than what the wireless carriers charge customers up front, and no phones/datacards are needed. This is particularly important as we aren't in bed with every mobile carrier like our friends from Ontario, Canada are. Please note that the use of Clickatell doesn't mean any specific endorsement by the developer; Clickatell was chosen based on recommendations on forums, capabilities, documentation supplied, and of course, cost. The notification module that has been coded for Clickatell can be hacked to use an SMS gateway of equivalent features with a JDK, text editor, and five minutes worth of work :)Clickatell offers multiple products, the one required here is the SMS gateway itself, or “Clickatell Central (API)”, details of which are at https://www.clickatell.com/products/sms_gateway.php and you can sign up at https://www.clickatell.com/central/user/client/step1.php?prod_id=2Once you have signed up for Clickatell, you will be provided with some basic details:



https://www.clickatell.com/products/sms_gateway.php

https://wiki.objectweb.org/sync4j/Wiki.jsp?page=PushTechnology

http://sourceforge.net/mailarchive/message.php?msg_name=C28B4455.19FB%[email protected]

http://tinyurl.com/28j5rl

● Username● Client ID● Password

Once you log into the Clickatell Central administration interface, head right to “Manage My Products”.On top of the Clickatell account, you need to create what Clickatell calls a “connection”. This is an ID which identifies an application using your account. We need to define a HTTP connection, which can be done with the drop down menu on that page:

All that is required to be defined for a connection is the ID for it. After it is created, take note of the API ID that has been assigned.

Creating the BMClickatell configuration fileBMClickatell is the name of the notification module that has been created for Funambol, and you would've installed that earlier as part of email_libs. To use BMClickatell we need to create a bean configuration file in the Funambol directory structure, specifically in ds-server/config/com/funambol/server/notification. Its an XML file and can be called anything. In the example below I've called it ClickatellSender.xml. An example configuration file is included in the distribution.



As seen in the example above, you will need the following details from Clickatell:● Clickatell Username● Password for Clickatell account● The numberic API ID (shown in the screenshot before hand) assigned to the Clickatell

'connection' you created.Once those details are filled in, save the file. BEWARE: The configuration file has to be valid XML. Try to keep your editing to the details that need to be changed.

Configuring Funambol to use BMClickatell.To configure Funambol to use BMClickatell, start by launching the Funambol administration tool again. After login go to the “Devices” applet. Click “Search” to bring up a list of all devices that have synced – of which there should be one. Double click to see the properties for it. Three properties need to be configured for push:

● MSISDN – i.e the phone number for your device. In the format of country-code-[device-number-minus-zero]. i.e for an Australian mobile number like 0415011501, this will be 61415011501.



● Notification builder – this should be set to com/funambol/server/ notification/ DSNotificationBuilder.xml.

● Notification sender: this should be set to the file you created, i.e com/funambol/server/ notification/ClickatellSender.xml

In addition, the first time you edit a device config, Funambol will demand the type and description properties be defined – just fill in anything there. Click Save, and you're done!Send yourself an email and see if the device receives it after a few minutes!


Example device configuration.


Troubleshooting The Citadel Email connector, while throughly tested in service by the developer, things may still go wrong.

Some messages show up but others don'tThe connector sends the device output from Citadel's MSG0 command, and uses JavaMail to encode the text from it into an RFC822 message. While this approach works well most of the time, some messages – particularly Asian spam, may be unparseable by the device. If you find a real message that fails to be received, please send the author a copy.

Logging and turning up the log levelTo adjust the Funambol log level, go to Server Settings->Logging and select the Funambol node. This will bring up a Logger settings applet. Change the level to ALL.Due to the sheer amount of log data Funambol generates you probably want to set up a log rotation of 5MB instead of 10MB. As stated in the logger settings panel, Funambol's log file is in ds-server/funambol/logs/

Store logsThe “store” directory contains individual directories for each device that syncs. In addition to the database for each device, two HTML files are generated for each sync - “connector” and “store”. These aren't too useful without knowledge of the connector's inner workings, but can reveal the Message-ID of the message that caused the connector to halt.

Tomcat logsThe Tomcat launcher captures the STDOUT and STDERR while it is running Funambol. These can be found in Tomcat/logs/catalina.out. In rare cases, some Java exceptions may be logged here.

Recovery syncsTo force the device to do a 'full sync' – i.e replace its database and start fresh in the PocketPC Funambol client, go Menu->Tools->Recover. Full sync's can also be forced remotely, in the Principals applet in the Funambol administration tool. Select a username and device pair, double click to see its details and click reset for the email sync source.



Appendix: Funambol J2ME client

The Funambol J2ME client can be used with mass-market mobile phones that support J2ME. While it does support push email notification via SMS, we have yet to be able to make it work (please email me if it does for you!). However, using the J2ME client certainly beats out using your devices web brower to check mail. It supports message size and day filters so it is a good alternative to inbuilt device mail clients if you are concious of wireless data usage.Configuration of the J2ME client isn't a hard task if you are familiar with the configuration options. Note that a limitation of the J2ME client is that it doesn't share your phones address book; it will only share SyncML address sources, which can be those inbuilt into Funambol, or the GroupDAV Connector to share your Citadel address book. You can also have the client not share the address book by entering a bogus name into the address book source field, as shown in the example configuration image.

Citadel Push Email guide v1

Documents

funambol administration

java development

citadel email

email sync

funambol j2me

email address

j2me client

groupdav connector