Creating an ArcGIS Engine Application With C-Sharp and OpenGL

Creating an ArcGIS Engine application with C# and OpenGL

Danish Institute for Fisheries Research (DIFRES)

Brian James Cowan, [email protected], Software Developer Ole Skov, [email protected], Software Developer

Kerstin Geitner, [email protected], GIS Coordinator

Abstract As the ArcGlobe is rendered using the graphical OpenGL libraries it is possible to write custom applications that extend its functionality through the ArcGIS Engine 9.1. The purpose of extending the ArcGlobe using OpenGL was to create a tool that could be incorporated into a Windows application that could be used by anyone with an ArcGIS engine runtime license while not requiring a full ArcGIS desktop installation. The Windows application into which we intended to incorporate the tool was a Ship Information System to be used aboard the Danish Navy vessel Vædderen during its research expedition circumnavigating the globe. The application was written using the Microsoft .Net 2.0 language C# and therefore the ArcGlobe tool was also implemented in C#. In order to implement the OpenGL functionality from C# the Tao Framework was used. The functionality used to extend the ArcGlobe was for the rendering of Wavefront 3D models at specific global positions to signify the research vessel and models of instruments used at specific locations to perform certain experiments. The models of instruments were implemented so that they could be made interactive.

Introduction The Galathea3 Expedition [7][8] is a research expedition, planned and funded by the Danish Expedition Foundation, that is following the route taken by the previous Galathea expeditions in 1845-1847 and 1950-1952. The expedition, using the Danish Navy vessel Vædderen, will circumnavigate the globe, and while doing so the vessel will be a platform for a large number of research projects. In order for the scientists on board to be able to easily follow the progress as well as which research activities that have been performed at different locations, a tool that they could all utilise was required. The Windows application into which we intended to incorporate the ArcGlobe tool was a Ship Information System. The application was written using the Microsoft .Net 2.0 language C# and therefore the ArcGlobe tool was also required to be implemented in C#. The tool was required to be able to visually represent the route that the vessel had sailed and the position and type of research activities it had performed. The ArcGlobe control of the ArcGIS engine 9.1 is rendered using the graphical libraries that comprise OpenGL. The ArcGlobe control manages the initialization of all required OpenGL components and the creation of the main rendering loop. It also exposes some events (BeforeDraw and AfterDraw) that can be used to extend the functionality of the ArcGlobe control by adding custom OpenGL rendering to the graphics pipeline. [1] gives a good overview of ArcGIS 3D analyst and basic graphics rendering with OpenGL. The aim of this paper is to identify some useful layers to use with the Globe Control and introduce an improved set of OpenGL libraries for C#, the Tao Framework, and to show, using this framework, how 3D models can be rendered and interacted with on the ArcGlobe control.

Globe Layers The data used as a background for the scientific data from the expedition consists of the following six data layers, five of them originating from the ESRI Data & Maps Media kit [11], following an ArcGIS 9.1 license. Global Digital Elevation Model (ETOPO2) (shown twice) - raster This data represents gridded (2 x 2 minutes) elevation and bathymetry for the world. These data were derived from the National Geophysical Data Center (NGDC). The data set is from September 2001. The first visualisation of the data shows the depth in the ocean areas in 3D, using the grid values which represent depth of the layer. Only data below -5 meters are visualized, the colour is graduated from light to dark blue. The layer has a transparency of 15%. The second visualisation is using the layer to create the surface of the ocean. Also here, the data was graduated in colour from light to dark blue to get the best visual effect. The layer has a transparency of 40%. This layer is not shown when zoomed out beyond a distance of 200 kilometres. 2 km Cloud Free Image - raster WorldSat Colour Shaded relief represents a cloud-free view of the earth produced by stitching together hundreds of individual 1996 NOAA weather satellite images into a single mosaic image. The image has a cell size of 1 x 1 minute; this equals approx. 2 x 2 kilometres at the equator. This data was used to colour the land areas; it is using the elevation from the Global Digital Elevation Model. Satellite Image � 30 m These satellite images originate from Landsat images taken in the period from 1999 to 2002, their grid size is 30 x 30 meters. They were further developed by the Department of Geography of the University of Copenhagen. There are a number of rasters covering different areas along the route of the Galathea 3 expedition, i.e. mainly areas where the ship is going into a harbour. This layer is shown as wireframes when the scale is greater than 1:1.000. This layer is also draped over the Global Digital Elevation Model. Countries This layer is represented by two different layers. The first layer is the World Countries 2005 layer, which is visible when zoomed out beyond 1.000 kilometres. This data set represents the boundaries for the countries of the world, as they exist in January 2005. The scale is 1:15.000.000. When zoomed in beyond 1.000 kilometres, the World Vector Shoreline is visible. It has a general scale of 1:250.000, besides Denmark, which was cut out and replaced with a local map in a scale of 1:25.000. Furthermore, the map holds a layer containing latitudes and longitudes (10 x 10 degrees) and a layer containing world time zones. The globe properties were set, so that the data is shown with a vertical exaggeration of 10 for the floating layers. All data is in geographical coordinates, datum WGS84. The floating layers have a see-through position of +1. The water surface floats independent of the globe surface, draped on a surface of constant elevation. All the other layers float independent of the globe surface, draped on the Global Digital Elevation Model surface.

Tao Framework The Tao (道) Framework provides a number of libraries to help produce graphical applications. The Framework provides libraries for the core OpenGL libraries as well as libraries for graphics card programming, working with Images and Audio resources among others. The Tao Framework is an Open Source, Cross Platform, Language Neutral set of libraries. The Tao Framework, to our knowledge, started with the CsGL [2] library with the aim to extend the OpenGL libraries it

covered, making it cross-platform, i.e. running on both Mono and Microsoft runtime environments. The binding to the GL, GLU and GLUT libraries was made clearer by having a specific Tao library for each and they were created to be CLS-compliant, so they could be used by any CLS-compliant language, C#, VB.Net etc. For a more comprehensive history of the Tao Framework see the Forum post �Where did the name �Tao� come from?� under the General Discussion forum of reference [3].

Rendering events There are a number of useful events exposed by the IGlobeDisplayEvents_Event interface of the GlobeViewer.GlobeDisplay on the GlobeControl. The two events that we found most useful were the AfterDraw and BeforeDraw events. These could be initialised by: globeDisplayEvents = (IGlobeDisplayEvents_Event)axGlobeControl1.GlobeViewer.GlobeDisplay; if (globeDisplayEvents != null) {

globeDisplayEvents.BeforeDraw += new IGlobeDisplayEvents_BeforeDrawEventHandler( OpenGLPreRender );

globeDisplayEvents.AfterDraw += new IGlobeDisplayEvents_AfterDrawEventHandler( OpenGLRender );

} protected bool OpenGLPreRender( ISceneViewer viewer ) {

//... //Pre Globe Render Code Goes Here //...

} protected bool OpenGLRender( ISceneViewer viewer ) {

//... //Post Globe Render Code Goes Here //...

} The BeforeDraw event is executed before the globe is rendered while the AfterDraw event is fired after the globe has been rendered. We found that the BeforeDraw event was useful for setting up the light source(s) to be used for the 3D models, this only needed to be performed the first time the event was fired. The AfterDraw event was what we used as the work horse for rendering custom graphics onto the globe. This was where the models and other custom graphics were rendered. The first time a set of models was to be rendered they needed to be loaded and data specifying where they should be rendered was retrieved. This data and any additional models are then updated behind the scenes so as not to disturb the user�s experience while using the tool. For further details on this updating see the section �Threaded Data Retrieval�. Among other useful events that we have used/experimented with were the InteractionStopped and ActiveViewChanged. The InteractionStopped event is fired every time the user completes a navigation process within the GlobeControl. This can be used to scale a model after the user has completed a zooming process. This can be useful so models can be rendered with a large scale factor when the viewer is further away from them, but when they zoom into the model it is scaled to a more appropriate size. The ActiveViewChanged event is fired when switching between the standard view and scene view. This can be useful when certain graphical effects should only be displayed in a certain type of view only. For a full list of available events see [4].

Parsing and Rendering 3D Model Files The Wavefront OBJ file format is used to store and exchange 3D vector data. These data files can come in either ASCII or binary format. The binary format is a proprietary format and is undocumented, so only the ASCII format was used in our tool and is discussed here. However, the description given here will not be a comprehensive description of the file format, this can been seen from [5], but will only be concerned with the formatting used.

OBJ File Format The OBJ file generally starts with a header section supplying a comment regarding the contents of the file, although this is not required. Comments may also be supplied throughout the file to supply useful information, number of vertices in a section, identifier of what the vertices of a section relate to, etc. Comment lines begin with a hash mark (#). At the start of each line is a set of character(s) identifying the type of data contained. Additional blank spaces at the end of a line or blank lines can be added to an OBJ file to make them more readable. The line identifiers we used were:

Identifier Type Format

v Vertex x y z vt Texture coordinate u v vn Vertex normal x y z f Face v/vt/vn usemtl Material text

Table 1: OBJ line identifiers used. The value denoted by x, y & z are floats while u & v are also floats but range from 0 to 1. The values v, vt & vn are integer values greater than 0. In an OBJ file vertex, texture coordinates and vertex normal values are all listed before the section of the file containing the face data. In general the vertex, texture coordinate and vertex normals are placed in there own block, but this is not a requirement. The data was processed by reading all vertex, texture coordinates and vertex normals into indexed lists. The face data lines then list a set of v/vt/vn indices (although vt and vn values are optional) that make up a face of the model being defined. At the start of each face section there is a �usemtl� identifier line and this defines the material settings from the MTL file that should be used on the face being specified.

MTL file Format Identifier Type Format

newmtl New material Material name map_kd Texture image File name ka Ambient lighting f f f kd Diffuse lighting f f f ks Specular lighting f f f ns Shininess f

Table 2: MTL line identifiers used. The f in the format column stands for a float value The line identifier �newmtl� supplies a name and identifies the start of a new set of material settings. The �map_kd� identifier specifies the name of the image file used to texture map to sections of the model using this set of material settings. This is not a required setting and is quite frequently not specified. The ka, kd and ks identifiers specify the values to be used for the ambient, diffuse and specular light of the material. The �ns� identifies the �shininess� of any specular highlight present. The lower the value, the less specular highlight is present; this is by default 0.

For each �newmtl� identifier we created a data structure and stored the values supplied within this data structure. The appropriate data structure could then be referenced to a specific face or set of faces of a model to provide the material setting for them.

Figure 1: Representation of data structure used to parse and represent an OBJ file with association material settings

Model Rendering and Orientation Display lists are an optimised rendering technique as they cache the OpenGL commands for specific geometry so they can be applied multiple times where required. A display list must first be compiled and how this is done is shown below. When compiling a display list its geometry must be fully specified. For further explanation and examples of display lists see chapter 7: Display List from [10]. m_intDispListNo = Gl.glGenLists( 1 ); Gl.glNewList( m_intDispListNo, Gl.GL_COMPILE );

Render( ); Gl.glEndList(); When the model is first created it is fully rendered as a triangle mesh into a display list. This was achieved by creating a triangle primitive for each triplet of vertices in a Face Collection. The final triangle of a Face Collection used the last 2 vertices and the first vertex of the Face Collection as its coordinates. If a vertex normal and/or texture coordinates existed for a given vertex they were associated with it. On subsequent times a model was rendered its corresponding display list was called. Gl.glCallList( m_intDispListNo ); When rendering the display list of a model, the model should be orientated in a way that the y axis of the model is aligned with the normal vector of the globe at the position to be rendered. The normal vector at a given position X on the globe can be computed by the normalised cross product of the coordinate X and the origin of the globe. From the resulting vector the x component can be used to construct a rotation matrix around the X-axis while the z component can be used to construct a rotation matrix around the Z-axis. These rotation matrices can be applied by calling Gl.glMultMatrix.

Figure 2. Θ & Φ show the angle of rotation to align the Y axis of the model with the normal vector at the position on the globe surface. If a model needs to be aligned in a specific direction (0 to 359°), this can be achieved by using Gl.glRotated and rotating around the Y-axis (up vector).

OpenGL render effects One of the issues that needed to be addressed in order to render the models well within the ArcGlobe control was that of the lighting and material effects to be used upon the models. The lighting and material effect states are controlled from within the OpenGLRender method, this method is the one called by the AfterDraw event on the GlobeDisplay. From using the function call glIsEnabled, as well as a bit of trial and error, it was possible to work out which of the light sources were being used to render the globe and which were available to use for custom objects. To ensure that a standard local illumination lighting effect was used on the models rendered, the globe�s lighting effects were disabled during the model rendering process then re-enabled at the end to ensure that the globe itself was rendered correctly. The ArcGlobe control by default uses the material colouring mode GL_COLOR_MATERIAL. This is a technique for minimising performance costs when only single material parameters for most vertices in a scene need to be changed. When more than a single material parameter needs to be changed, as is the case with Wavefront models, it is best to use the glMaterialf* call. The material properties were taken from the appropriate setting in the MTL file associated with the model and were set using the function called glMaterialfv. In order for these properties to take effect the GL_COLOR_MATERIAL had to be disabled for the duration of the model rendering process then re-enabled at the end to ensure that the models and globe itself were rendered correctly.

Figure 3 � The model used for a Triaxus activity is shown. The model to the left is rendered without lighting effect and no material, the model in the centre is rendered with lighting effect but no material. The model to the right is rendered with lighting and material effect, including an ambient lighting effect of 0.2 in R, G and B channels. protected void OpenGLRender( ISceneViewer viewer ) {

Gl.glDisable( Gl.GL_BLEND ); Gl.glEnable( Gl.GL_DEPTH_TEST ); Gl.glShadeModel( Gl.GL_SMOOTH );

Gl.glEnable( Gl.GL_LIGHTING ); Gl.glDisable( Gl.GL_LIGHT0 ); Gl.glDisable( Gl.GL_LIGHT1 ); Gl.glEnable( Gl.GL_LIGHT2 ); Gl.glDisable( Gl.GL_COLOR_MATERIAL ); //... //Model Rendering Process //... Gl.glEnable( Gl.GL_LIGHT0 ); Gl.glEnable( Gl.GL_LIGHT1 ); Gl.glDisable( Gl.GL_LIGHT2 ); Gl.glDisable( Gl.GL_LIGHTING ); Gl.glEnable( Gl.GL_COLOR_MATERIAL );

Gl.glEnable( Gl.GL_BLEND ); Gl.glDisable( Gl.GL_DEPTH_TEST );

}

Threaded Data Retrieval In general, OpenGL applications render their scenes within an infinite loop. The scene being viewed alters either according to the movement of the viewer or some event taking place within the scene. The GlobeControl functions in a similar fashion whereby the scene is redrawn during

the process of the user interacting with the globe, after a user has interacted with the globe or when the Redraw method is called on the GlobeDisplay.ActiveViewer. If the data that is required to be rendered needs to be retrieved every time the AfterDraw event is fired this would put a considerable strain on the performance of the application. In order to avoid this problem we decided to only retrieve a full set of data the first time it is requested and then use a separate thread to continually monitor when additional data is available. When additional data points (e.g. the cruise route) become available, they are retrieved by this thread and inserted into the data structure used to store the cruise route. The next time the globe view is redrawn the additional data points are also drawn, without any time delay involved in retrieving the data.

Figure 4: A simplified overview of the structure how the threaded data retrieval functions The threaded data retrieval can also be used to render additional activities as they become available. In this case it may also require a new model to be loaded, but again, as this is being performed by a separate thread to the user interface interaction thread, this process should not affect the users experience with the ArcGlobe tool.

Additional features of ArcGlobe Features for navigation, switching between viewing mode, etc. can be selected from the ToolbarControl linked with the GlobeControl. However, if there are some custom features that are required these can easily also be added and their functionality programmed directly. Two such features we added were a HitTest feature for the activity symbols and a zoom-to-ship feature.

HitTest The HitTest feature was created so that a user could click upon an activity symbol and details about that activity would then be displayed. This was performed by creating a line vector from the current camera position and the position that was clicked upon the globe. The camera position was retrieved by: IPoint po = m_ActiveView.Camera.Observer;

While the position clicked upon the globe was retrieved by: IPoint pt = new PointClass(); object powner; object pobject; axGlobeControl1.GlobeViewer.GlobeDisplay.Locate(

axGlobeControl1.GlobeDisplay.ActiveViewer,e.x, e.y, true, true, out pt, out powner, out pobject );

Once the line vector was defined, intersection tests between the bounding box of an activity and this line could be performed to determine which activity was hit. Although this hit test is not 100% accurate, as it is possible for the line vector to intersect an activity bounding box without intersecting the activity object itself as activity objects do not always completely fill the bounding box, it still has sufficient accuracy for our needs.

Zoom to Ship By manipulating the ActiveViewer.Camera.Azimuth, Inclination and ViewingDistance in the GlobeDisplay, while ensuring that the viewing mode was not set to scene view, it was possible to create a simple zoom to ship function whereby the current Longitude position was set to the Azimuth, the current Latitude position set to the Inclination and the viewing distance given an appropriate value.

Future There are a variety of features that would be interesting to include in future versions of this tool. Some of these would relate to the layers used on the globe, e.g. additional data that could be included, while others could relate to additional effects being rendered on the globe. One current idea for an extension is to e.g. embed video clips with certain activities, so that associated video clips can be played when activities are selected. Another is to plot a simulation of a fishing trawl while in scene view in order to visualise the depth and coverage of a fishing trawl. It might also be interesting to add real-time Day/Night lighting effects so the globe would be rendered with a day side and a night side.

References 1. [Montgomery & Moore 2004] - Developing 3-D Analyst Enhancements Using OpenGL 2. CsGL project on SourceForge (http://sourceforge.net/projects/csgl) 3. http://www.taoframework.com/Home 4. ESRI ArcGlobe Documentation 5. http://www.fileformat.info/format/wavefrontobj/ 6. http://www.csit.fsu.edu/~burkardt/data/mtl/mtl.html 7. http://www.galathea3.dk/uk 8. http://www.difres.dk/galathea 9. [Pilouk & Fine 2006] - Best Practices for Developing with ArcGlobe Handling and Rendering

Dynamic Content in 3D 10. [Shreiner, Woo, Neider & Davis] OpenGL Programming Guide 5th Edition. 11. http://www.esri.com/data/data-maps/overview.html

Acknowledgements We wish to thank Informi GIS and ESRI for donating the software used in the Galathea 3 project.

Appendix I � Screen Shots

Figure 5: Initial view of the globe, a red line can be seen extending from the Faroe Island to Greenland, showing the cruise route of the ship, when moving closer additional features will be visible.

Figure 6: Scene view of a section of the cruise route off the coast of Greenland is shown. Two activities were performed during this section and their symbols can be seen. In the lower left corner of the screen the results of clicking in the left most activity are shown.

Figure 7: On this view there are white rectangles shown over parts of Iceland and Greenland, these indicate places where detailed satellite images are present.

Figure 8: Close up view of the satellite image around Nuuk (Greenland) and the routes sailed through the coastal fjords.