KHEOPS Technologies JMap Spatial OLAPTM 1 Copyright KHEOPS Technologies 2005 On-Line Analytical Processing for Spatial Databases Innovative technology to support intuitive and interactive exploration and analysis of spatio-temporal multidimensional data Off-the-shelf generic solution for Geographic Business Intelligence KHEOPS Technologies JMap Spatial OLAPTM 2 Copyright KHEOPS Technologies 2005 Table of contents What are spatial data? What are the limitations of GIS for decision support? What is On-Line Analytical Processing (OLAP)? Why are today?s OLAP tools not efficient to exploit spatial data for decision support? Coupling GIS and OLAP capabilities unlocks your spatial data in ways never achieved before. An integrated solution called Spatial OLAP (SOLAP) How does SOLAP help organizations to analyze spatial data in their decision support processes? JMap Spatial OLAPTM KHEOPS Technologies JMap Spatial OLAPTM 3 Copyright KHEOPS Technologies 2005 What are spatial data? ?About 80 percent of all data stored in corporate databases are spatial data' (Franklin, 1992) Data are a key element of decision-making. They are the raw material to produce the information that leads to knowledge. However, these data are not used to their full potential because a hidden part of their richness, that is their spatial component, is often unexploited. Hidden in most data is a geographical component that can be tied to a place: an address, postal code, global positioning system location, region or country? It has been estimated that about 80 percent of all data stored in corporate databases are spatial data (Franklin, 1992) for which the spatial reference may be described in terms of position, shape, orientation and size. Displaying a phenomenon using its spatial component helps the user to have a better understanding of the phenomenon by visualizing its location on the territory, its extent and its distribution (concentrated, scattered, by groups, unpredictable, regular, etc.). It may also allow for the discovery of new information related to the phenomena. The use of the spatial component of data also allows to meet another need, that is to discover spatial relationships between various geographic phenomena (e.g. spatial correlation between the frequency of a disease X and a rate of emission of a pollutant Y). Cartographic data visualization facilitates the extraction of insights from the complexity of the spatio- temporal phenomena and processes being analyzed. Maps offer a better understanding of the structures and relationships contained within the dataset. In the context of data exploration, maps and graphics do more than just make the data visible; they are active instruments to support the end-users thinking process. The use of maps as data exploration medium allows the user to have a model that is closer to his reality and that requires less abstraction efforts, which in turn, increases his efficiency. Geographic Information Systems (GIS) are built especially for the gathering, storage, manipulation and display of spatial data. The first GIS applications appeared at the beginning of the 1960s while their commercial success started in the early 1980s. Well known products include ArcGIS (ESRI), GeoMedia (Intergraph) and MapInfo. With the emergence of Internet and international standards, new companies have also developed innovative products, such as KHEOPS?s JMap® product line, to help users to access and analyze their spatial data on the Web. KHEOPS Technologies JMap Spatial OLAPTM 4 Copyright KHEOPS Technologies 2005 What are the limitations of GIS for decision support? 'GIS alone cannot fill the "analysis gap"' 'High levels of user-friendliness and fluid interactivity are required to navigate in the database "at the speed of thought" but they are not supported by GIS' GIS has been used for certain decision-support applications for over a decade, typically to support decisions that are highly structured, repeatable and which are amenable to programming code. However, these applications do not address high-level decisions properly, those decisions that need relevant, summarized information at the fingertip. Those decisions that require fast information about trends analysis, about correlations between several phenomena, and about their spatio-temporal distribution need to be answered by a different kind of product. GIS alone cannot fill this ?analysis gap?. They cannot efficiently support the proper use of experience, knowledge, gut feelings and intuition, all of which are important in decision-making. First, in a decision-making process, users must focus on the results of the analyses, not on the analysis process itself (i.e. focus on ?what to obtain? rather than on ?how to obtain it?). Second, to efficiently support decision-making processes, navigation in the data must be done the same way humans think and analyze. This process must not be interrupted by complex manipulations, cryptic file names and slow response times. Spatial queries, cartographic exploration and map displays must support various levels of details, must use appropriate graphic symbology and must satisfy Newell?s cognitive band by providing results in less than 10 seconds (Newell 1990). These high levels of user- friendliness and fluid interactivity are required to navigate in the database ?at the speed of thought? but are not supported by today?s GIS. Thus, in spite of many interesting spatial analysis capabilities, actual GIS per se, with their transactional architecture, are not adequate for many decision support applications. Like Database Management Systems (DBMS), their query interfaces are too complex and they require query languages such as SQL (Structured Query Language). Their transactional architecture provides slow response times to aggregated, summarized queries, as they are not designed for this type of query. In order to better support decision-making, a tool based on a multidimensional database structure is required. It is rapid and easy because the results of major computations and cross-referencing of data are explicitly stored and the data are presented in a natural, hierarchical view. Such tools already exist in the Business Intelligence (BI) domain and are defined as On-Line Analytical Processing (OLAP) tools. KHEOPS Technologies JMap Spatial OLAPTM 5 Copyright KHEOPS Technologies 2005 What is On-Line Analytical Processing (OLAP)? 'OLAP is geared towards decision support as it is designed from the start to be easy and rapid.' 'OLAP allows the user to maintain his train of thought, his attention not being distracted by slow response times and complex queries.' OLAP is a category of decision support tools often used to provide access, in an efficient and intuitive manner, to a data warehouse or datamarts. Some of the best known commercial products include Microsoft SQL Server Analysis Services, Cognos Powerplay, MicroStrategy, Business Objects and Oracle OLAP, to name a few. Given that OLAP exploits multidimensional database structures, which reflect the user?s cognitive data model, the deduction process is less fastidious and the analysis is conducted more easily. The OLAP approach supports the iterative nature of the analysis process as it allows the user to explore and navigate among the different dimensions (i.e. analysis themes) at different levels of detail. This gives access to all the possible views or combinations of the data, facilitating the emergence of new hypotheses and encouraging knowledge discovery. Such easiness and rapidity are two essential conditions for analysts (decision-makers) to maintain their train of thought when exploring or validating hypotheses. First, the analyst interacts directly with the data and focuses on the results of the analysis rather than on the procedure required by the tool to perform the analysis. Second, it is rapid because data are pre-aggregated. Computation time is then reduced and very fast answers to complex queries are possible. This contributes to maintain the users? train of thought, their attention not being distracted by slow response times. OLAP is one of the cornerstones of BI. The common OLAP architecture is usually composed of three elements: 1) the database that supports the multidimensional data structure; 2) the OLAP server that manages the database and the aggregation calculations; and 3) the OLAP client that allows end-users to explore and analyze the data using different visualization methods and adapted operators. Figure 1. Common OLAP architecture. Database OLAP Client OLAP Server KHEOPS Technologies JMap Spatial OLAPTM 6 Copyright KHEOPS Technologies 2005 Why are today's OLAP tools not efficient to exploit spatial data for decision support? 'Without a cartographic display and spatial operators, OLAP tools lack an essential feature' Tools from the BI domain, like OLAP, usually exploit the aggregated levels of the data. In addition, as opposed to GIS, they fully support temporal data since time is a key element for decision-support. But, today?s OLAP tools, even though they are well-suited for decision support, are not adapted for spatial data as they do not fully exploit the geometric characteristics of cartographic objects. In fact, spatial information is treated as any other descriptive dimension and relies solely on location names for their spatial analyses (e.g. names of countries, states, regions, cities). Accordingly, OLAP tools present serious limitations to support spatio-temporal analyses (no spatial visualization, practically no spatial analysis, no map-based exploration of data, etc.). Data visualization facilitates the extraction of insights from the complexity of data by offering a better understanding of the structures and relationships contained within a dataset. This particularly prevails for phenomena distributed over space, thus for spatial data. Without a cartographic display and spatial operators, OLAP tools lack an essential feature. Existing tools like DBMS, GIS and OLAP can exploit a different type of data (geometric vs. non-geometric, aggregated vs. detailed), but insofar, no technology existed to optimally fulfill the aggregated-geometric needs required for spatial decision support. Figure 2. Position of the different technologies with regards to the nature and the aggregation level of the data. KHEOPS Technologies JMap Spatial OLAPTM 7 Copyright KHEOPS Technologies 2005 Coupling GIS and OLAP capabilities unlocks your spatial data in ways never achieved before 'The coupling of OLAP and GIS functionalities paves the way for the emergence of a new category of decision support applications that are better adapted for the exploration and analysis of spatio-temporal data. Such applications are termed Spatial OLAP or SOLAP' New types of applications can be developed to better fulfill the needs of spatial decision-support. Among the possible solutions, the coupling of spatial and non-spatial technologies, GIS and OLAP for instance, has shown top results (Bedard 2005). Besides developing everything from scratch, there are three ways of coupling GIS and OLAP: (1) OLAP-centric solutions, (2) GIS-centric solutions and (3) fully integrated solutions which offer as much OLAP functions as GIS functions (LGS Group 2000). OLAP features can be implemented either using the multidimensional database supplied with an OLAP server or a relational (or object-relational) DBMS using star, snowflake or constellation schemas. The advantages in using an OLAP server for the descriptive part include the access to the aggregation features and the optimized access to the data, which increases the speed of analysis when dealing with very large volumes of data. When dealing with smaller data volumes, the simulation of an OLAP server by a multidimensional structure (e.g. star schema) can turn out to be very advantageous, because the calculations of aggregations can be made in a selective way and be controlled using SQL queries. The cartographic features may be supported by a GIS or a visualization tool. The OLAP-centric solutions exploit all the capabilities of an OLAP server and only a subset of the functionalities offered by a GIS (e.g. map display). The GIS-centric solutions offer all GIS functionalities but only a subset of OLAP functions (and in most cases, they use a relational database with a multidimensional structure such as a star schema, instead of using an OLAP server). By contrast, the integrated solutions offer most functionalities of OLAP and GIS technologies built in a specific integrated environment, typically for a unique application. The OLAP functions (spatial, temporal and descriptive) are available, as well as the spatial and temporal analyses and the synchronization between maps, tables and graphical displays. This new technology can be considered as ?spatial data-centric? since spatial reference is constantly used during data exploration and analysis, as easily as the non-spatial dimensions. KHEOPS Technologies JMap Spatial OLAPTM 8 Copyright KHEOPS Technologies 2005 'This new technology can be considered as spatial data- centric since spatial reference is constantly used during data exploration and analysis, as easily as the non-spatial dimensions' 'The vision of an integrated solution comes true' The complexity and hard work needed to develop integrated applications paved the way for the emergence of a new category of software better adapted for spatio-temporal exploration and analysis of data. In a visionary presentation at the Canadian Institute of Geomatics? Montreal Congress Geomatics VI in 1997, Bedard termed this category Spatial OLAP technology (or SOLAP). Eight years later, after important research projects and numerous experimentations combining GIS and OLAP, his vision of an integrated solution comes true. The advantages of using SOLAP technology are various for the exploration, manipulation and update of the data. As presented in Figure 3, the missing block has been filled. Figure 3. Position of SOLAP with regards to the nature and the aggregation level of the data. Commercial solutions coupling BI and GIS recently appeared on the market. These systems, some OLAP-centric, some GIS-centric, some offering both possibilities without full integration, offer only a subset of the desirable functionalities of a Spatial OLAP. They offer solutions that may satisfy users requiring only better interoperability between both technologies, but they likely do not satisfy users looking for a fully integrated environment. Moreover, most of them offer only a static cartographic visualization of OLAP query results. Some tools require to store each possible map view individually on the server, thus affecting the update effectiveness of spatial data. These solutions present many limitations with regards to interactive data manipulation and exploration through cartographic views. Even when they offer minimal spatial functionality, cartographic exploration is not as easy and rapid as it is with tabular and chart views, which is fundamental to the SOLAP concept. KHEOPS Technologies JMap Spatial OLAPTM 9 Copyright KHEOPS Technologies 2005 An integrated solution called SOLAP ?All required know- how to produce maps is embedded in SOLAP techno- logy. End-users don't need to know a GIS query language, GIS functions, the database structure and graphic symbology rules.? Spatial OLAP Technology was designed to sit on top of multi-scale spatial databases or warehouses, to enrich data exploration concepts based on an explicit spatial reference and to support the multidimensional paradigm. SOLAP can be defined as a visual platform built especially to support rapid and easy spatio-temporal analysis and exploration of data following a multidimensional approach comprised of aggregation levels available in cartographic displays as well as in tabular and diagram displays (Bedard 2004). The Spatial OLAP technology supports: Multidimensional data structure used in Business Intelligence, which gives a huge advantage over existing web-mapping applications; even though some support drill operations, they are based on the transactional structure; Cartographic and non cartographic displays created dynamically with the multidimensional data. This dynamic aspect allows the user to create hundreds of thousands displays (maps, tables and diagrams) using the dataset without having to store each display individually. Spatial dimensions where members are associated to geometric shapes spatially referenced on a map to allow their visualization, querying and drilling in a cartographical manner. These spatial dimensions are used to interactively explore data into cartographic displays as well as in tables and charts. It is much more than a static map displaying the results of a query, it is interactive! Cartographic exploration into the map objects and into the map symbols using different spatial drill types. Click in a way that is similar to hyperlinks on web pages. Graphical symbology rules (e.g. color, border, and shading) used in all the displays (e.g. tables, maps, and graphics) to produce state-of- the-art views on the data. These rules avoid potential collisions since, theoretically, the same rules do not always apply to maps, pie charts, bar charts, tables, etc. Consistency of graphical symbology between display types allows for easier highlighting of relevant information. Synchronization of interactive exploration between the displays to facilitate the identification and the interpretation of the data. Drill in the map, the corresponding tables and charts are also drilled! This kind of synchronization may only be offered by a fully integrated tool that manages both components (OLAP/GIS). KHEOPS Technologies JMap Spatial OLAPTM 10 Copyright KHEOPS Technologies 2005 How does SOLAP help organizations to analyze their spatial data? Quebec Ministry of Transportation Road Network Management Application 'One hour of training is enough' Laval University Students? Recruitment Application A Road Network Management application helps the Quebec Ministry of Transportation to find the effects of variations in the annual average daily traffic on the average road conditions according to road functional classes and other parameters, determining if the average road conditions reach the intervention thresholds, or calculating the intervention costs according to the intervention thresholds, the functional classes and the type of physical environment. They take minutes instead of days to obtain the information, without a GIS specialist. Using a Spatial OLAP application in Environmental Health, the Quebec National Institute of Public Health typical users take 2 hours to meet their spatial analysis needs (e.g. to explore possible relations between respiratory diseases and air quality) as compared to one day of complex manipulations and calculations required by a GIS specialist also familiar with the statistical software used. Experiments with various end-users have shown that one hour of training is enough to use Spatial OLAP technology powerfully, to freely explore spatial data and to produce an infinity of meaningful maps, even for those who have never touched a GIS! For the users of Laval University Students' registration application, SOLAP offers more flexibility than pre-programmed reports from their well-known web-reporting system. This Spatial OLAP application offers a potential of 20 160* cross-tables that users can display as maps or diagrams, in a few seconds, with a few mouse clicks. * This number corresponds to the number of aggregations available in their multidimensional data. SOLAP allows non-GIS users to become efficient very quickly. Unlike GIS users, SOLAP users do not need to know a GIS query language, GIS functions (statistical, map display), the mysteries of a database structure and the complex graphic symbology rules to produce a map. All required know-how is embedded in the Spatial OLAP technology. KHEOPS Technologies JMap Spatial OLAPTM 11 Copyright KHEOPS Technologies 2005 JMap Spatial OLAPTM ?JMap Spatial OLAPTM allows for rapid and easy exploration of spatial databases and offers many levels of information granularity, many themes, many epochs and many display modes synchronized or not: maps, tables and diagrams.? Sometimes qualified as ?keyboardless-GIS?, but being much more than that in terms of data exploration within multidimensional datasets, JMap Spatial OLAPTM is the first integrated web-enabled technology that answers these spatial decision support needs. Resulting from years of R&D at Laval University under the direction of Dr Yvan Bedard, it offers an intuitive user interface allowing non-technical users to easily access, visualize and analyze their spatial data. Accordingly, the non-expert users can perceive SOLAP as a new type of user interface for multi-scale GIS applications and web mapping, although it is much more. JMap Spatial OLAPTM: Aims at supporting the way humans think and analyze data Is used without having to know any query language; mouse clicks are used to interact with the data or the legend Allows the users to focus on the results of data exploration rather than the analysis process itself (i.e. focus on ?what to obtain?, not on ?how to obtain it?) Provides practically instantaneous response times The displays may include several thematic maps, statistical diagrams (bar charts, pie charts, etc.) and tables affected by a symbology defined by values or member classifications. It has never been as easy and rapid to build and compare a series of yearly maps! Using SOLAP operators, users can easily explore, with a few mouse clicks: Different levels of details (e.g. local -> regional -> provincial) Different themes (e.g. asthma -> greenhouse gas emissions) Different epochs (e.g. 2004 -> 2005, and 2005 -> 06/2005) Different analysis elements (e.g. count -> average -> sum) It is possible for the end-user to define calculated measures to produce new values with formulas based on existing values in the data cube. The following figure shows a spatial drill-down operation on Quebec Province Eastern Region, followed by another drill-down on its Northern district, to show more local details, always for the same variables and KHEOPS Technologies JMap Spatial OLAPTM 12 Copyright KHEOPS Technologies 2005 ?Experiments with various end-users have shown that one hour of training is enough to use Spatial OLAP technology powerfully.? ?Typical users needed about 2 hours to meet their spatial analysis needs as compared to 1 day required by an experienced GIS user? using the same graphical symbology rules based on the variables under study. The various displays can be synchronized as shown in figure 5. Figure 4. Spatial drill-down operations. Figure 5. JMap Spatial OLAPTM Interface with the dimension tree (left panel) and different synchronized display types (map, pie chart and table). 1 mouse click 1 second KHEOPS Technologies JMap Spatial OLAPTM 13 Copyright KHEOPS Technologies 2005 Display of yearly maps to visualize spatial trends; 3 clicks, 5 seconds ?JMap Spatial OLAPTM Administration tool supports the user in the design of the application.? Defining schema type Defining dimension type JMap Spatial OLAPTM is delivered with a web-enabled administration tool designed to assist the database administrator of the application to: Define dimensions (descriptive, spatial and temporal). Map dimensions to a cube. Define measures in a cube. Define calculated measures (as well as within the client tool). Define the classification to be used by the different displays. Add new cartographic layers. Customize the symbology of the different displays. Link the data with their metadata for display purposes. Figure 6. The JMap Spatial OLAPTM administration tool helps designing the application. KHEOPS Technologies JMap Spatial OLAPTM 14 Copyright KHEOPS Technologies 2005 Sales and Consulting Offices KHEOPS Technologies 300, Saint-Sacrement, suite 114 Montreal, Quebec, Canada H2Y 1X4 Tel: (514) 285-1211 Fax: (514) 285-1177 firstname.lastname@example.org http://www.kheops-tech.com Bibliography and further readings KHEOPS and Laval University thank their R&D collaborators, and in particular the Quebec Ministry of Transportation and the GEOIDE network of centres of excellence. JMap® is a registered trademark of KHEOPS Technologies. All other trademarks are the property of their respective owners. Bedard, Y., 1997. Spatial OLAP, Annual Forum or R&D, Geomatics VI, Canadian Institute of Geomatics, Nov. 13-14, Montreal, Canada. Bedard, Y., 2004. Improvement of decision-support capabilities of GIS by using a SOLAP (Spatial On-Line Analytical Processing) module. École Polytechnique Universitaire de Marseille, April 8, France. Bedard, Y., 2005. Integrating GIS and OLAP: a New Way to Unlock Geospatial Data for Decision-making. Directions on Location Technology and Business Intelligence, May 2-4 Philadelphia, USA. Bedard, Y., P. Gosselin, S. Rivest, M.J. Proulx, M. Nadeau, G. Lebel & M.F. Gagnon, 2002. Integrating GIS Components with Knowledge Discovery Technology for Environmental Health Decision Support, International Journal of Medical Informatics,Vol. 70, No. 1, p. 79- 94 Franklin, C., 1992. An Introduction to Geographic Information Systems: Linking Maps to Databases. Database, April, pp. 13-21. LGS Group Inc., 2000. Analysis of Health Surveillance Business Intelligence Tools and Applications, Final Draft, 111 p. Newell, A., 1990. Unified theories of cognition, Harvard University Press, Cambridge, Mass, 549 p. Rivest, S., Y. Bédard & P. Marchand, 2001. Towards better support for spatial decision- making: defining the characteristics of Spatial On-Line Analytical Processing. Geomatica, Vol. 55, No. 4, p. 539-555.