Mapping the Fourth Dimension

Our 3-dimensional world is the usual subject of 2-dimensional maps; their purpose is to accurately depict geographic extent on a readable/publishable surface, with or without thematic content.  The purpose of adding the 4th dimension of time is somewhat intuitive- maps of the same area through time can give a viewer an understanding of change in some feature.  The problem in depicting the 4th, temporal, dimension on a 2-dimensional surface is remedied by the use of various spatiotemporal visualization techniques, the most effective of which, it could be argued, is the map animation.  The simplest form of this variety of cartographic visualization is the display of two or more maps, adjacent to one another, showing the same geographic area at different times.  The change in the area depicted is evident in the difference(s) between the various maps.  While this method is effective, actual animation on a map is an even more salient method of displaying areal change through time.  

       
The above graphics represent two years- 1790 and 2000- and contain proportional symbols indicating the most populous cities in the U.S. at those times.  The two maps are actually the beginning and ending frames of an animation, which shows the same map for each 10 year period between 1790 and 2000, and thus gives the viewer an idea of where the major centers of population in the U.S. were for each decade.  The ease of doing so with a simple animation, cycling through each decade, is fairly large in comparison to creating and displaying something singular including separate maps, one for each decade.        

Bivariate Maps of the Choropleth Persuasion

The depiction of multiple variables on the same map, displayed to highlight a relationship between the two, is no simple feat of cartography.  The choropleth map, in particular, is best suited for indicating the geographic prevalence of a single, or single incidence of multiple, phenomena.  Mapping with varying color shades to indicate the prevalence of two separate variables- the areas where there is a high prevalence of one and not the other, and areas where there is a high prevalence of both- is an ambitious task by any estimation.  

The use of the bivariate choropleth map may be employed for this depiction of two separate variables, and thus indicates the relationship between the two.  Above is a representation of both rates of obesity and physical inactivity, by U.S. county, with the use of two different hues with separate quantitative color progressions.  The legend in the lower left illustrates these with a 9-square color palette, which varies by hue according to the X- and Y-axes, and by lightness progressing across both.  As the colors progress towards the upper right corner of the cube, the colors are meant to indicate a high prevalence of both variables.  Thus, the viewer is able to use the 9 different colors there to visualize the spatial relationship between the two.  The current version of ArcMap in use, though, doesn't support the automated creation of this variety of legend, and it must be assembled by hand.  Though creating a 9-color legend, which varies in hue and lightness along two axes, is no easy feat, it is unarguably a valuable practical lesson in applied color theory.   

Infographics: a Method for Cartography Display and Presentation

Making maps, to present and display to others, is all well and good, but what of the context for the map's thematic content?  Presenting information to an audience with a comprehensive understanding of the map's content, perhaps in the form of a presentation, can render the map itself a supporting visual aid.  It is when the map stands alone, and is meant to be viewed and interpreted as a stand-alone visual, that the "infographic" can be an indispensable tool.  Infographics add information that can be integral to understanding a thematic map's purpose, and can lend support to the map as an educational or explanatory visual.  Though this information is, by necessity, often simplified and graphically embellished, it's a valuable tool for presentation nonetheless.  

Above is a visual intended to convey the seriousness of the possible link between increasing rates of drug poisoning mortality and rates of uninsured adults.  This is, by no means, extensive or scholarly research, but is rather intended to raise a question of whether this is an issue that may require more investigation.  The maps, on the left, give an abridged picture of what the geographic distribution of the two factors- rates of drug poisoning mortality and percent of uninsured adults- currently looks like. The darker colored counties that are displayed as such in both maps are locations where prevalence of both factors is increased, which, as the other supporting graphics indicate, may be cause for some concern.  The two charts on the right, a scatterplot showing the general relationship between the two variables, and a bar chart of where the five states with the highest percent of uninsured adults fall in relation to drug poisoning mortality average, summarize some of the quantifiable relationships immediately evident in the data.  The graphics in the center give a few statistics that may support the contention that the relationship between the two variables is serious enough to warrant further investigation.  The infographic, as a whole, sometimes tells more of a story than a map, displayed on its own, is capable of. 

Terrain Visualization- 3D Information in a 2D Map

Maps which include 3-dimensional information are ubiquitous, and not just in the form of those that employ stereoscopic effects for the wearer of the right special glasses- 3 dimensions are regularly portrayed on a 2-dimensional surface with the use of contour lines, hillshading, and hypsometric tinting.  The purpose of these techniques is to allow the viewer to more accurately visualize the varied elevations and land formations within the mapped area.

Hillshading, in particular, can provide a great deal of detailed visual information, which is readily apparent to even a map user unfamiliar with the finer points of deciphering contour lines.  Above are two hillshaded images of the same overhead view- the left image is created with shading that imitates a light source located in approximately the sun's position for this area on the 15th of January at 8 am.  The right uses the shading created by a light source in a particular position and elevation, one which is a default within ArcGIS.  Thought these two images use the same color scheme- that is, the same colors in both for high and low elevation values- it's apparent that the visual images given by each are quite disparate.  It could be contended that the image on the right, with the default light source position, gives a more readily interpreted picture.  The depression in the terrain made by the canyon is immediately evident, outlined with the darker red/brown shadows in the mostly yellow terrain.  The left image could easily be misinterpreted to be depicting a series of hills, in yellow, rising out of the terrain shaded in darker red/brown.  Clearly, the light source's position is an important consideration in the creation of a hillshaded image- and is not always best-placed in the "accurate to earth" location.  

Thematic Mapping's Finest: the Choropleth Map

No cartography education or body of knowledge is complete without the (seemingly) ubiquitous choropleth map- that which portrays a derived quantitative measure by some administrative unit.  The map above displays the percent change in population for Georgia's counties, which includes both population gains, with positive percentage values, and population losses, which are shown with negative percentage values.  The classification scheme used to group the counties according to their percent increase and decrease is the Natural Breaks method, slightly modified.  This method gives a decent visual of where the county gains and losses were, without obscuring any patterns or trends, by maximizing the similarity of values within classes, and maximizing the differences between the classes.  The colors chosen for each class use a diverging color ramp, where the lightness and saturation increase with the subsequent relative size of the population increase and decrease, which uses two different hues, in order to distinguish easily between gains and losses.  Thus, the class directly in the middle represents counties with very little to no population change, and is the least saturated and lightest shade on the map.  The legend itself gives indication of the value ranges for each class, with the average percent change at the bottom.  The labels for "increase" and "decrease" are added with vertical orientation to the side of the classes, for an easy visual of the direction of the scale.  The processes of normalizing derived data, evaluating thematic and data content, choosing effective colors and scale, and a plethora of other small decisions are integral to the construction of a functional and aesthetically pleasing choropleth map.  These skills, in particular, form the foundation of an effective cartography skill set.     

Map Symbols & Legends

The symbols and legend included on a finished map are details carefully chosen by the map's designer- they convey the primary meaning of the map, and, if they are unclear in any manner, can arguably obscure the map's intended purpose.  In the case of symbols rendered proportional to the relative size or magnitude of the thematic feature, it is imperative to choose colors and shapes that maximize the viewer's understanding of the phenomena being displayed on the map.

The above thematic map intends to convey the net increase and decrease in jobs for each state in the U.S. for the period between 2007 and 2015.  The circle sizes are proportionate to the magnitude of net loss/gain, and the color indicates whether the number of jobs represents an increase or decrease.  The legend, in the lower left corner, uses hollow circles to represent the relative numbers, as it applies to both the increase and decrease amounts, and indicates the colors red and green represent the net loss and gain.  The increase and decrease amount circles on the map are two feature classes, created by querying and exporting both sets of values from an original layer that contained both positive and negative values in its attribute table.  This was accomplished by adding a field to the feature class containing negative values, and calculating the absolute values of those numbers, as the layer symbology, as a display of proportional symbols, does not support the use of negative values.  Thus, each layer- one representing net increase, and the other net decrease- is symbolized separately, but with the same symbol range, which allows each set of values, both negative and positive, to be symbolized in differently colored circles.    

The Understated Importance of the Cartographic Element of Color

Color, inasmuch as it is used as a cartographic tool, is probably taken for granted by most map viewers- another map element unremarkable the vast majority of the time, except when it is used poorly, and then becomes unmissable.  The color palette chosen by the map maker isn't purely an aesthetic decision either, it is carefully considered, so that the feature being displayed appears in a logical context.  A map with purple water features and blue forest tracts wouldn't be considered seriously by most, save as a piece of art.  Color is also of utmost importance within the context of the choropleth map, which displays some feature, grouped into classed categories, based upon some measurement.  Color variation for different classes is often seen as a change in saturation or lightness, within the same general hue.

Above is a linear color ramp- the darkest and lightest shades vary by a certain amount for the R, G and B values, and the classes/shades between are a linear progression of those values, yet the color remains a constant hue.  

The light/dark progression of one hue is, however, augmented, as in the ramp above, by adjusting the amount of the interval between the R, G and B values between the darkest colors.  The object here, were these colors being used in a choropleth map, would be to retain the same hue for all of the classes, while appreciably varying the lightness and saturation, so that whatever is being symbolized on the map is shaded in colors different enough to tell one apart from the other, based on the value of the hue.

Finally, we have a color ramp produced by Colorbrewer.org, which is an exceedingly useful tool for generating color ramps such as these.  The above ramp uses a slightly different hue for the top, darkest class, and then varies the lightness of the pinker hue in the classes below.  The use of this type of ramp would be invaluable in a map with which one might want to display different areas in different shades, to indicate the amount of some measurable thing- the colors are all in approximately the same hue, and yet vary enough from one another such that one area would be visually distinct from another.

Typography & Making Map Labels (part II)

Of the many small details essential for a well-made map, labels rank among the most likely to be overlooked.  They are a small element that, when done well, warrant no particular notice, but, when executed poorly, have real potential to destroy a map's aesthetic beauty and practical use.  A working knowledge of typography basics is an integral skill in creating functional and visually appealing map labels.

Mapping the City of San Francisco presents a perfect opportunity for practicing skills and various techniques in effective map labeling, and the above map is my submission.  What would be considered a "good" label has a number of characteristics, among them its legibility, and the degree to which it contributes to a harmonious and visually pleasing display.  Thus, there is a balance to be struck between making the label easily readable, but not so ostentatious that it overpowers other, and possibly more important, details within the map.  Another potential challenge is keeping each label, in accordance with general map-making convention, within the boundaries of, or adequately adjacent to, whatever it is intended to be labeling.  Treasure Island in the map above, for example, isn't wide enough to accommodate its label at the appropriate text size, and thus the decision to change its orientation.  Small choices such as these, and a plethora of various others, lead to useful labels and a functional map.     

Coordinate Systems & Projections

Map projections and coordinate systems are at the heart of a map's utility as location and navigational aid.  The principles involved in selecting and/or analyzing a projection or coordinate system for a map are intuitive in nature, but the underlying concepts can become a bit complicated.

This map of Michigan is projected using the Hotine Oblique Mercator Azimuthal projection, which is an intimidatingly lengthy name, but means that this is a cylindrical, oblique version of the Mercator projection.  As such, it is conformal, and displays the state as accurately shaped.  This particular projection is suitable for Michigan because it doesn't divide the state into multiple zones, like a State Plane or UTM grid would, as the state encompasses multiple zones of both.  Also, as this is intended to be used as a general reference map, the projection is well suited because of its conformal nature, which preserves the distinctive mitten shape of the lower peninsula.    

Revisiting the Basics of Cartography & Map Design Principles

Thus begins a new semester, and a welcomed review of cartography basics.  Having learned to make maps the right way in the past year, it is with particular satisfaction that I revisit the novel concepts of good map design once again this week.  Specifically, there are five map design principles accounted for, including legibility, visual contrast, figure-ground relationships, hierarchical organization, and balance.  

The above, ideally, represents all five of the principles of map design.  Legibility is preserved by the simple display font, and the sizes of the various text.  The text is of a size and color that allows for maximum visual contrast of the graphics.  The symbols for the various mapped features are large enough to be seen, but one doesn't obscure all of the others.  Visual contrast is also maximized with the less saturated color hues used for the background of Travis County, and the surrounding area.  The inset map colors are less saturated as well, to preserve their lower position on the visual hierarchy, but still have enough contrast to visually differentiate them from the other map elements and the background.

Figure-ground relationship is represented with the darker color background for Travis County, the map's focus, and the lighter shade of the surrounding area.  This allows the focus county to stand out, as it is highest on the visual hierarchy among the map's graphics.  Further, the most important map symbols, the golf courses and rec. centers, are the most saturated and brightest hues, bringing them forward based on their relative importance to the map's theme.  Hierarchical order is also seen with the relative size of the various elements- the title and subtitle text and the legend text are the largest, as they are meant to be among the most visible elements.  The scale bar, north arrow and inset map are smaller, as they represent ancillary map elements, and the citations at the bottom are the smallest, as they are least central to the map's theme.

Balance is preserved with the position of the various graphic objects on the page.  The mapped area itself is centered, with the title and subtitle text centered above it.  The legend and inset map are positioned on the bottom, left and right justified, with the scale bar and north arrow centered below and between.  Finally, the citation text for the map author and data sources are positioned along the bottom margin, and are justified to the right and left sides of the page as well, which hopefully balances the various graphics and whitespace.  The objective here is to create a harmonic balance on the page, so that nothing looks crowded or out of place, which, when properly executed, creates a finished and professional-looking map. 

  

Sharing Custom Tools in an ArcGIS Environment

When a custom script tool is created one measure of its intrinsic value may be its potential as something that can be shared with others.  Unsurprisingly, ArcGIS provides an easy method of doing so- with a custom script tool, created in ArcMap.  All this requires is a custom toolbox and a python script, and from there just a few modifications to the input variables and the script can be shared with others, and run from ArcMap.  

The above is a dialog box, which is for user input parameters, and must be populated before the script is run.  The parameters are set in the "Properties" of the script tool, and can be specified to a plethora of different requirements.  The text in the right pane of the dialog is customizable in the "Item Description" of the script tool, and can be modified to include specific information and instructions for not only the tool itself, but also the user input fields.  

  The output of the script is the above- as layer of random points with buffers of a specified distance.  The script can also be embedded in the actual script tool as well, so that a separate .py file isn't required if it's shared with someone else.

Custom Tools within ArcMap

ArcGIS, being the powerful program that it is, provides ample customization abilities, particularly with the creation of custom tools by way of Python scripts.  Tools are accessed by a user within the GIS Desktop interface via dialog boxes, which receive the user's input file locations, desired output location, and values for whatever parameters may be specified.  One of the handiest things about them, though, is the average user's potential to create a custom tool with a Python script, and easily format it to run within the ArcMap interface- potentially by GIS users of any and all skill levels.

A custom dialog window, such as this, looks fairly identical to the system provided tools within ArcGIS, but the process behind the user interface is a custom Python script.  One only need create a new toolbox in ArcMap, and go through the steps within the Add Script Wizard to specify the script's location, and the parameters required to run the tool.  After a few minor edits within the script itself- mainly replacing lines specifying file paths with GetParameter functions- the script can be run with any valid user-specified parameters.  

  
The Results window, which opens upon the script's execution and displays messages about the success or failure of the script, can be customized from within the source-script as well.  Print statements, used in the Python environment to print messages to the Interactive Window, are changed to AddMessage functions, which add output information to the Results window.  The above are the output messages from the custom script, modified to become a custom tool within ArcMap.  The messages specified to display upon successful completion are displayed here, and are replaced with the appropriate red-lettered warnings and notifications if there are any errors or exceptions encountered instead.    

Analyzing Statistical "Hotspots" with GIS

Using GIS to identify and locate “hotspots,” or areas with statistically significant amounts of various phenomena, is both intuitive and useful, given its powerful ability to both analyze and visually display this kind of information.  Martino, et al. (2014) use GIS to this end to explore locational patterns and statistical algorithms employed in the analysis of brain cancer in “Spatio-temporalhotspots and application on a disease analysis case via GIS.”  With this research the authors sought to identify locations with statistically significant proportions of brain cancer incidence in New Mexico, with further consideration of temporal factors, for the years 1973 to 1991.  The data used for the research was a collection of 5,000 point locations of cancer incidence for the indicated time period, with the enumeration units as the counties that compose the state of New Mexico.  The authors’ examination focused on a comparison of two methods of cluster analysis- the Extended Fuzzy C Means (EFCM) and the Extended Gustafson-Kessel (EGK) algorithms- employed within the GIS environment.  The EFCM algorithm they used employs a circular shaped hotspot area, determined by the patterns of point event clusters across space, and makes a temporal comparison for a specified period of time.  The EGK method is an analogous procedure which produced very similar results, but employs an ellipsoid shaped hotspot area, and uses a different variation of membership calculation within one if its formulas.  Analysis of the results of this research lead the authors to conclude that both algorithms, used within a GIS, are equally effective in predicting the spatial area of the diffusion of this disease, and may be of use in larger-scale studies seeking to identify temporal and/or spatial determinants within a specified study area.        

Reference

Martino, F., Sessa, S., Barillari, U. S., & Barillari, M. R. (2014). Spatio-temporal hotspots and application on a disease analysis case via GIS. Soft Computing - A Fusion Of Foundations, Methodologies And Applications, (12), 2377.

http://dl.acm.org/citation.cfm?id=2689536

Working with Rasters in Python

The corollary to working with feature geometry in vector datasets could be described as the use of the Spatial Analyst extension to access the arcpy.sa module, which allows one to work with raster datasets in Python.  This module comes complete with its own tools, functions and idiosyncrasies, which were the focus of this week's Python script.

     
Reclassification of raster data can be a cumbersome process using the Spatial Analyst tools available in ArcMap, particularly when the required result is a classification raster involving the combination of several re-classified inputs.  This task is relatively simple by comparison within a Python script though, which takes each intermediary input as a temporary dataset, and saves only user specified results.  The Python script also completes with a few short lines of code what would otherwise be completed through the creation of multiple separate raster datasets with the tools available within the Spatial Analysis Toolbox.  The efficiency of the code's ability to reclassify, and then combine to create a final output of areas containing specified characteristics, various raster data cannot be overstated.  The above image represents an area with characteristics meeting all of a set of some specified conditions- namely that the land has a particular type of landcover, and a specified range of slope and aspect.  Two input raster datasets were employed in its creation- one with landcover types, and another with elevation.  The elevation data was used to create datasets with slope and aspect characteristic attributes, and the landcover was reclassified to contain only those areas categorized as "forested."  The resulting output is the combination of the suitable areas within each of the inputs, and represents the land within the specified areal extent that meets the landcover, slope and aspect suitability requirements.   

Geometries in Python

Geometry, within the context of GIS and Python programming, refers to the points, connecting lines and arrays that compose points, polyline and polygon features.  All of the vector data objects used within a GIS can be expressed in terms of these geometries, which can be easily (relatively speaking) accessed with Python script.

The task at hand for this topic was the creation of a script that output the above- the object ID, a sequential "vertex ID," the x, y coordinates and feature name- all taken from the attribute data of a polyline shapefile of a set of rivers in Hawaii.  The script has to iterate through each line segment and print the coordinates of all of the vertices, placing each pair on a separate line of the text file.  In this process we make use of the concept of the "search cursor" introduced last week, which allows the script to access each row (or record) within the feature class.  Also put into play is the for loop, which provides the means of iteration through each row (in this case each river), and further through each line segment and set of point vertices that compose each one.  The 25 different rivers within the feature class are represented as 25 different object IDs (the first number of the line), with each coordinate pair (vertex) given a sequential ID (the second number), the x, y pair, and finally the feature's name, taken from the name attribute field.  Reading and outputting data such as this from feature classes could be a very useful task for a script to complete, as the required steps, performed manually, would surely become very time consuming, especially if one were working with a large number of datasets.   

Practicing Damage Assessment: Structural Damage Caused by Hurricane Sandy

Assessing damage in the aftermath of a natural disaster is a weighty task- often there is significant loss of life and property involved, but someone has to actually quantify exactly how much and of what is damaged or gone all together.  This is easily accomplished using aerial photos and GIS analysis.

Above is an aerial image of a block on the New Jersey shore, taken shortly after the late October 2012 storm event of Hurricane Sandy.  This image was used in conjunction with one of the same areal extent, but taken before the storm.  Comparison of the two images allows for a quick, but not overly accurate, inventory of the structures directly affected, and some of the damage on the ground.  The final qualification of said damage is more than likely completed with reference to actual field-derived ground information, but image analysis such as this provides a decent estimate to start off with.  When the two aerial images, pre- and post- storm, are compared a point layer with locations of structures can be created, with descriptions of the types of damage observed as a part of the feature attributes.  Upon completion, the layer can be symbolized according to observed damage, and presented in map-form with an image like the one here.  This type of analysis provides maps that can give a good general overview of damages resulting from storm events such as Hurricane Sandy, and can also be used to analyze and quantify the extent of the damage.  

 Creating a feature layer of the actual coastline and drawing a buffer around it allows for a count of buildings, and their associated damage levels, within a specified distance of the water's edge.  The results of this analysis, in the table above, indicate that the majority of the damage- about 60%- occurred in the area 100 to 200 meters from the water.  Quantitative results within a table like this also support the intuitive assumption that the level of damage is greatest nearest to the water's edge, and the magnitude of structural damage generally decreases with increased distance.  It is worth mentioning, though, that despite the quantitative nature of this summary, the data it is based off of is anything but.  Subjective interpretation by one person of the amount of damage by way of visual comparison of aerial images would hardly suffice as an accurate assessment in a real-life scenario.  

Using Python to Manipulate Spatial Data (and also possibly to test your ability to refrain from gratuitous use of hyperbole in writing...)

Computer programming, writing script, everything involved with learning the "language of the machines"(so to speak) isn't supposed to be easy. No one will tell you it's easy.  Going into your first class on the subject is intimidating, to say the least, but there isn't really anything to decently prepare an average intellect for the exact nature of the challenge ahead.  The minor victories along the way are as sweet as any devoted student has experienced- spending hours experimenting with this and that to persuade your script to perform the way you require, and finally getting it right.  The rush of satisfaction is unparalleled, but the dark and frustrating hours toiling to get there are enough to render helpless even the most stalwart resolve.  This week and last week were dedicated to grasping the finer points of spatial data manipulation with Python. Some of the concepts in Python introduced were a bit cryptic for a novice, but extensive experimentation with them in creating the assigned script was sufficient to reveal their basic workings, if not some of their many vagaries.

We have with the above screenshot, once again, a section from the printed output of the script that was created for the assignment.  The simple output displayed above is, like the previous assignments for the semester, the mere tip of the iceberg that was the required script.  The required outputs, some of which are displayed in the above screen capture, varied from a simple geodatabase created in a specified location, to a search cursor function that queried a feature class' table to output specific attributes of records meeting certain criteria.  Suffice it to say, getting the right output from the myriad processes the script performed was an epic test of patience for your humble blog author, and fortunately (for all involved) did not result in a frustrated burst of destructive rage.  For example- the concept of a "search cursor" is perhaps not so intuitive to the nascent programmer, and so employing its functionality became a heuristic lesson, as reading about or listening to someone describe these kinds of things isn't always sufficient to create an understanding.  The employment of some SQL functionality in Python and the necessity of "field delimiters" is another case in point from this week's assignment.  The varied plethora of capabilities that the humble list can have within a Python script is also something probably not immediately evident, but when iterated through with a process from a for loop this unassuming structure can be used to complete many a useful task.  In the end, though, after all of the frustration of learning and acquiring various skills and knowledge, the objective has been met- the concepts introduced in this series of lessons are now fully grasped, and a working understanding of the material covered is the prize.  It strikes one as rendering the difficult process of getting there completely worthwhile.            
     

Sea Level Rise and Coastal Flooding

Weather events have the ability to impact large numbers of people in this country in very drastic ways.  This was made very evident with Hurricane Sandy's landfall along the eastern seaboard in 2012, and hopefully brought some attention to the fact that any increase in sea level, either on its own or in combination with a large weather event, can displace, harm and cause great economic impact to a large portion of the population.  Planning and analysis of the likelihood and nature of these kinds of potential catastrophes is essential.

Hawaii, because of its isolated location in the warm tropical waters of the south Pacific, is especially vulnerable to water-related catastrophes, such as tsunamis and typhoons.  On all of the Hawaiian Islands many of the schools and other public facilities that are located on the lower elevations nearest to the shore have tsunami drills, in order to prepare for these events.  The above map details some of the potential impact of a 6 foot sea level rise to the District of Honolulu, on the south eastern edge of the island of Oahu.  The elevation of the district is measured, and the extent of a certain rise in the water level can be anticipated by isolating those areas below a certain threshold.  The potential depth of the water at these lower shoreline locations can be calculated as well, and overlaid with various census measures to better describe the characteristics of the population directly affected.  The analysis completed in the creation of this map revealed that the total population within the 6 foot sea level rise flood zone is 60,005 persons, the majority of which are Caucasian, under the age of 65, and do not own the residence that they occupy.  These kinds of measures are important to account for in modeling the impacts of sea level rise, as awareness of especially vulnerable populations within the flood zone is necessary to anticipate measures needed to protect the affected population at large.  

Crime Analysis

Crime, to some degree, is a function of human society.  When humans live in groups we tend to declare certain actions as being "wrong," and structure punishments and consequences correspondingly.  What specific actions are considered criminal obviously varies across space and time, and the motivations underlying those actions do as well, but analysis of where criminal activity occurs is an interesting application of statistical principles in spatial analysis.  The where doesn't necessarily give us the why, but it does make for some compelling theories, and a good introduction to spatial statistics in GIS.  

  
There are several decent measures of statistical "hotspots" one can employ, and the three used above to look at burglaries in Albuquerque, NM are Local Moran's I, Kernel Density and Grid Overlay.  All of the measures attempt to categorize, to some degree, the areas in which crimes are most frequent, by way of examination of their mapped locations.  The tools available in GIS to complete this analysis range from (relatively) simple point-in-polygon counts involved with Grid Overlay to the Z-scores and p-values used in the Local Moran's I.  Although all three methods used the same source data, clearly they have arrived at different resulting crime "hotspot" areas.  The variations as such are a function of the varied techniques involved in calculating statistically significant areas of highest crime.  

Geoprocessing with Python

After covering a bit of geoprocessing last week, this week we delve into the heart of the matter- geoprocessing with Python.  Geoprocessing is at the heart of the ability of GIS to solve problems with models and analysis by way of powerful computing tools, and Python could be argued to be the GIS professional's key to a more advanced use of such.  Suffice it to say, though, that moving into this territory things begin to get a bit more complicated.

Our script for the week, a screen shot of the results of which is above, performs a basic series of geoprocessing tasks, executed entirely by way of Python code.  The first operation adds x-y coordinates to a shapefile with hospital locations.  The second process is a 1000 meter buffer around the hospital points, and the third dissolves the overlapping lines of the buffers, so they become a single feature.  Each task is completed via a Python command, and file names are entered as parameters, but in order to do so a "workspace" location must be set up first.  This allows the files being manipulated to be referred to by name, and not their full file path on disc, which makes the script a bit shorter and more compact.  After each operation is completed, a message is output- indicating that it was successful, and naming the operation that was performed.  This introduction to simple geoprocessing functions within Python script is the mere tip of the iceberg of Python's capability in this realm.

Spatial Accessibility & Network Analysis

The concept of evaluating and analyzing spatial accessibility and networks could be considered the quintessential use of GIS- the program is, after all, a powerful tool for spatial analysis, and accessibility to various networks of travel (streets, sidewalks, railways, etc.) is a very intuitive GIS task in some respects.  

The scenario above involves the closing of the Cypress Creek Campus of Austin Community College, in Travis County, Texas.  The yellow crosses are the other ACC campuses, with the map on the left including Cypress Creek.  The concentric polygons surrounding the campus locations represent the amount of time one might spend driving to that location from that distance, in increments of 5 minutes.  The overlap areas of the drive-time areas is immediately evident, and visual comparison of the two scenarios reveals the fact that a portion of the north-west part of Travis County will no longer be within a 15-minute drive of an ACC campus with the closure of Cypress Creek.  Models like this are useful for analysis of spatial access, as they take into account factors such as road accessibility (one-way restrictions, dead ends, etc.), speed limits, and even historic traffic patterns can be used as a model parameter.  If one were required to assess the impact of closing a college campus, like the above scenario, ArcGIS would be an invaluable asset for analyzing the situation.

Geoprocessing In ArcGIS

Geoprocessing is a cornerstone of ArcGIS, and is a general term that describes most of the spatial analytic functions of the program.  In my previous jobs working with GIS I didn't really use these tools, as I was primarily making maps for visual display. The use of GIS in any capacity beyond this typically requires use of geoprocessing tools.

The graphic above is a polygon representation of a basin area, with areas containing soil types designated as "not suitable for farming" removed. This was derived from polygon vector graphics of the basin and the various soil types that this area consists of, but the notable aspect of it is that the operation to remove the specific areas with the farming-unsuitable soil was performed with a Python script/model within ArcGIS.  Normally, one might use each tool within the program- clip, select and erase- one at a time, with the end result being the polygons above.  To streamline the workflow, which would be necessary for dealing with data in greater volumes, we can create a model within ArcGIS which will perform each operation in sequence when it is run, and can also be exported and modified as a Python script.  Models exported and modified as scripts have the advantage of potentially performing more complicated analyses, can work without the necessity of having ArcMap open, and can be scheduled to run at a designated time, automatically.  In my previous GIS career at a city board of public works, which administered water, sewer and electricity services, I worked with someone who created these kinds of scripts for our daily use, and they were essential to our operation.  I feel fortunate to now have the opportunity to learn more about how that might be done, and hope to possibly provide something as useful to someone's workflow someday myself.  

Visibility Analysis and Camera Surveillance

Modern society, in public places, is under nearly constant video surveillance.  (http://www.cnn.com/2013/04/26/tech/innovation/security-cameras-boston-bombings/) The merits and ethics involved in public video surveillance are endlessly debatable, but the logistic issues inherent in constant camera visibility do present some uniquely challenging problems involving spatial analysis.  Finding the best locations and viewing angles for cameras capturing the finish line of the Boston Marathon is the task du jour, aided by use of the 3-dimensional spatial analytic tools available in ArcGIS.

Above is the location in Boston where the finish line of the marathon is placed- symbolized with a pink star.  The purple circles are the camera locations, and the shades of blue and green are locations visible from one, two and all three of the cameras.  The cameras are placed to maximize the viewing angles and coverage of the finish line and the street on that block.  The blue and green colors are the output generated by the 3D Spatial Analyst to indicate the views offered by this combination of cameras, which is created with the use of an elevation layer that indicates the location and height of visible obstructions (buildings, etc).  The camera's locations are also input with a vertical offset, as if they were being placed on the top of the adjacent buildings, which further increases the size of their field of view.  If one were attempting to assess the placement of surveillance equipment these tools would be invaluable, as they would provide a convenient means of evaluating potential camera locations by indicating specifically which areas are visible from which cameras.          

Issues of Ontological Perspective and Formalization in GIS

In the so-called “Digital Age” of the 21^st century we are privy to an ongoing debate between the merits of increased digitization of our world and the idea that this strengthening focus and reliance on computational method has the real potential to dehumanize and pervert our traditional empirical sciences.  Nadine Schuurman argues in her 2006 piece for the Annals of the Association of American Geographers “Formalization Matters: Critical GIS and Ontology Research” that this debate is unnecessarily distracting within the field of GIScience, and that the ontological focus that many feel is required is now being fully integrated within the discipline itself. 

Addressing the issues raised by many traditionalists within the GIScience field regarding the increased need for perspectives critical of the process of formalizing the conceptual, particularly with respect to those processes involving computational algorithms, needs to acknowledge the concept of “Code Space” (Schuurman, 2006, p. 729).  “Code Space” is the necessary intertwining of computer code with virtually every aspect of our day-to-day lives, including those aspects which are affected by spatial models, analysis and decision making.  Those who may be wary of this increased focus on the computational aspects of GIScience, within the discipline of geography in general, need to recognize that this shift is somewhat inevitable, given society’s prevailing and ubiquitous integration of computers and computer code/algorithms.  Schuurman argues that because this shift is common to every area of modern life, including geography’s formalized representation of the real world, it is necessary for those opposed to the paradigm shift to at least recognize the efforts being made at integrating an awareness of ontological and epistemological principles into GIScience (p. 735).

Expressing any kind of natural or human spatial relationship involving the physical environment is a process of abstraction, of formalizing the conceptual, and of necessarily losing some amount of nuance and detail.  Computer code and models exemplify this simplification, regardless of their attempts at complex accuracy, and at times it is completely necessary to purposely compromise the detailed accuracy of spatial analysis and cartographic information, in order to render it more palatable to certain audiences (p. 730).  Whether it is even possible to produce a completely accurate model or representation of these processes and analytics is a debate between Epistemelogical and Ontological Complexity, and is not likely to ever be truly definitively settled.  Schuurman contends, though, that the implicit acknowledgement and recognition of ontological principles is now becoming a dedicated facet of the disciplines of GIScience, GIS and geography (p. 735).  If the mere recognition that formalization with computational methods raise unique and fundamental ontological and epistemological issues can become innate within the formal framework of these disciplines, the traditionalists should have nothing to fear.  As long as the dialectic remains present, address and awareness of potential issues will abide.

 

Reference source:

Schuurman, N. (2006). Formalization Matters: Critical GIS and Ontology Research. Annals Of The Association Of American Geographers, 96(4), 726-739. doi:10.1111/j.1467-8306.2006.00513.x

Debugging, Error Handling, and more Trials of Perseverance & Stoicism

Programming errors are unavoidable.  For the novice programmer they are an omnipresent threat, waiting at the ready to crash a script and destroy the prospect of an early evening, to test the already frayed nerves and severely waning patience of a long suffering student.  They are how we learn- the crux of our trial and error Python education, and we must adapt to handle them with grace.  Or at least know how to deal with and fix them, and not smash our computers in the process.

Debugging script, or finding and remedying errors therein, is a skill and an art in and of itself.  There are plenty of ways to do it- the dubugging tools in PythonWin, for example, can be exceedingly helpful, or inserting print statements to isolate where in the script the error is occurring.  Finding one's preferred method is, appropriately, a trial and error process.  The screenshot above is the output of one of three simple scripts, the task at hand was to isolate and fix the errors secreted within.  Simple syntax errors- incorrect capitalization, indentation or spelling- will prevent a script from running entirely, and, like all typos, are easy to inadvertently add when typing.  These errors are relatively easy to find and fix, though, especially with the spellcheck-like "check" tool available in PythonWin.  More difficult, without question, are logic and exception errors, which occur within the code's structure, and can be exceptionally difficult to isolate and fix.  These are the errors we are faced with in our second, more insidiously villainous, script.

Here we have the screenshot of an output that is a list of the layers in a map document.  This script was a rather wonderful lesson in perseverance, as well as finding and isolating exception errors.  There were eight total errors to locate, and it was not meant to be easy.  There is something to be said, however, for the satisfaction that comes from spending upwards of 2 hours scouring tutorials and Help documents, and finally reaching that Gestalt moment when all becomes clear and the error is removed.  It is one of the more gratifying experiences one can have, to be sure.  Working with the various modules, tools and processes within the script was pretty illuminating as well- as running through it line by line with the debugger several dozen times provides ample opportunity to become familiar with them.  Exception errors are not always easy to locate and remedy, but learning to do so is an invaluable skill.

The final output, which could be contended to have the easiest "solution," was also probably the trickiest, possibly because the solution wasn't very complicated.  The idea here was to allow an error to occur within the first part (part A), using a try-except statement, so that it didn't cause a runtime error that would stop the script, and the second part (part B) could produce its output regardless.  The concept behind the try-except condition was not the easiest to grasp though, which made its correct implementation quite onerous.  The idea behind it is to instruct the program to try to execute some block of code, and in the event of an exception to produce some alternate output- like an error message- rather than stopping altogether.  A condition that allows for exceptions without crashing the entire process is a useful one to know, and the time spent on this one was, like the first two, worth the trouble and frustration for the insight it provided.   

Watershed Analysis & Stream Models

The Hawaiian Island of Kauai is a volcanic landscape that receives tremendous amounts of precipitation throughout the year, like all of the Hawaiian Islands, but is generally more forested and less developed than its fellows.  Creating an accurate model of the watersheds on the island presents a unique challenge because of the excess of rain, heavy and ultimately greatly varied amounts of forest cover, and a landscape surface composed primarily of volcanic material.  All of these factors can effect the amount of surface water present, and all contribute to the size and location of the island's many rivers and streams.

One of the aims of this week's final map was to depict the visual differences between the model-produced streams and watershed layers and the vector graphics of what actually exists.  The map above shows quite clearly that there are a myriad of discrepancies between both the stream and watershed locations produced by the ArcGIS model and those actually present.  One of the user-specified parameters involved in both models is the "accumulation threshold," or the drainage area required to produce an actual stream.  If this threshold is set too high the model output will contain too few streams, if it is set too low the output will contain too many. Upon comparison between a few generated outputs created using different specified thresholds with both aerial photos and the existing vector layers, it becomes evident that creating a model that produces results accurate to reality is no easy feat.  The above was produced with a rather low threshold, which allowed for more streams in the output- presumably logical, given the climate and terrain of the island.  One outcome affected by this, though, is the size of the model-generated watershed. Clearly the model's output (in orange) falls quite short of what is recognized as the actual watershed boundary (the yellow).  This exercise was a good lesson in the potential challenges involved with creating an accurate spatial model, and how the choices of data input can drastically affect the model's results.

Python Fundamentals, part II

This week we continue with the theme of Python scripting basics, and throw in an introduction of conditional statements.  Python, like most programming languages, accomplishes a plethora of different tasks with conditional statements; one could contend intimate knowledge of their use is a cornerstone of any successful Python tutelage.  Another integral function of Python's ability to control different workflows is the loop structure- such as while and for statements.

This screenshot displays an output of a script that uses lists of strings and numbers, a random number module, and some loop and conditional statements.  The output displayed (at the bottom of the script) is a list of random numbers, with an "unlucky" number defined, identified within the random list, an output of the list with all instances of that number removed, and one of two statements- dependent upon whether the defined number was present in the random list or not. It is also the fruits of a few hours of what might be considered, on the part of this blog author, the ultimate test of "trial-and-error patience."  Merely explaining to someone how a given process works doesn't make for a very effective lesson.  To really teach someone a concept like writing Python script it is necessary to, at a certain point, state the required output, and omit the step-by-step instruction.  It is then up to the student to sink or swim- learn how to use the code and produce the results, or don't.  It is an effective teaching strategy, to be sure, but an unfortunate by-product (for those of us with pathetically short tempers) is vexation of a level difficult to enumerate in words.  The script is written, though, the task completed, and the lesson not one to be easily forgotten.

Corridor & Cost Path Analysis

The cost of movement is usually something thought of in terms of money, in the context of shipment or transport.  Cost can take the form of a whole host of other factors, though, when it refers to some expenditure to move across a landscape.  We can express cost in terms of time, energy required, or even impact on the environment.  Anything that effects movement can be modeled as a cost, which can also be translated into a predictive measure, as in the case of corridor modeling and analysis.

  
The above is a map of the results of a corridor model for black bear movement between two disconnected portions of a National Forest.  The placement of the corridor was chosen in terms of an analysis of the elevation, land cover, and distance from roads- with its final size and position determined by the preferred habitat of the black bear.  Each cost, or factor affecting the bears' possible movement, was given a weight, and the GIS output was a predictive corridor between the two protected areas.  Possible movement of something as unpredictable as a wild animal can never be definitively plotted, but expanding a possible path into a corridor, with placement based on measurable factors, can provide a useful predictive tool.

Python Fundamentals, part I

This week we begin with the fundamentals of Python- how it is structured, and how it handles various types of data, like text, numbers, and lists.  Programming language, like communicative language, uses syntax, and the way various elements included are ordered and input makes the difference between a desired output and an angry red error message.

The output pictured above- my last name and the number 36- doesn't seem like much to produce, but the script used to create it took no small amount of effort.  It was not a simple "print" (to the screen) command to return those values, but rather a structured script that took my full name, put my first, middle and last names into a list, extracted my last name from the list, took the number of letters in my last name, and finally output that number times 3.  This process is completed in with Python script, using variable assignments, functions, methods and expressions.  It is a fairly basic procedure, and may seem a bit arbitrary, but the concepts learned and mastered therein are essential to the development of a decent working knowledge of Python.  It is upon these basic skills we build a greater understanding and mastery of the programming language.  

Land Use Suitability Modeling

If you've ever taken the time to consider, for some location, why something is there, whether it be man-mad or naturally occurring, you may be able to understand why land use suitability analyses and models are done.  The particulars of how these are typically structured and carried out is our topic for this week.

The above is a comparison of two methods of suitability analysis, both performed on raster data, and both created using a weighted overlay process.  The weighted overlay model assigns a weighted relative value to each raster cell, the difference between the two maps above being the equal weights scenario assigns the same level of importance to each of the five input values, and the alternative scenario assigns a different percent weight to each.  Each cell's value in the final output is a result of combining the relative values of the five inputs, which were raster data indicating certain criteria- such as land cover, slope, distance from rivers and roads and type of soil.  The values of the input cells for each criteria were based on a 1 - 5 scale, with higher values representing locations more desirable, such as those nearer to rivers and with certain types of soil.  By combining the five weighted values for the input (criteria) rasters, an output composed of cells indicating the relative suitability of an area, based upon the desired criteria, can be produced for a location like the study area in the maps above.  

An Introduction to Python & Computer Programming for GIS

A new semester, a new challenge- and this summer finds your humble blog author embarking on an endeavor to learn the basics of writing script for GIS with Python- ESRI's scripting language of choice for ArcGIS.  It is probably relevant to point out here that my prior experience with scripting in GIS, in its totality, involved listening to a particularly vocal co-worker complain about simple VBA operations for the version 9.2 ArcGIS we were using in those years, and I do recall him mentioning Python a few times as well.  Now (many years later) we are on ArcGIS version 10.2, and Python has been declared the victor in becoming the go-to script for automating various tasks in ArcGIS.

Python lists being open source and relatively simple/user-friendly among its advantages, and it is essential for any competent GIS professional to be at least familiar with its workings.  Even better, though, is to be at least somewhat well-versed in its use for automating geoprocessing tools, and that is the aim of this course.  We begin with learning the basics of opening the scripting and interaction windows in our Python editor- PythonWin.  From there we move on to becoming familiar with some of the basics on problem solving processes with scripts, and modeling with pseudocode and flowcharts.  Like any language, Python has its idiosyncratic ins and outs, and familiarity with some of their fundamentals is necessary as a start.

The above is a screenshot of a directory of folders that was created with a Python script.  The script was run, and the folders created, on the remote desktop that we use to access ArcGIS Desktop.  ESRI does, however, provide a free-of-charge, trial version of the software for students, good for one year.  Working on the remote desktop can be slow at times, especially on weekends when it seems that everyone in class is finishing their work, so it's often easier to use the software version run locally, without the remote connection.  The script provided for this week's exercise, that creates a directory of folders that will be used this semester, specifically references the drive used on the remote desktop connection though, and as-is must be run there to work.  Upon opening the script file in the PythonWin editor I found the specific line that does this with the simple command of "courseFolder = x:\xxx."  Running the script locally, on my laptop, I changed the location referenced (the "x"s in my example here) to the information for my personal hard drive, and ran the script there.  Lo and behold, it created the file directory (as in the screenshot above) right where I specified- on my computer.  This small triumph of successfully modifying provided code is to be savored, for the impression that these will be relatively rare this semester is rather striking.  Onward and upward we will continue though, on this exploration into the arcane and hallowed world of computer programming... 

Final Project: a GIS Assessment for the Bobwhite-Manatee Transmission Line

http://students.uwf.edu/emv5/IntroGIS/4043FinalProjectPresWeblink.ppsx

http://students.uwf.edu/emv5/IntroGIS/4043FinalProjectPresentText.pdf



The above are links to my final project- an assessment of the ways in which GIS can be used in planning a route through Manatee and Sarasota counties for FPL's new Bobwhite-Manatee Transmission Line, and what effects the line's route may have on local residents and the environment.  Included also is an estimate of the line's length, and its potential cost.

GIS is an integral tool for planning construction projects like this.  A 25 mile 230kV transmission line running through 2 very densely populated counties in south Florida is a huge project to undertake.  The route must be strategically planned, so that it doesn't effect too many local residents, and also shouldn't put too much stress on potentially fragile environments.  The potential route also must be accurately measured, so that an estimate of cost can be made.

All of the above objectives can only be completed with the assistance of GIS.  Maps can be used as powerful visual aids to convey some concern or idea, and can also be used to perform various analyses essential to the transmission line route's planning and construction.