I Get By With A Little Help From My Friends

Talking to a friend the other day it occurred to me how fantastic it would be if Vr could import and display OpenStreetMap data.  Nothing fancy mind you but maybe create an image that you could use as a background for a quick check of the surrounding area or even flight prep.  Wow that sounds like a lot of work unless of course someone else has already done it.  One of the many beauties of Python is the number of fantastic libraries out there if you just spend a little time tracking them down.  If you take that a step farther and are willing to pass a little of the responsibility outside python using system calls or batch files the possibilities are nearly endless.  Let's just say I can now display an OpenStreetMap map image in Vr and it only took about 90 lines of python.  Here are a few of the major helpers besides the map site which is so worthy of support, contribution, and praise.

I know I have mentioned it before but I can't say enough about pyproj which I use to convert my local coordinates to geographic for sending out to various places.

Once I have the corners in Lat / Long the rest will be done by a batch file which is about the only thing my python script creates.  It could be done in other ways but I liked the idea of creating something that could stick around and be easily modified for use again.  So the batch file next makes a call to the openstreetmap.org API for grabbing the map data.

Once the OSM file is in house it will be converted to a bitmap using Maperitive.  I only use such a tiny fraction of the capabilities of this amazing program, but for my purposes using it's command line version with a custom generated script will fit the bill perfectly.

The original bitmap is created in Web Mercator and can also throw in a TFW and KML file for generic use or display by anybody if you want to send them along.  I however want to display it behind my map in a local coordinate system in whatever system I choose so the last step is to re-project the image using GDAL.

Is this the most advanced, professional, elegant way of doing it?  Of course not, you obviously don't know me very well.  As usual it is something you can hack together in just a short time with minimal duplication of effort that gets the job done.  Hooray for real programmers!!!

Don't forget to support your favorite projects with contributions or cash when possible.

Note to self: Step backward through LiDAR when removing, and repr ( round ( a number )) is awsome

First, apparently I never noticed this before but while trying to step though a bunch of LiDAR points to remove a certain class it apparently changes something dynamically (like the point number in the buffer).  I noticed this because I was doing a

    for PointNum in range(Punt.GetCount()):
        if Punt.Cla(PointNum) == Whatever:
            Punt.DelPunt(PointNum)

To get rid of a certain type of point, in this case based on class.  What I noticed though was that each time it ran there were a bunch of points left over (half??).  I decided that even though I wasn't doing a record until the loop was done, the point number in the buffer must change as points are deleted.  So what to do?  How about starting at the last point and removing them from the top end of the buffer so as you work backwards the previous point numbers would not have changed.  I came up with

for PointNum in range(Punt.GetCount(),-1,-1):

and it seems to work fine.

Second and totally unrelated I needed to rebuild a tile naming structure for a local dataset.  There is a tile scheme that is based on 5000 foot tiles that is easy enough to get the names of.  The problem is that there is also a subset of 1250 foot tiles that starts from the corner of each 5k block but the names are based on 1000 foot increments.  When you digitize a random point in a block you can't necessarily round down to the lower 1000 because they just truncate the thousands from a rounded 1250.  Ok hard to imagine but the corners would be 0, 1250, 2500, 3750 so a number 3749 would fall in a 2500 block and truncate to 2000 meaning the tile would have 2 in the name, not to mention dropping a million off one of the coordinates.  Now for the why I love python part!

repr(round(1773691/1250)*1250)[1:4]

yields '772' just like is necessary to build the proper tile name. 

Note to self: Find a reason to use geopy

Full disclosure; This has nothing to do with anything except that I was playing with a really cool module today and need to make sure I remember what I just learned.  The module is geopy (which by the way I installed using pip for the first time ever ) which can be used to geocode natural language addresses using a variety of APIs.  Don't know when, how, or why I'll use it but I'm certain I will.

Info can be found at https://pypi.python.org/pypi/geopy
Doc at http://geopy.readthedocs.org/en/latest/

Here is what I just did (using my current address)

>>> import geopy
>>> geopy.geocoders.GoogleV3().geocode("4090 weaver ct s 43026")
Location((40.035917, -83.144145, 0.0))
>>> geopy.geocoders.GoogleV3().geocode("4090 weaver ct s 43026").raw
{u'geometry': {u'location': {u'lat': 40.035917, u'lng': -83.144145}, u'viewport': {u'northeast': {u'lat': 40.03726598029149, u'lng': -83.1427960197085}, u'southwest': {u'lat': 40.03456801970849, u'lng': -83.1454939802915}}, u'location_type': u'ROOFTOP'}, u'formatted_address': u'4090 Weaver Court, Hilliard, OH 43026, USA', u'place_id': u'ChIJpUOLF-iTOIgRT-Vqu5rHhLo', u'address_components': [{u'long_name': u'4090', u'types': [u'street_number'], u'short_name': u'4090'}, {u'long_name': u'Weaver Court', u'types': [u'route'], u'short_name': u'Weaver Ct'}, {u'long_name': u'Northwest Industrial Complex', u'types': [u'neighborhood', u'political'], u'short_name': u'Northwest Industrial Complex'}, {u'long_name': u'Hilliard', u'types': [u'locality', u'political'], u'short_name': u'Hilliard'}, {u'long_name': u'Norwich', u'types': [u'administrative_area_level_3', u'political'], u'short_name': u'Norwich'}, {u'long_name': u'Franklin County', u'types': [u'administrative_area_level_2', u'political'], u'short_name': u'Franklin County'}, {u'long_name': u'Ohio', u'types': [u'administrative_area_level_1', u'political'], u'short_name': u'OH'}, {u'long_name': u'United States', u'types': [u'country', u'political'], u'short_name': u'US'}, {u'long_name': u'43026', u'types': [u'postal_code'], u'short_name': u'43026'}, {u'long_name': u'1197', u'types': [u'postal_code_suffix'], u'short_name': u'1197'}], u'partial_match': True, u'types': [u'street_address']}

LMSTFY ( Let Me Sealay.py That For You)

I should have mentioned this long ago before you invested time in reading my blog, but here goes... Fact is I'm not particularly bright, I'm just inquisitive (and as mentioned previously constructively lazy).  This means a couple of things; first I love to find an easier way to do almost anything, and second sometimes I spend more time researching than I would if I had just done the task (oh but next time it will be so much faster). Because of this, sometimes I know things, and when you know things people like to ask you things because you might know that particular thing they want.  Yes almost daily I am tempted to just say "Let me Google that for you" because that would be much easier for me than actually explaining it (reference lazy thing above).

I have about 150 out of the 240 layer numbers that I use daily in mapping projects memorized.  Over time though you need to add new layers or break them down to a more granular level so consequently you can see that there are a bunch I don't know.  I could memorize them (and a few I will) but the majority of the others I just want an easy way to find the rare one I need.  Hence sealay.py or "Search Layer Names".

print 'sealay.py modified 7:31 AM 4/16/2015'
# Display layers matching search string
'''
Copyright 2015 Dennis Shimer
Vr Mapping copyright Cardinal Systems, LLC
No warranties as to safety or usability expressed or implied.
Free to use, copy, modify, distribute with author credit.

Prompts user for layer name (or any part) then displays layer
numbers and full names of layers matching string.
'''

def lines2lists(AListOfDataLines):
    '''
    Function readlines returns an entire file with each line as a string in a
    list of data.  This function will convert each string into a list of words,
    then return a list of lists. Why? Just because I like to work this way.
    Example:            lines2lists(['first line','the second line','or sentences'])
    Would return:       [['first','line'],['the','second','line'],['or','sentences]]
    '''
    DataList=[]
    for Line in AListOfDataLines:
        DataList.append(Line.split())
    return DataList

PrintString=''
PrintList=[]

#If sent argument use as search string else prompt.
if VrArgs:
    SearchText=VrArgs[0]
else:
    SearchText=PyVrGui().InputDialog ('Search String', 'Layer numbers by name')[1]
   
LayerFileName = VrCfg().GetLayerNameFile ()
LayerFile = open(LayerFileName , 'r')
LayerData = LayerFile.readlines()
LayerFile.close()
LayerList = man.lines2lists(LayerData)

#Search through data, if string found add it to a multi-line printable string.
for DataLine in LayerList :
    if len(DataLine) > 1 :
        if DataLine[1].count(SearchText.upper()):
            PrintString=PrintString+DataLine[0]+' '+DataLine[1]+'\n'
            PrintList.append(DataLine[0]+' '+DataLine[1])

# Two possibilities, the first commented line just displays the found layers
# The rest will let the user select one correct entry and send a LAY= command.
if PrintString:
#    PyVrGui().MsgBox(PrintString , 'Matching Layers')
    PromBox = VrPromBox ("Set Layer", 30, 1)
    PromBox.AddList ("Layer", 40, len(PrintList)+1, 0)
    for LayerItem in PrintList:
        PromBox.AddListItem (LayerItem)
    if (PromBox.Display(0) == 0):
        SetToLayer= PromBox.GetListByPrompt ("Layer")
    if SetToLayer : PyVrGui().PushKeyin('lay={:s}'.format( SetToLayer.split()[0]))
else :
    PyVrGui().MsgBox('No Match to {:s}'.format(SearchText), 'Matching Layers')

You gotta love a good library

The Columbus Metropolitan Library is arguably one of the best libraries in the nation.  I use it so much I have had my library card memorized for over 25 years.  The idea that there is information and services there at my fingertips any time I want is fantastic.  I was thinking about this yesterday when I started pulling functions out of PyVrGeom().  Libraries (modules) can be everything from little bits of code snippets that you want to reuse over and over to complex collections of classes that you could only dream up but never realistically code yourself. 

For example PyVrGeom is just a collection of math that some would find easy enough to code, but why do so when somebody has done the heavy lifting and offers you the calls to do it with some preexisting code.  In this case very professionally created and presented, but it is just as easy to hack together some snippets that just save you typing.  For example when I was working on a KML export routine I didn't need to learn something complex and universal (which libs probably exist), I just knew what it looked like and wanted to replicate those entities by passing along a few basic parameters.  In this case I slapped together some code to create the KML entities, but at the same time the coordinates needed to be in geodetic coordinates so I had to grab a projection library that someone way sharper than me had already put together. Then it struck me that a KMZ is just a file zipped using the standard algorithms so if I import that one I can just as easily write both KML and KMZ files.

I'm not going to offer any code this time just another reminder that there are some good reasons to install python if you are going to do some scripting.  Here are some of the libraries that I really enjoy and use often.

pyproj - for converting local coordinates to global (or from one to another).
liblas - for all things LiDAR related.
zipfile - for reading and writing compressed files.
xlrd/xlwt - for directly reading and writing Excel spreadsheets.

Along with some of the standard system libraries that you can find a million uses for like sys, os, time, and math.

And here is a tip I'll add for free, check out http://www.lfd.uci.edu/~gohlke/pythonlibs/