30 Second Functions

When is it faster to write the function than to even find out how to do it in the help.  Well ok, I did actually search and didn't find out how, but here is what would have been faster.

I use handwheels,  I wanted an easy way to switch the z input from the footdisk to the right handwheel and back.  There is probably a way to pass an argument into a macro but I couldn't find it so enter....

zhand.py

print 'z handwheel'
VrCfg().SetHwLocZ (2)

The beauty of VrCfg is that it modifies the environment in real time just like calling any of the functions that would do the same thing via a dialog box.

Saying anything else would just not have a point, except, always be thinking of ways to extend the usefulness of existing processes.

I Get By With A Little Help From My Friends

Talking to a friend the other day it occurred to me how fantastic it would be if Vr could import and display OpenStreetMap data.  Nothing fancy mind you but maybe create an image that you could use as a background for a quick check of the surrounding area or even flight prep.  Wow that sounds like a lot of work unless of course someone else has already done it.  One of the many beauties of Python is the number of fantastic libraries out there if you just spend a little time tracking them down.  If you take that a step farther and are willing to pass a little of the responsibility outside python using system calls or batch files the possibilities are nearly endless.  Let's just say I can now display an OpenStreetMap map image in Vr and it only took about 90 lines of python.  Here are a few of the major helpers besides the map site which is so worthy of support, contribution, and praise.

I know I have mentioned it before but I can't say enough about pyproj which I use to convert my local coordinates to geographic for sending out to various places.

Once I have the corners in Lat / Long the rest will be done by a batch file which is about the only thing my python script creates.  It could be done in other ways but I liked the idea of creating something that could stick around and be easily modified for use again.  So the batch file next makes a call to the openstreetmap.org API for grabbing the map data.

Once the OSM file is in house it will be converted to a bitmap using Maperitive.  I only use such a tiny fraction of the capabilities of this amazing program, but for my purposes using it's command line version with a custom generated script will fit the bill perfectly.

The original bitmap is created in Web Mercator and can also throw in a TFW and KML file for generic use or display by anybody if you want to send them along.  I however want to display it behind my map in a local coordinate system in whatever system I choose so the last step is to re-project the image using GDAL.

Is this the most advanced, professional, elegant way of doing it?  Of course not, you obviously don't know me very well.  As usual it is something you can hack together in just a short time with minimal duplication of effort that gets the job done.  Hooray for real programmers!!!

Don't forget to support your favorite projects with contributions or cash when possible.

Note to self: Step backward through LiDAR when removing, and repr ( round ( a number )) is awsome

First, apparently I never noticed this before but while trying to step though a bunch of LiDAR points to remove a certain class it apparently changes something dynamically (like the point number in the buffer).  I noticed this because I was doing a

    for PointNum in range(Punt.GetCount()):
        if Punt.Cla(PointNum) == Whatever:
            Punt.DelPunt(PointNum)

To get rid of a certain type of point, in this case based on class.  What I noticed though was that each time it ran there were a bunch of points left over (half??).  I decided that even though I wasn't doing a record until the loop was done, the point number in the buffer must change as points are deleted.  So what to do?  How about starting at the last point and removing them from the top end of the buffer so as you work backwards the previous point numbers would not have changed.  I came up with

for PointNum in range(Punt.GetCount(),-1,-1):

and it seems to work fine.

Second and totally unrelated I needed to rebuild a tile naming structure for a local dataset.  There is a tile scheme that is based on 5000 foot tiles that is easy enough to get the names of.  The problem is that there is also a subset of 1250 foot tiles that starts from the corner of each 5k block but the names are based on 1000 foot increments.  When you digitize a random point in a block you can't necessarily round down to the lower 1000 because they just truncate the thousands from a rounded 1250.  Ok hard to imagine but the corners would be 0, 1250, 2500, 3750 so a number 3749 would fall in a 2500 block and truncate to 2000 meaning the tile would have 2 in the name, not to mention dropping a million off one of the coordinates.  Now for the why I love python part!

repr(round(1773691/1250)*1250)[1:4]

yields '772' just like is necessary to build the proper tile name. 

Note to self: Find a reason to use geopy

Full disclosure; This has nothing to do with anything except that I was playing with a really cool module today and need to make sure I remember what I just learned.  The module is geopy (which by the way I installed using pip for the first time ever ) which can be used to geocode natural language addresses using a variety of APIs.  Don't know when, how, or why I'll use it but I'm certain I will.

Info can be found at https://pypi.python.org/pypi/geopy
Doc at http://geopy.readthedocs.org/en/latest/

Here is what I just did (using my current address)

>>> import geopy
>>> geopy.geocoders.GoogleV3().geocode("4090 weaver ct s 43026")
Location((40.035917, -83.144145, 0.0))
>>> geopy.geocoders.GoogleV3().geocode("4090 weaver ct s 43026").raw
{u'geometry': {u'location': {u'lat': 40.035917, u'lng': -83.144145}, u'viewport': {u'northeast': {u'lat': 40.03726598029149, u'lng': -83.1427960197085}, u'southwest': {u'lat': 40.03456801970849, u'lng': -83.1454939802915}}, u'location_type': u'ROOFTOP'}, u'formatted_address': u'4090 Weaver Court, Hilliard, OH 43026, USA', u'place_id': u'ChIJpUOLF-iTOIgRT-Vqu5rHhLo', u'address_components': [{u'long_name': u'4090', u'types': [u'street_number'], u'short_name': u'4090'}, {u'long_name': u'Weaver Court', u'types': [u'route'], u'short_name': u'Weaver Ct'}, {u'long_name': u'Northwest Industrial Complex', u'types': [u'neighborhood', u'political'], u'short_name': u'Northwest Industrial Complex'}, {u'long_name': u'Hilliard', u'types': [u'locality', u'political'], u'short_name': u'Hilliard'}, {u'long_name': u'Norwich', u'types': [u'administrative_area_level_3', u'political'], u'short_name': u'Norwich'}, {u'long_name': u'Franklin County', u'types': [u'administrative_area_level_2', u'political'], u'short_name': u'Franklin County'}, {u'long_name': u'Ohio', u'types': [u'administrative_area_level_1', u'political'], u'short_name': u'OH'}, {u'long_name': u'United States', u'types': [u'country', u'political'], u'short_name': u'US'}, {u'long_name': u'43026', u'types': [u'postal_code'], u'short_name': u'43026'}, {u'long_name': u'1197', u'types': [u'postal_code_suffix'], u'short_name': u'1197'}], u'partial_match': True, u'types': [u'street_address']}

LMSTFY ( Let Me Sealay.py That For You)

I should have mentioned this long ago before you invested time in reading my blog, but here goes... Fact is I'm not particularly bright, I'm just inquisitive (and as mentioned previously constructively lazy).  This means a couple of things; first I love to find an easier way to do almost anything, and second sometimes I spend more time researching than I would if I had just done the task (oh but next time it will be so much faster). Because of this, sometimes I know things, and when you know things people like to ask you things because you might know that particular thing they want.  Yes almost daily I am tempted to just say "Let me Google that for you" because that would be much easier for me than actually explaining it (reference lazy thing above).

I have about 150 out of the 240 layer numbers that I use daily in mapping projects memorized.  Over time though you need to add new layers or break them down to a more granular level so consequently you can see that there are a bunch I don't know.  I could memorize them (and a few I will) but the majority of the others I just want an easy way to find the rare one I need.  Hence sealay.py or "Search Layer Names".

print 'sealay.py modified 7:31 AM 4/16/2015'
# Display layers matching search string
'''
Copyright 2015 Dennis Shimer
Vr Mapping copyright Cardinal Systems, LLC
No warranties as to safety or usability expressed or implied.
Free to use, copy, modify, distribute with author credit.

Prompts user for layer name (or any part) then displays layer
numbers and full names of layers matching string.
'''

def lines2lists(AListOfDataLines):
    '''
    Function readlines returns an entire file with each line as a string in a
    list of data.  This function will convert each string into a list of words,
    then return a list of lists. Why? Just because I like to work this way.
    Example:            lines2lists(['first line','the second line','or sentences'])
    Would return:       [['first','line'],['the','second','line'],['or','sentences]]
    '''
    DataList=[]
    for Line in AListOfDataLines:
        DataList.append(Line.split())
    return DataList

PrintString=''
PrintList=[]

#If sent argument use as search string else prompt.
if VrArgs:
    SearchText=VrArgs[0]
else:
    SearchText=PyVrGui().InputDialog ('Search String', 'Layer numbers by name')[1]
   
LayerFileName = VrCfg().GetLayerNameFile ()
LayerFile = open(LayerFileName , 'r')
LayerData = LayerFile.readlines()
LayerFile.close()
LayerList = man.lines2lists(LayerData)

#Search through data, if string found add it to a multi-line printable string.
for DataLine in LayerList :
    if len(DataLine) > 1 :
        if DataLine[1].count(SearchText.upper()):
            PrintString=PrintString+DataLine[0]+' '+DataLine[1]+'\n'
            PrintList.append(DataLine[0]+' '+DataLine[1])

# Two possibilities, the first commented line just displays the found layers
# The rest will let the user select one correct entry and send a LAY= command.
if PrintString:
#    PyVrGui().MsgBox(PrintString , 'Matching Layers')
    PromBox = VrPromBox ("Set Layer", 30, 1)
    PromBox.AddList ("Layer", 40, len(PrintList)+1, 0)
    for LayerItem in PrintList:
        PromBox.AddListItem (LayerItem)
    if (PromBox.Display(0) == 0):
        SetToLayer= PromBox.GetListByPrompt ("Layer")
    if SetToLayer : PyVrGui().PushKeyin('lay={:s}'.format( SetToLayer.split()[0]))
else :
    PyVrGui().MsgBox('No Match to {:s}'.format(SearchText), 'Matching Layers')

You gotta love a good library

The Columbus Metropolitan Library is arguably one of the best libraries in the nation.  I use it so much I have had my library card memorized for over 25 years.  The idea that there is information and services there at my fingertips any time I want is fantastic.  I was thinking about this yesterday when I started pulling functions out of PyVrGeom().  Libraries (modules) can be everything from little bits of code snippets that you want to reuse over and over to complex collections of classes that you could only dream up but never realistically code yourself. 

For example PyVrGeom is just a collection of math that some would find easy enough to code, but why do so when somebody has done the heavy lifting and offers you the calls to do it with some preexisting code.  In this case very professionally created and presented, but it is just as easy to hack together some snippets that just save you typing.  For example when I was working on a KML export routine I didn't need to learn something complex and universal (which libs probably exist), I just knew what it looked like and wanted to replicate those entities by passing along a few basic parameters.  In this case I slapped together some code to create the KML entities, but at the same time the coordinates needed to be in geodetic coordinates so I had to grab a projection library that someone way sharper than me had already put together. Then it struck me that a KMZ is just a file zipped using the standard algorithms so if I import that one I can just as easily write both KML and KMZ files.

I'm not going to offer any code this time just another reminder that there are some good reasons to install python if you are going to do some scripting.  Here are some of the libraries that I really enjoy and use often.

pyproj - for converting local coordinates to global (or from one to another).
liblas - for all things LiDAR related.
zipfile - for reading and writing compressed files.
xlrd/xlwt - for directly reading and writing Excel spreadsheets.

Along with some of the standard system libraries that you can find a million uses for like sys, os, time, and math.

And here is a tip I'll add for free, check out http://www.lfd.uci.edu/~gohlke/pythonlibs/

It's all part of the process (or subprocess as the case may be)

On occasion I make the case for installing python in order to extend the usefulness of the environment in Vr, or at least getting hold of some of the most useful standard libraries.  Well here is another good reason.  This is just a snippet that I will use later but I figure this is as good a place as any to keep track of it.

Before too long I want to write a script that will depend on running an external process, then interacting with the results of the process (no spoilers yet).  In order to do that I'm going to make something similar to the following call to process some files leaving the results in the working directory. The research I did here was to make sure I had a way to not proceed until the original process was complete.  Later I'll need to make sure I can run a shell command and capture the resulting output but this will do for now.

import subprocess
process = subprocess.Popen('lasboundary -i *rgb1.las -otxt')
process.wait()
print process.returncode
print '\a'

the last line is just to beep so I knew when it was done.

Reminder snippets - liblas and reading laz files

The other thing that I used to do with this blog is post things that I wanted to remember.  At one point the Vr forums were a good place to do this because it also gave an opportunity to share and collaborate.  Since that doesn't really seem to be happening any more I suppose I could drop things here and then come back to them later if necessary.  Yes I could just do this in a document but who knows maybe a new opportunity for some kind of sharing community will pop up again.

In this case I just want to remind myself that if Vr never adopts the .LAZ LiDAR compression format for reading, it would still be possible using liblas. This is simple a little interactive session that shows it is theoretically possible using my own data.

There may indeed be other better ways, but this seems to work fine.

>>> import liblas
>>> f=liblas.file.File(r'c:\tmp\N1915250.laz',mode='r')
>>> f

>>> p=f.read(100)
>>> p.classification
5
>>> p.x
1919853.35
>>> f.header.compressed
True
>>> f.filename
'c:\\tmp\\N1915250.laz'

5 Minute Functions - 2

I haven't stopped coding, I've just stopped talking about it.  Again more a function of the fact that I doubt anybody really cares.  I noticed the other day that I have slightly over 300 python programs that have accumulated over the years.  Certainly there are the ones I am really proud of like the ones that translate a Vr file into a KML file or a TIN into  LandXML format because of the crucially useful functionality they add.  Then there are the ones I use multiple times an hour, like double points on a line, drive to points by user specified parameters, storing helpful window states, adding clamping points to DTM lines, and on it goes.  Lately I have noticed that I also just stop sometimes and write a quick program because as I mentioned in a previous post of a similar title, I can spend 5 minutes programming and save 15 minutes (and lots of wear and tear on the mouse clicking finger).  Not only is it worth it but it is also fun, keeps my brain lubricated, and gives me a base in case I find the task useful and want to come back later to add options, dialogs, or arguments that would make it more universal.

Here is an example, I have a few thousand points that got dumped into a file without regard to any distinction.  about 300 of them would be really helpful if they were in a certain layer and I don't want to miss any of them.  The only thing that makes them different is that there is a common string of text inside the feature code.  There may very well be an easy way to do this with Vr, but after a couple of tries I couldn't come up with a parameter that worked so I thought, "why not just write a quick script?".  In this instance I'll just hard code the search string and resulting changes. If I find it useful I may come back later and add a simple dialog to make it more agile.

print 'fcglob.py modified 6:23 AM 3/11/2015'
# Globally change something based on text in it's feature code
'''
Copyright 2015 Dennis Shimer
No warranties as to safety or usability expressed or implied.
Free to use, copy, modify, distribute with author credit.

Simple beginning of a function to change entities based on their
feature code.

Variables of interest:
    None
'''
Ws=PyVrWs()
Sym=PyVrSym()
WsNum=Ws.Aws()
Ws.UndoBegin(WsNum,'fcglob')
for EntNum in range ( Ws.GetSymCount(WsNum)):
    Sym.Load (WsNum, EntNum)
    if Sym.GetFc().lower().count(' fh'):
        print EntNum
        Sym.SetLayer(1042)
        Sym.SetGpoint(21)
        Sym.ReRec()
print ('\a')
Ws.UndoEnd(WsNum)

 It started out as about a half dozen lines in pyedi and doesn't amount to much more now. Nothing fancy, just something I can come back to if I need it again, and in this case it really helped.

I don't know, maybe I'll just start dumping more of these here. Doesn't hurt anything and though most of them aren't universally interesting it might give somebody an idea.