Time out

It is that time of year when I begin to pursue my second job as middle school soccer coach, and high school referee (and parent spectator). Because of this, I'm not even going to think about posting until late October.

Finally A Vr Command

Well enough playing around with external files. It's about time to head back to what VrPython was intended for and write a new function. This one in particular allows the user to parallel part of an existing line. This is a good example of how a program evolves (to say it was intelligently designed probably infers too much), because as it started off the user had to digitize the offset distance. Then with the advent of VrArgs in 3.3.? it seemed like a good idea to put in the option to enter an offset as a argument after the user typed parpart. Of course with the opportunity to put in a argument as an offset, it seemed silly to not have a dialog somewhere. The problem was that I didn't want dialog every time, and you already had to Id the line, start point, end point, and offset which was approaching the limits of how much I wanted the user to have to do. Hence the decision that if you got all the way to the point of digitizing the offset and hadn't specified one but hit an "end" or "#" without doing so the program would bring up a dialog to allow typing one in. Well let's take a look

#This has become the standard I use to start each program.
print 'parprt.py ParallelPartial modified 8:23 AM 7/13/2007'
'''
Copyright 2007 Dennis Shimer, M.A.N. Mapping Services Inc.
Vr Mapping copyright Cardinal Systems, LLC
No warranties as to safety or usability expressed or implied.
Free to use, copy, modify, distribute with author credit.

Inserts a line parallel to the identified line.
Allows for picking start and end points of new line, and offset.
If a number is passed it as the first it will be used as an offset.
If the <#> End button is pressed during offset measurement a
dialog will come up prompting for offset.
'''

Ws=PyVrWs()
Line=PyVrLine()
Gui=PyVrGui()
WsNum=Ws.Aws()
# Create all the objects we will use.

while Line.Id() != -1:
# In theory you could partially parallel many lines
# though in reality it doesn't happen much, but will do so
# until the user doesn't Id a line which returns -1.

....Ws.UndoBegin(WsNum,"inspar_part")
# The undo is inside the Id loop so each line paralleled
# could be undone individually.

....Gui.DspMsg0('Dig start of new line')
# Since GetCoord() isn't very customizable as far as buttons
# and messages, we'll display instructions in the message area.

....Stat1,NewX1,NewY1,NewZ1=Gui.GetCoord('Point1')
# And get a coordinate for where the newly paralleled line
# Should begin.

....Gui.DspMsg0('Dig end of new line')
....Stat2,NewX2,NewY2,NewZ2=Gui.GetCoord('Point2')
# And the same on the other end.

....if VrArgs:
........print VrArgs
........Offset=float(VrArgs[0])
# This whole if statement was added later after the introduction
# of VrArgs. If a string (or multiple strings) are placed after
# the script name, they are passed in as a list of strings. So the
# test would fail if VrArgs was empty (nothing passed) and would
# pass to the else statement. If however something was passed,
# convert it to a real number and assign it to the offset. Bonus
# points for anyone who identifies the huge bug I just noticed in
# this 3 line block.

....else:
# In other words if VrArgs is not true (is empty).

........Gui.DspMsg0('Dig offset # End for dialog') ........StatS,SnapX,SnapY,SnapZ=Gui.GetCoord('Offset')
# Same as above, display instructions and read a coordinate
# that represents how far the cursor is from the line.

........Offset=-1.0*(Line.SnapPoint (SnapX, SnapY)[3])
# Use the computed offset. The -1 is just to correct for
# how SnapPoint computes the distance.

........if StatS:
# This comes about if the user hits the end button instead
# of a 1 to cancel the digitize offset.

............PromBox=VrPromBox('Prompt title',20,1)
............PromBox.AddDouble('Enter a real number',.5,2)
............if (PromBox.Display(0) == 0):
................Offset=PromBox.GetDouble(0)
# Just a simple prombox to allow for entering the offset if
# it wasn't passed or computed in any other way.

....Num1=Line.SnapPointAdd (NewX1,NewY1,0.0)
....Num2=Line.SnapPointAdd (NewX2,NewY2,0.0)
# We are adding points to the line object that was identified
# this is ok because later we will Rec() instead of ReRec() which
# will create a new line.

....if Num1>Num2:
........tmp=Num1
........Num1=Num2
........Num2=tmp
# At one point DelPoints didn't work properly if the beginning
# point was higher than the end point (it may not still, but doesn't
# really matter if it is checked and swapped if necessary.

....Line.DelPoints (Num1, Num2, 1)
# Mode 1 in effect deletes points outside begin and end rather
# than between them (which would be mode 0)

....Line.Offset(Offset)
....Line.Rec(WsNum)
....Line.Plot()
# Take that line object, offset it by the distance entered or
# computed, record it as a new entity, plot it and there you go.
# Part of the original line has been stored as a new line parallel
# to it.
....Ws.UndoEnd(WsNum)

Reminder that as with all other scripts, you can either copy and paste what you see here (substituting your prefered indentation for the holding characters "....", or find the original at http://python4vr.googlepages.com

Life Outside Vr Wrap Up (almost)

Before going back to writing functions for Vr let's wrap up a couple of things. First binary files, I have recently begun to like the idea of persistent parameters for several of my programs. I really need to create some basic generic parameter code to paste in. In the end I'll probably settle on ascii files with keywords, or at least comments. But what if you want to use binary, or what if you want to read a binary file that you know (or could guess) the structure of. An example is the old .COO coordinate files used by orientp3 in pre-Vr days. We still use it and often want to work directly with the .COO files for various things. But they are binary, here are some basics working with something similar to the parameter file we used before. This will be plain python, you can copy and paste this, download testbinary.py from my source code website, then either run it from a command line like
python testbinary.py
or open or paste it into pyedi and run it inside Vr. The only requirement is that you will need to have python installed because it uses two standard modules. By this time I am going to assume some python knowledge and ability to read the code. I am thrilled to answer any questions that come up, but I will post this code in a very plain, line at a time manner rather than using multiple methods on a single line. I'm also going to use my favorite method of accessing binary data since as is common in python there are probably dozens of ways of doing it. You could probably do this in about 8 lines but the point is to explain what is going on. I'll also assume that anyone interested in this would take the time to look up the modules listed to see how many cool things they will do... So here goes

If you were to run this script 3 consecutive times it would give 3 different responses. First that the parameter file was created but without any values read. Second it would read the default values, tell you what the values are and modify them. Third it will read the modified values print them out and tell you that it found modified values and not the defaults.

import os,struct
# We'll use os to test for the existence of the file, an struct to
# interact with the binary data.

FormatString='10sddi'
'''
The format string tells several parts of the program what we are working
with in regards to binary data. The format string listed above represents
a 10 character string, 2 double precision real numbers, and an integer.
Same as before but I added the int to show by example. Note that it is
just a string with letters representing different kinds of data.
'''

ParFileName='testbinary.par'
# The file by this name will show up in the directory the script is run in.

print 'Bytes occupied by binary data',struct.calcsize(FormatString),'\n\n'
'''
Sometimes you need to know how many bytes to read from a file in order
to get one set of data when multiple sets exist, a good example goes
back to the coordinate file mentioned earlier. In order to read and process
one point at a time you can only read the necessary bytes. the calcsize
method in struct will read a format string and tell how many bytes are
represented. You can go into a python interpreter, import struct then
evaluate some single item format strings like the following for a
float, a double, an int, and a 1 character string.
>>> struct.calcsize('f')
4
>>> struct.calcsize('d')
8
>>> struct.calcsize('i')
4
>>> struct.calcsize('s')
1
'''

if not os.path.isfile(ParFileName): # If the file doesn't exist.
...ParFile=open(ParFileName,'wb')
...# Then open it in write (create or erase) mode with a binary modifier

...BinaryData=struct.pack(FormatString,'Word',.2,.0,0)
...'''
...The pack method in the struct module uses the format string to process
...the data that comes next into the bytes which represent them in binary
...form. It is important that the amount and type of data following the
...format string match, notice I am passing in a string (less than 10
...characters allowed, 2 real numbers and an integer.
...'''

...ParFile.write(BinaryData)
...ParFile.close()
...# We have a binary file and binary data so just and write and close it.

...print 'Program run with no existing file'
...print 'Data set to defaults.'
...print 'testbinary.par should now exist.'
else:
...ParFile=open(ParFileName,'r+b')
...# If it does exist open for reading + writing with binary modifier.

...BinaryData=ParFile.read()
...'''
...This time since the file exists we are going to get the binary data
...right out of the file with a read. Since we want the whole file there
...is no reason to use calcsize to tell it how many bytes to read.
...'''

...DataList=struct.unpack(FormatString,BinaryData)
...# Going the other direction, lets use unpack to use the format string
...# to convert the binary data into a list of values.

...print 'Data from file, note it is a list\n ',DataList
...# and print the list.

...ParFile.seek(0,0)
...'''
...Now assuming that there are new values that need to be used to update
...the parameter file. The first thing we need to do is go back to the
...beginning of the file, read up on seek, but this basically moves you
...0 bytes from the beginning. If in the coordinate file example I had a
...data structure that require 24 bytes and wanted to get the 3rd value
...from the file I could to the 48th byte and read 24. This way you can
...read, write, and overwrite data in place within a file.
...'''

...print 'File position after seek ',ParFile.tell()
...# Every open file has a tell method which gives the current position.

...BinaryData=struct.pack(FormatString,'Updated',1,2,3)
...ParFile.write(BinaryData)
...# Same idea as above, just using the changed data (wherever it came from)

...if DataList[0].split('\x00')[0]=='Updated':
......print 'You have seen it all.'
......print 'Remove testbinary.par to start over'
......'''
......There may be a better way to convert this 10 character string to
......what you normally think of as a string, but it comes back padded
......with the 0 bytes and this works simply enough. Think of it this
......way 'Updated\x00\x00\x00' has 3 extra bytes, if you split on that
......character you get back ['Updated', '', '', ''] of which 'Updated'
......is the first or [0] element. Ugly but I've never really look into
......a better way.
......'''

...else:
......print 'Program run with an existing file'
......print 'Data was updated run again to see change'
......# 2nd run string doesn't equal 'Updated' so print appropriate message.

...ParFile.close()
...# Good form to explicitely close files.

I need to wrap it up for now, I'll have to save the parsing command for next time.

Life Outside Vr 4

We've looked at external files with the goal of getting useful information out of them. At the same time how about saving something there. We use a program that takes symbols in a certain layer, compares their elevation to an active dtm, and if they are over a specified distance from the elevation displays the error. It is a very useful program and when it was first written had all the variables hard coded, but stored at the beginning of the file so they could be easily found and changed. With the advent of PromBox it just became too easy to allow the user to confirm and change the variables each time. You could even easily use the old hard coded values as defaults. However it seemed like a certain set of values would be used over and over again based on job specifications. The obvious solution would be to have the last values used stored between runs to be used as the new defaults. Storing values like this is easy enough, but what elements would the most elegant solution contain? I have no idea but I did come up with a couple of features I really wanted.

First lets store them in a logical location like /vr/hostdir where all the other function parameter files are stored. Truthfully it would probably be better to make a user parameter directory, but I'll be lazy for now because the next thing I wanted was a test to see if the parameter file existed already and if not, would create one with the original default values. Yes I could test for a folder and if it wasn't there create it but maybe that will come in the next version but for now I didn't really want to change the directory structure.

import os
# os is necessary to test the existence of the parameter file.

ParFileName='c:/vr/hostdir/save_params.par'
# Call it what you want, this is great code to save for later use in other
# programs that require persistence.

if not os.path.isfile(ParFileName): # Does the file exist.
open(ParFileName,'w').write('%d %.3f %.3f'%(86,.2,0.0))
# If not then create it in write mode and write an integer and 2 floating
# point numbers with 3 decimal places.

Params=open(ParFileName,'r').read().split()
# Since it either existed or was created all we need to do now is open, read
# and split the values into a list that would look like ['86','.2','0.0']

PromBox=VrPromBox('Display Sym to Dtm',20,1)
PromBox.AddInt('Layer to Check',int(Params[0]),1,1000)
PromBox.AddDouble('Display delta >',float(Params[1]),2)
PromBox.AddDouble('Text Rotation',float(Params[2]),1)
# Using the values stored in the Params list make a prombox which will allow
# the user to change the values.

if (PromBox.Display(0) == 0):
...LayerToCheck=PromBox.GetInt(0)
...DisplayDelta=PromBox.GetDouble(1)
...TextRot=(PromBox.GetDouble(2))
# Grab the new values.

open(ParFileName,'w').write('%d %.3f %.3f'%(LayerToCheck,DisplayDelta,TextRot))
# And write them out to a file.

I won't spend the time on all the uses for keeping persistent variables between sessions, but this is the basics.

This just about wraps it up for reading and writing files, except for maybe writing parameters as a binary file, just for the fun of it and maybe walking through the single line parser I posted on the forum. It isn't very flexible but offers some interesting lessons. Here's a peek.

alist=[]
for line in open('hughes.pxyzopk','r').readlines():Alist.append(line.split())

Life Outside Vr 3 (well not very far outside)

Actually not at all, but the point isn't that we need to work outside Vr, it is that there is a whole world of files available using some straight python functionality. Now that we have a little file opening and reading under our belts, lets come back into Vr and bring a little of that technique with us. Let's also work on tightening up the code a little. If you are fairly new to VrPython, I think you'll really like this one. We are going to let Vr tell us the name of the file to open (in this case the current function key file) then using standard python we'll open, read, and parse the data we want out of the file, then use Vr to build a prompt box. For this example the data we want is all the function key names, and the prompt box will allow us to pick a function key name. Why? Well how about if you were going to write a parallel function using the existing inspar command but wanted to change the PARMOD= to "Function Key" and wanted to select the function key from a list then send it using the FKEY= key-in. If so, here is one way to get the function key name. It is rather long and convoluted but I'll try to make it worth what you are paying for the look.

Cfig=VrCfg()
# We'll need a Vr config object to access the name of the currently
# loaded function key file name.
FileName=Cfig.GetFkeyFileName()
# Grab the name.
FkeyFile=open(FileName,'r')
# Open the file
FileData=FkeyFile.read()
# We're going to read the file twice. This time as a string so we can count the
# instances of the key-word "KeyName" to figure out how many entries our
# prompt box will need.
NumberOfFunctionKeys=FileData.count('Fkey KeyName')
# Use the string method count to do just that.
PromBox = VrPromBox ("Function Keys", 60, 1)
# Create a PromBox object.
PromBox.AddList ("Select a Function Key",NumberOfFunctionKeys , 40, 0)
# I'll skip helpful text and just get right to the listbox of length equal to
# the number of function keys.
FkeyFile.seek(0,0)
# Rewind the file to read it again.
FileData=FkeyFile.readlines()
# This time read it as a list, each element of which is a line from the file.
for FileLine in FileData:
# For each line in the data list.
..if FileLine.count('Fkey KeyName') != 0:
# Test to see if it is a line containing the function key name.
....FkeyName=FileLine.split()[2]
# If so split the line data into the three words
# 'Fkey','KeyName','ActualName' which is in itself a list, the third element
# of which ( [2] ) is the actual name of the function key.
....PromBox.AddListItem(FkeyName)
# Add that to the list of items to be displayed.
FkeyFile.close()
# Could have closed it after the readlines() but just go ahead and do it now.
if (PromBox.Display(0) == 0):
..FunctionKey=PromBox.GetList (0)
# Standard prompt box usage to grab a list item
print FunctionKey
# and print it out (or use it in an interesting way)

Now as far as tightening it up. I could have read the file only once and built a for loop to count the instances of the KeyName key-word, but I'm going to go in a totally different direction. I'm still going to read the file twice, as a matter of fact I'm going to open it twice. The goal of this exercise is to get as few lines of code as I can. I'm not going to profess that this is the tightest and most efficient. It just has a lot less lines of code and is intended to make you think. Look at it first, then read the explanation.

PromBox = VrPromBox ("Function Keys", 60, 1)
PromBox.AddList ("Select a Function Key", open(VrCfg().GetFkeyFileName (),'r').read().count('Fkey KeyName'), 40, 0)
for FileLine in open(VrCfg().GetFkeyFileName (),'r').readlines():
..if FileLine.count('Fkey KeyName'):PromBox.AddListItem(FileLine.split()[2])
if (PromBox.Display(0) == 0):print PromBox.GetList (0)


Line 1: We have to create a prompt box object because I can't find any way around it and we'll need to use methods of this specific object in the next line.

Line 2: Let's take this a step at a time.
A) Add to the list using the formula AddList (Prompt, NumItems, Height, DefaultItem) Prompt is set to "Select a Function Key", NumItems is the tricky part, Height is 40 and DefaultItem is 0.
B) Look at the NumItems formula and read it as follows
Open a file using the string returned by a VrCfig() object method GetFkeyFileName in read mode ( 'r' ) which open file object you should read() the data, of which data you should count() the number of occurrences of 'Fkey KeyName'. Notice that when all this is said and done it just give a number which represents the number of function key names and therefore the number of function keys

Now that we have a prompt box which contains a list set to the length equal to the number of function key names, let's populate the list with those names.

Line 3: For each element in a list, the list comes from opening the same file again, in read mode again, but this time instead of read() to get one solid string, we use a method that will return a list which is exactly what the for loop is working on. We are looping on a list that was created and populated just for the loop, each element of which is a line from that file.

Line 4: FileLine represents the line of data out of the file which is being examined as we loop over the entire list of lines. count() tells how many occurrences of a work exist in the string 0 means none which in terms of an if means false so any other number that comes back means true (the string exists, therefore it is a function key name). if it is true we want to add the actual name to the list. The actual name as we saw above means splitting the string, which creates a list, from which we want element [2]. So add to the list the third word which is the result of splitting the string because it does (true) contain the phrase 'Fkey KeyName'. I didn't add another line and indent it because there is only one statement to execute if true which allows you to just put it after the colon and be done with it.

Line 5: Same idea, standard prombox syntax, but since there is only one action, just put it on the same line. Ugly code but remember my only goal was to reduce the line count. And why put the string in a variable just to print it on the next line, print it as it returns from GetList()

And I haven't even taken the time to explain the forum post regarding reading and parsing aerosys .pxyzopk files, a great example of the kind of data we mappers work with all the time. Maybe next time.