Thursday, December 28, 2006

Simple task offers many examples

Let's look at a very simple task that will offer a wealth of examples of using various Vr modules and tasks. This will go beyond the idea of using as a simple macro language into the early stages of writing what I consider true programs that extend Vr functionality and will include some basic ideas as well as some more advanced concepts. This example will include...

  • using python modules (non-vr straight python)
  • selecting entities
  • creating new entities
  • displaying information in various forms
  • and grabbing window attributes.
The problem is that there are several lines in the file for which the area is required, along with the total of all areas. It would be nice to allow the user to ID as many lines as necessary, display the area for each line, then present the total at the end. We'll throw in some variations at the end that make for interesting exercises, and more useful programs.

Here is the code....

import math

Gui = PyVrGui()
Ws=PyVrWs()
Gr=PyVrGr()
Line=PyVrLine()
TotalArea=0.0
NumberOfLines=0
text=PyVrText()
text.SetFontNum(4)
text.SetRot(45.0)
text.SetSize(Gr.GetWinScale ()*.15)

WsNum=Ws.Aws()
while Line.Id() != -1:
..TotalArea=TotalArea+math.fabs(Line.Area())
..NumberOfLines=NumberOfLines+1
..print TotalArea
..label='%.1f ac' % math.fabs((Line.Area()/43560.0))
..text.SetText(label)
..(MinX, MinY, MinZ, MaxX, MaxY, MaxZ) = Ws.GetLineMinMax (WsNum, Line.GetLineNum())
..text.SetCoord((MinX+MaxX)/2,(MinY+MaxY)/2,(MinZ+MaxZ)/2)
..text.SetJustX(2)
..text.SetJustY(2)
..text.SetLayer(88)
..text.Plot()


print 'Total Area ',TotalArea,' Sq Units ',TotalArea/43560.0,' acres'
Gui.MsgBox('%d Lines contain %.2f acres'%(NumberOfLines,(TotalArea/43560.0)),'Area by Selection')

Taken a line or two at a time


import math
We are going to use a purely math function later. It would be easy enough to write the function needed (math.fabs() to get the absolute value of a real number), but this illustrates why it is nice to have python installed even though it isn't strictly necessary for VrPython to work, there are times that accessing standard (or very exotic) modules that are already available can be nice.

Gui = PyVrGui()
Ws=PyVrWs()
Gr=PyVrGr()
Line=PyVrLine()
text=PyVrText()
text.SetFontNum(4)
text.SetRot(45.0)
text.SetSize(Gr.GetWinScale ()*.15)
text.SetJustX(2)
text.SetJustY(2)
text.SetLayer(88)
TotalArea=0.0
NumberOfLines=0
First create or initialize all the Vr objects that will be used (first 5 lines), pre-set any entity parameters that the user may want quick access to later (lines 6-11), and initialize variables that need to start with a pre-determined value.
In this case Gui will be used to display a message box, I always initialize Ws out of habit, Gr will give us access to the Vr graphics window, Line will be used to ID line entities, and text will be used to display the areas on the screen in the graphics window as text entities. We'll discuss all these as we use them but text values are declared early and in the variable area I put at the beginning because they directly affect how the text displays and the user may want to change the appearance. The one important thing to notice here is the use of the Gr.GetWinScale to grab the map scale of the current graphics window which is then multiplied by the size (in inches) I want it to appear on the screen since the text object expects size in ground units, also note that SetSize is used rather than setting height and width independently


WsNum=Ws.Aws()
I just always throw this in to make sure we all know what workspace is active.

while Line.Id() != -1:
The line object has a function called Id which not only contains all the overhead of allowing the user to identify a line in the graphics window complete with button dialog, but also returns with that line loaded and ready to work on.

..TotalArea=TotalArea+math.fabs(Line.Area())
..NumberOfLines=NumberOfLines+1
The names give away the purpose, and each time a line is selected we bump the accumulated area and the line count. The reason these were initialized to zeros before is that we want to make sure they start from there.

..print TotalArea
Sometimes I just like to send things to the console screen as a debugging tool or so there is something of a temporary record.

..label='%.1f ac' % math.fabs((Line.Area()/43560.0))
..text.SetText(label)
For this program I decided to display individual line areas in the Vr graphics window as text entities so the first thing that needs to be done is setting the text to the area of the line. Here is where the math module is used to make sure the area is positive. The single quote marks and % symbols convert the floating point number returned to a formatted text string with only one decimal place. If I wasn't worried about the formatting I could also use the python repr() function to simply change the number to a string because the SetText function expects to be handed a string. These two lines could also be easily written as one line but that makes it a little more obscure for this example.

..(MinX, MinY, MinZ, MaxX, MaxY, MaxZ) = Ws.GetLineMinMax (WsNum, Line.GetLineNum())
Since the text needs to be place somewhere on the screen, I decide to just place it in the middle of the lines extents. In order to do that we need the Mins, and Maxs. Those values are stored with the line header in the workspace so within the workspace object Ws we use the GetLineMinMax on WsNum (which we set to the current earlier) to pull the data for the entity number of the currently loaded line.

..text.SetCoord((MinX+MaxX)/2,(MinY+MaxY)/2,(MinZ+MaxZ)/2)
Once we have the extents just take the average and set the text to be displayed at those coordinates.

..text.Plot()
The have Vr plot the text entity on the screen. Note that there is no save done on the text so if the screen is refreshed or overwritten it will disappear.

print 'Total Area ',TotalArea,' Sq Units ',TotalArea/43560.0,' acres'
When the selection is all done (note that we are out of the indented block) print various numbers and text to the console screen for the same reasons listed above..

Gui.MsgBox('%d Lines contain %.2f acres'%(NumberOfLines,(TotalArea/43560.0)),'Area by Selection')
More importantly display it all in a message box that will come up nicely formatted and with an "OK" button to dismiss.

Variations include:
Prompting for a layer number and just summing the areas for all lines in a given layer rather than selecting them.

As the lines are selected, how about changing some graphic attributes (again don't save anything) that make the line stand out so it is obvious which lines have been ID'd.

As lines are computed the individual areas along with some identifying info (entity number, or feature code) could be written to a file followed by the total.

Fix it so that if the user identifies the same line multiple times it only gets added to the total once. (caution geeky hint: I would probably use the entity number, a list, and the list.count function)

Tuesday, December 19, 2006

Quickie tip

Already posted for the week but found a forum that shows quite a bit of promise. It is at TheScripts developer network, and looks pretty interesting.

Python as macro revisited

In the first VrPython example I gave which has now take on it's official name of fix_sym_z.py my purpose was to show how it is possible to get some working code very fast just using some basic Vr keystrokes. I very rarely use the vr button (B1, B2, B# etc) commands to actually push the buttons, but it makes a point. The real problem with that script in a public setting is the drive point functionality that wasn't explained, not to mention that the person using it had several key suggestions to make it much more useful (don't end users drive you crazy like that sometimes). In any case, let's look at the current evolution of the script and the positive and negative effects of the changes. While I'm still going to use the button press commands at the end to actually do the work, by necessity this will start looking more like a python script and less like a simple extended macro, but then that's the whole point.
Remember the dots are used for place holders and to visually represent the indentation which is life and death in python (2 dots = 1 tab)

Gui = PyVrGui()
stat=0
Layer2Drive=25
Sym=PyVrSym()
Ws
=PyVrWs()
WsNum=Ws.Aws()

Gui.PushKeyin ("digxyz")
for EntNum in range (Ws.GetSymCount(WsNum)):
..if Layer2Drive==Ws.GetSymLayer(WsNum,EntNum):
....Sym.Load (WsNum, EntNum)
....x,y,z=Sym.GetCoord()
....Gui.PushKeyin ('xyz '+repr(x)+' '+repr(y)+' +0')
....stat, x, y, z = Gui.GetCoord ('Select Coordinate')
....if stat==0:
......Gui.PushKeyin ("chaele")
......Gui.PushKeyin ("sealin=0")
......Gui.PushKeyin ("seasym=1")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b#")
......Gui.PushKeyin ("chalay")
......Gui.PushKeyin ("sealin=0")
......Gui.PushKeyin ("seasym=1")
......Gui.PushKeyin ("lay=76")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b#")

Looking at it a line at a time

Gui = PyVrGui()
stat=0
Layer2Drive=25
Sym=PyVrSym()

As before, just setting up some variables. Creating an instance of the Gui object, same for a Symbol, and initializing the others. This is a good place to note that in Python you don't actually have to declare variables, or initialize them in any particular place. I could have put the Layer2Drive=25 on the line right before the first place I used it, or if it were the result of a calculation, just let it be created in place. I am only setting them up early like this because I want to know what they are set to, and putting it all together seems like sound practice. In the first pass at this script the external "drv2lay" script prompted for the layer, but in reality the user just wanted it to always use the same layer. I could have used the desired number rather than a variable later, but using a variable and setting it at the outset makes it easy for other users to find and change if they want.

Ws=PyVrWs()
WsNum=Ws.Aws()

Gui.PushKeyin ("digxyz")
Because Vr allows several workspaces to be open at the same time and in many cases you have to specify which one you are interacting with you need to create an instance of a workspace object, and in this case set it to the currently active workspace. Then let's make sure that the stereo digitizer is active before we start driving to points.

for EntNum in range (Ws.GetSymCount(WsNum)):
..
..if Layer2Drive==Ws.GetSymLayer(WsNum,EntNum):
This is going to represent one of the biggest changes to this script, driving to the points will actually become part of the script, but before we can drive, they have to be identified. The for loop in Python is fantastic and I won't take time to explain it now (remember the books recommended in the introduction). Let's just say that in plain English these two loops (the for and the if) would sound like "For entity numbers in the range zero through how ever many symbols there are, if the desired layer to drive to is equal to the layer of the symbol with the entity number currently being checked". Yes we are going to loop over the entire file and check the layer of each symbol, the power behind this loop is the way the checking is done which I won't really go into, but suffice it to say it seems instant on even the largest files.

....Sym.Load (WsNum, EntNum)
....x,y,z=Sym.GetCoord()
At this point if the symbol does match the layer we plan to drive to, load the symbol based on the workspace we are working in, and the entity number of the symbol currently being checked. Then from the loaded symbol grab the coordinates. Note that since GetCoord returns 3 numbers, I can assign all 3 at the same time. This is also an example of assigning data to a variable that has been neither declared nor initialized. It becomes a floating point variable because that's what I assigned to it.

....Gui.PushKeyin ('xyz '+repr(x)+' '+repr(y)+' +0')
Once I have the coordinates I can drive the instrument using the Vr command 'xyz' then supplying an x, a y, and the +0 just means rather than an absolute z use one relative to the current position and add zero. Since PushKeyin expects a string we have to construct one using a text+text method. In this case since x and y are numbers, they have to be converted to text before they can be added to the string using the Python repr() command. There are some extra spaces inserted after the xyz and before the +0 to keep things separated, so if x was 1.0, y was 2.0 the final string pushed to vr would be.
xyz 1.0 2.0 +0

....stat, x, y, z = Gui.GetCoord ('Select Coordinate')
....if stat==0:
Once the appropriate symbol has been found, and the instrument has been positioned there, use the same method as before to pause for the operator to put the dot on the ground and digitize the elevation at that point. The main difference from before is that now if the user digitized a point all the changes will be made. If they hit end, rather than abandoning the whole script, it simply doesn't change anything and continues looking for symbols that match the layer criteria. There really isn't any way to abandon the script, but in this case the user wanted it this way.

......Gui.PushKeyin ("chaele")
......Gui.PushKeyin ("sealin=0")
......Gui.PushKeyin ("seasym=1")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b#")
......Gui.PushKeyin ("chalay")
......Gui.PushKeyin ("sealin=0")
......Gui.PushKeyin ("seasym=1")
......Gui.PushKeyin ("lay=76")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b1")
......Gui.PushKeyin ("b#")
Same basic set of changes, except that before we waited until the elevations had been changed, then globally changed the layer to put it in the dtm data set. Now since if there is no point digitized, no changes are made, we can put the layer change right into the symbol modifications which includes everything including and after the chalay line. Note that I hard coded the layer to 76. As I write this I'll probably change that to a variable for the same reasons I mentioned for the layer to check. The parameter for the PushKeyin will be something like..
("lay="+ChangeToLayer)
There won't be a repr() command because I'll just store the variable as a string in the first place, something like...
ChangeToLayer='76'

All in all still much more brute force than elegance, but now we are doing some basic looping over entities including loading and pulling information from them.

Friday, December 15, 2006

I really only plan posting about once a week. I've already decided that the next "real" post will work on cleaning up the Vr script (which has come to be known as fix_sym_z) I mentioned earlier to make it much more useful and fully independent of the drive functions. However this quick one is just too cool to wait. But just in case the corpscon stuff is totally worthless to many folks who plan to write python in Vr, let me pass along one of my favorite tips, as pasted from one of my utility modules (dsutil). While the comment sections explain what it does, you may want to look up lambda, though it isn't necessary. If you understand printf, you understand this function.

printf = lambda fmt,*args: sys.stdout.write(fmt%args)
'''
c like implemetation of printf, for example:
dsutil.printf("this is %s test with the number %.2lf",'a',9)
yields...
this is a test with the number 9.00
'''

fprintf = lambda afile,fmt,*args: afile.write(fmt%args)
'''
Same as printf but like c's fprintf sends output to open file

usage: dsutil.fprintf(AnOpenFile,"format",Values)
'''

Tom Blankenship is pretty high in my relatively short list of amazing programmers I actually know and can glean information from. As usual, if it is a matter of education on coding practices, Tom was fast to weigh in. The corpscondll is pretty well documented but my problem was that I didn't know (and didn't know that I didn't know) how to pass a c style double pointer using c_double(). Tom put together a working script that has all the information necessary to incorporate corpscon transfomations into production programs, is this stuff amazing or what.

As always a big thank you and hats off to Tom.

Thursday, December 14, 2006

There is a patch up that includes python 2.5. I have installed it and want to make a quick report.
The main implications include the facts that I...

  • Rely heavily on standard imported libraries.
  • Have lots of my personal modules containing generic utilities.
  • Program everything in python and need to compile exe files in many cases.
  • Utilize several open source libraries like PIL, pyserial, etc.
Before I installed the patch, I un-installed all python 2.2 libraries as well as python itself. I then installed the patch and played with vr a little to make sure that everything worked the way it should. Next I installed python 2.5 and my most used extra libraries (primarily pywin32, PIL, py2exe, pyserial). So far everything is working out fine except that I couldn't easily get py2exe to work the way it always did. I'll admit that I didn't spend a huge amount of time and it is my problem, I just got frustrated trying to figure out how to configure everything to eliminate the message "LoadLibrary(pythondll) failed" when running a program without the full path. It works fine on simple programs, but there is apparently something in the way I code that is causing problems.

While screwing around I found that pyinstaller worked fine for what I was doing and I was able to get it running in just a few minutes, so I'm going to migrate there for now.

So far I'm quite pleased with the results of the upgrade. I've got as much weird stuff going on as most people so if it works here, I would think it would work for most anyone and I would suggest going for it.

No major earth shattering changes but I like being on the latest python in case I find new libs I would like to use. I'll also toss out that in 2.5 ctypes is included in the standard install. My hope would be that this would make it possible for someone who knew what they were doing to use external DLLs. So if you know of a way to use the corpscon DLL to write a coordinate transformation function within program, I would love to hear from you.

Monday, December 11, 2006

I'll come back to learning python but thought I would toss in a little Vr as well. Had a situation where a colleague wanted to perform a fairly simple and repetitive task. When it came right down to it Vr already had most of the tools he needed, I thought about stringing it together into a macro for him, but realized that some conditional statements would add a little zip to the solution. In the end the solution is a good example of just using python as an extended macro language. Nothing earth shattering, just using basic Vr commands, but putting a little of the power of python behind them.

The situation was that he wanted to be able to quickly and visually throw points into a file in a random pattern using the mouse. Then to make these same points part of a valid 3d data set he wanted to have the instrument drive back to the points, pause for the dot to be placed on the ground, set the elevation of the symbol to the current instrument position, then go on to the next point without having to issue any command other than ending the id process.

The outline of the solution goes as follows.

  1. Digitize the symbols in a layer that makes them unique.
  2. Capture the xyz locations of those symbols by looping over the points in the file.
  3. Loop over the locations with a process that
    1. Pauses to digitize a new z at each location
    2. Set the symbol to that elevation.
    3. Drive to the next point.
  4. When all done globally change the symbols to make them part of the normal data set.
I'll address a couple of the redundant or wasteful portions of the solution later, but to some extent this will also expose that fact that sometimes I write ugly code just to get something out for testing in a fairly short time, I regularly go back when things are slow and tighten up procedures. This one definitely qualifies as one of the ugly ones, but it's still a good example. Keep in mind that to this point I haven't found a way to make the proper indentation persistant in this blog so until further notice I'll use periods to fill the space where indentation belongs. Lets look at the code line by line...

Gui = PyVrGui()
stat=0

Gui.PushKeyin ("py drv2lay")
Gui.PushKeyin ("digxyz")
while stat==0:
..Gui.PushKeyin ("dri")
..stat, x, y, z = Gui.GetCoord ('Select Coordinate')
..if stat==0:
....Gui.PushKeyin ("chaele")
....Gui.PushKeyin ("sealin=0")
....Gui.PushKeyin ("seasym=1")
....Gui.PushKeyin ("b1")
....Gui.PushKeyin ("b1")
....Gui.PushKeyin ("b#")


Gui = PyVrGui()
This one just imports the appropriate library that will do most of the work in this script.

stat=0
The routine used to grab the current instrument position returns a list which includes the process status to evaluate whether the used in fact digitized a point or just hit end.

Gui.PushKeyin ("py drv2lay")
This is the most obscure part of the code. I already have a script that asks the user for a layer number (or number line), creates a drive file containing a drive point for every symbol in that layer, then opens the drive file. I will discuss that program in a later post but for now any function that grabs the xyz values for every symbol in a pre-determined layer would be fine. I chose to go the drive route so the values would be stored temporarily, and I could just use the Vr "dri" function later. "PushKeyin" just sends the string in quotes to the Vr command line as if the user had typed it

Gui.PushKeyin ("digxyz")
Because the first time someone used it, they started the program while still set to the mouse which had been used to digitize the symbol locations, as well they should. Just need a toggle it over if it hasn't already been done.

while stat==0:
Python flow control is fairly standard, In this case if the user hits "End" rather than digitizing the new elevation at the symbol position the routine would return a 1, so if this happens "game over".

Gui.PushKeyin ("dri")
In the actual work part of the loop the first thing we want to do is drive to the next valid coordinate position, remember that there is already a drive file open with all the positions I want because of the other python program I called earlier.

stat, x, y, z = Gui.GetCoord ('Select Coordinate')
Once we get to the position, pause and wait for the operator to put the dot on the ground and hit the button sending a status code and especially important the new Z.

if stat==0:
There is probably a more elegant way of doing this, but the fact is that after the user digitizes a point I needed to know if they had hit "End" and if so just skip the rest. If they digitized a new z location I want to do the following using the keyin routine.
chaele - start chaele because that's what I actually want to do.
sealin=0 - make sure that line search is off.
seasym=1 - make sure symbol search is on.
b1 - hit button 1 to identify the symbol at the location where the instrument is sitting.
b1 - hit button 1 to release the point.
b# - hit the # to end chaele and resume the loop which then drives to the next point.

So in this example I'm just using python to create the drive list, and the control flow to loop over it. I could also turn off any layers that aren't necessary to make sure that nothing errant gets identified. It could also have been done in one step without the second loop by not creating the drive file, but simply making a go to xyz statement part of the first loop which identifies the proper points.

Friday, December 08, 2006

I'll admit that when I get new hardware or software I'm the kind that installs and just jumps in with both feet playing and experimenting. I've always thought that it is much better to get into the documentation with at least a basic grasp of what interfaces and such look like. At the same time though I am a huge proponent of structured progressive training. That is really where I'm headed right now with the following recommendations.

For anyone that is already programming in another language I think it is worthwhile to install Python on a computer even though it isn't strictly necessary for VrPython. I actually think it's a good idea for anyone, but if you just want to jump into VrPython, skip this for now you'll probably find other posts more helpful.

As of the time of this writing the current implementation of Python in Vr is 2.2. If that is what is installed then an extension called pywin32 which not only adds windows elements to Python, but also includes an editor and programming environment that I like much better than Idle.

Once you have the current Python that matches whatever is built into Vr, I recommend the following books. Skip any reference to gui programming and tkinter for now.
For basics start with.
"Sams Teach Yourself Python in 24 Hours"
"
The Quick Python Book"
For more advanced reference.
"Python Essential Reference"
"Python Cookbook"
I actually found all of them at my local library, though I like them so much I bought the last 3 to have around as a reference. I would read the first two at the same time or in that order. "Quick Python" was amazing in the information you can pick up fast if you have some programming experience. It is especially helpful for someone who might switch over to Python for routine programming. Pay special attention to the concept of lists, I think they are by far one of the coolest things available in Python, and will offer a lot of power as time goes by.

Why would a person want to switch to Python? Well for me it came down to the fact that I was spending time in it on a daily basis within VrOne anyway. Now with the introduction of the "pyedi" command, coding within Vr is even more convenient. However the more time I spend with it the more powerful and fast coding in this environment seems. I decided to try rewriting some basic file manipulation programs and found that the lines of code dropped to about %25 of the original once I switched from something like C++. The more I tried, the less restrictions I found, for example by adding libraries like PIL (Python Image Library) I was able to write a program that applies an external level file to an image. Sure I have that in ImageUtil, but I had mine a year earlier, and on every computer I own. So programming in general, and certainly Python, isn't for everyone, but it is a great place to start for beginners, and pure fun for those more experienced, in my opinion anyway.

Wednesday, December 06, 2006

By way of introduction let me just mention that I have been working with Vr or it's noble predecessor since 1989. While I always have suggestions on how it could be improved, I am not going to apologize for how much I like this software, or the fact that at times I may sound like too much of a cheerleader for it. I have no relationship with Vr other than satisfied customer, and have nothing to gain from the time spent here other than providing an outlet for my enthusiasm for programming and making maps. If anyone has some good tips or information that get passed along to me then that would just be a wonderful plus.
My first introduction to computers as a working tools were HP-UX workstations, collecting data in CADMAP, and programming in C. Over the years I have had the fortune to migrate to C++ and now to Python. I have never been particularly clever or ambitious in my programming, but simply see it as a tool to take the task at hand, and either make it possible, or easier. I have the blessing to work in an environment that encourages me to think, code, and experiment. Time spent learning and improving practices is considered time well spent. Thankfully most of the limited knowledge I do have about good programming flow and practices came from early discussions with Mike Kitaif who ranks up in the top handful of the sharpest people I personally know. I will go ahead and credit any good looking code I happen to come up with to him, though to be honest, most of it is just throwing something together to get a job done.
My thoughts at the moment are to ramble a little about Python, VrPython, and Vr in general. I am a huge fan of stretching the capabilities of Vr with Python. I may mention some of the 100+ tools that we have developed for specific tasks and need to be careful to say that I don't consider this to be an indication of anything Vr is lacking, rather that while offering one of the most user friendly, practical, powerful data collection environments "out of the box", it goes beyond that to offer a potential for power and customization that is truly incredible. Not everyone would like the tools I like, but the beauty is that if not, they can create their own, and if they do the same tools are only minutes away with a few lines of Python.
I have over time have begun to do all my coding in Python and also hope to share some of the ways it can be useful to folks to go ahead and install the full system and find uses for it.

For anyone interested in trying VrPython for the first time or if you are early in the game, I suggest going to the earliest posts and working forward. I use VrPython every day for many wonderful things, needless to say it will change and could potentially damage a file. Any risk associated with using VrPython or any code or scripts mentioned here lies solely with the end user.

The "Personal VrPython page" in the link section will contain many code examples and an organized table of contents to this blog in a fairly un-attractive (for now) form.