Convert DEM to xyz for Visitor 3 Using MicroDEM

[DC39]

Hello all!

I'm following the ZGM Arma 2 Map Making Tutorial (can be found on armaholic but I can't post links yet) in my first attempt at a terrain project for Arma 2.
I haven't gotten very far into the tutorial as it says to use 'Global Mapper' to convert my DEM file to xyz, but Global Mapper isn't free and I can't find a trial key for it, so instead I am using MicroDEM.
This has presented a big problem because I haven't been able to find any in depth guides for the program relating to Arma 2 and Visitor 3 compatibility - yes I HAVE searched the forums, and armaholic, and google... lol

SO, basically I have a DEM file, which I know how to open in MicroDEM, and I think I've worked out how to crop (or 'subset') the area of terrain that I want. (But I'm not sure about that :/)

My first question is how can I crop or subset part of my DEM at a specific size? I want to make a map 15km by 15km, I have the long & lat co-ords for the area I want to use from my DEM but not sure how to get an exact size - which I think is necessary for Visitor.

Next i obviously want to export it as an xyz file, so my second question is what settings I need in the export process to make the xyz suitable for Visitor.
Again I've played around with the software and have a vague idea of what to do, but don't know what settings I need to make it visitor compatible, I've tried a few different combos but no luck so far.

When I try to import the xyzs I create with MicroDEM the width and height are not what expected (don't know how to get exact measurements) and my 'square size' is 1.0 instead of 10. (so atm result size = 4096 sqm)

Any help or pointing me towards where I might find a solution would be greatly appreciated.
Thanks! DC

[Bushlurker]

Hi DC39!

Eeeeh..... tricky question!
MicroDEM is a great little program which does all sorts of good things usually only found in the very expensive programs like Global Mapper, but it's not the most obvious program in the world to use at first...

Worse still, I don't actually have it installed right now, nor have I used it in a while, so I'll have to work from memory.....

Hmmm...

OK - first you want to crop - MicroDEM does call this "subsetting" as you've noticed...
There's a couple of ways you can do this... Easiest is just to use the crop tool (probably called the "subset tool" though... it's a wee sort of dotted rectangle icon in the menu bar... just drag a rectangle from Top Left to Bottom Right to define the area you want...

You have coordinates though? That's handy...

There's another option - under the menu item "Modify" - Something like "Map Extents or Areas" and then the "subset corners" or "something corners" option...
You should be able to enter coordinates for that Top Left and Bottom Right pair and it'll come to the same thing - you'll have defined a square...

*Remember Visitor only likes square heightmaps!!

Assuming you've defined the actual area you want successfully - export it as a file.....

Don't worry about scale or cell sizes, or even .xyz format at this stage..... MicroDEM outputs a fair number of different formats - none of which I can remember

Look for .BT (Binary Terrain) - use that one if it's there... if not, maybe .HFZ or .HF2? .TIF (Geotiff), if necessary, I guess...

Next... go HERE and grab a Free copy of L3DT Standard Edition.... this is the best free heightmap editor around, and you'll probably want to come back and play with it again sometime, but right now we'll just use it as a file converter...

Load your saved heightfield - it should display as a greyscale image.... Now go to the top menu...

"Operations/Heightfield/Change Horizontal Scale" - set this to 7.50 meters (this is your "cellsize")

Then...

"Operations/Heightfield/Resize Heightfield" - set this to 2048

That should produce a 2048x2048x7.5m heightfield - Same size as Chernarus & Takistan... approx 15.3km x 15.3km

Now go to "File\Export\Export Active Map Layer" and export the heightmap as .xyz


Assuming I've remembered the MicroDEM bit OK, that should get you started at least.....


B

[lecholas]

Hi DC39!

Eeeeh..... tricky question!
MicroDEM is a great little program which does all sorts of good things usually only found in the very expensive programs like Global Mapper, but it's not the most obvious program in the world to use at first...

Worse still, I don't actually have it installed right now, nor have I used it in a while, so I'll have to work from memory.....

Hmmm...

OK - first you want to crop - MicroDEM does call this "subsetting" as you've noticed...
There's a couple of ways you can do this... Easiest is just to use the crop tool (probably called the "subset tool" though... it's a wee sort of dotted rectangle icon in the menu bar... just drag a rectangle from Top Left to Bottom Right to define the area you want...

You have coordinates though? That's handy...

There's another option - under the menu item "Modify" - Something like "Map Extents or Areas" and then the "subset corners" or "something corners" option...
You should be able to enter coordinates for that Top Left and Bottom Right pair and it'll come to the same thing - you'll have defined a square...

*Remember Visitor only likes square heightmaps!!

Assuming you've defined the actual area you want successfully - export it as a file.....

Don't worry about scale or cell sizes, or even .xyz format at this stage..... MicroDEM outputs a fair number of different formats - none of which I can remember

Look for .BT (Binary Terrain) - use that one if it's there... if not, maybe .HFZ or .HF2? .TIF (Geotiff), if necessary, I guess...

Next... go HERE and grab a Free copy of L3DT Standard Edition.... this is the best free heightmap editor around, and you'll probably want to come back and play with it again sometime, but right now we'll just use it as a file converter...

Load your saved heightfield - it should display as a greyscale image.... Now go to the top menu...

"Operations/Heightfield/Change Horizontal Scale" - set this to 7.50 meters (this is your "cellsize")

Then...

"Operations/Heightfield/Resize Heightfield" - set this to 2048

That should produce a 2048x2048x7.5m heightfield - Same size as Chernarus & Takistan... approx 15.3km x 15.3km

Now go to "File\Export\Export Active Map Layer" and export the heightmap as .xyz


Assuming I've remembered the MicroDEM bit OK, that should get you started at least.....


B

I don't want to overcomplicate things, but...
Aren't Aster DEMs in an unprojected geographic coordinate system (WGS84)? If yes, wouldn't that mean that cutting a square raster (ie. with even pixel size edges, eg. 2048x2048) from it would result in a representation of non-square-like area?

Edit: Ok, I've glanced through the ZGM's tutorial and looks like it's covered there for Global Mapper. One thing to remember when you use Microdem instead of Global Mapper is to define a projection in Microdem as well (I suppose it allows for that - if not, I'd suggest using for example opensource QGIS).

[DC39]

Bushlurker:
So I found the corners bit you were talking about, in my version of MicroDEM; 'Modify' - 'Map Area' - 'Keyboard Corners'

and then when I got to export the only option I can find that you listed is .TIF - options are save as image - (colour or grayscale geotiff) & save as DEM - ASCII XYZ, DTED, USGS ASCII, BIL, ASCII Arc Grid, Grid Float, and then a list of options under 'Caveat Emptor', so I've got as far as exporting my Height map DEM as a smaller Geotiff image.

I think I need to look through the l3dt instructions a bit more because there is a baffling array of options for projects etc which look very interesting - I'll get back to you properly when I've worked out how to convert, as at the moment it says the area I want to use as a height map is too big for the free version.

Thanks for the advice, I think using L3DT should make things easier, I'll give a full description of what I did when I get it done right so others searching the forums will have a little guide.

Lecholas:

Not sure quite what you mean by 'define a projection' but I've downloaded the Opensource Quantum GIS and I'm having a look at that next, thanks/

Cheers guys, get back to you in a couple days with final thoughts, sorry I took so long to reply - hectic work schedule this month

DC

[Rocket]

Have you tried Global Mapper? I think they have a trial version. I used it in conjuction with L3DT on several occasions. Also, Aaron at L3DT is a great guy and very easy to deal with if you have questions. This makes exporting in different projections (to grayscale and XYZ) extremely easy straight to Visitor 3. It might be worth a shot, I found it much easier to use than MicroDEM and used it at my old work alot.

[DC39]

Hi Rocket, yes I have tried Global Mapper, it's the program advised in ZGM's tut so it was my first stop - trouble is he uses an older version than I could download, and AFAIK the new version wont let you save or export ANYTHING, quite annoying really, which is why I've been trying out other soft. like microDEM. Maybe you know a way around this using L3DT? I haven't really had a chance to use either yet, I'm VERY new to the whole mapping thing, still getting to grips with how the directories and tools work.

[Rocket]

You can export grayscale heightmaps using MicroDEM (I think as a GeoTIFF, but its been a while since I used MicroDEM). Have you tried importing into Visitor with a grayscale map? I wouldn't have a pipeline direct from MicroDEM to Visitor as the image will need some cropping, levels editing, etc... to get it correct. It will probably require some cleanup also.

A second option is to create the terrain in L3DT and then export as a grayscale image, couple with the correct details in a pbl file, and then import directly into Visitor that way. To achieve the exact effects I wanted, I once was importing/exporting between Global Mapper, L3DT, photoshop, back to L3DT, then into visitor (for my Aoraki map). As always I think the easiest thing is to start simple, then make small steps. See if you can get a very simple, small, grayscale heightmap imported into visitor no problems. Then try something a little bigger once you've got a good feel for the process.

[Mondkalb]

Creating_a_Heightmap with L3DT and saving for V3

[lecholas]

Creating_a_Heightmap with L3DT and saving for V3

Well, that's a way for creating a heightmap (and what is more, for randomly creating it). I think that the thing that DC39 wants to do is to use an existing real-world data from ASTER DEM.

Lecholas:

Not sure quite what you mean by 'define a projection' but I've downloaded the Opensource Quantum GIS and I'm having a look at that next, thanks/

Cheers guys, get back to you in a couple days with final thoughts, sorry I took so long to reply - hectic work schedule this month

DC

A projection is a cartographic method for unfolding or unwrapping an ellipsoidal approximation of Earth's surface into a flat plane. There are many ellipsoids (one of which, the most common thanks to it being used by a GPS, is WGS84) and for every elipsoid there are many projections, global or local (eg. UTM zones, older soviet pulkovo1942 and many, many national projections (or 'grids') as for example Polish PUWG1992. The thing here is to remember that geographic degrees are not flat plane coordinates. They must be projected onto a plane in one way or another. For most Arma purposes I'd assume that UTM projection with a correct zone would be a good choice. Unless you're making a map based on a historical scanned map which is in a different coordinate system (or you have other reasons) as I am at the moment.

I'm personally using QGIS combined with other software (GIMP, L3DT, Willbur). I'm using my hand-made DEM (from contour lines from a historical map), but if I were to use an ASTER DEM my workflow would be as follows:
- download and install QGIS; open it and go to Plugins->Download more Plugins (the names of menu items here and below can differ as I'm using a localized Polish verision of QGIS) and find and install 'Rectangles, ovals digitizing' plugin
- download a for the desired area DEM (I assume you already know how to do it)
- load it into QGIS
- go to Settings->Project settings and tick 'project on the fly' and change the projection to the one which is suitable for you (eg. UTM with the correct zone)
- convert the DEM layer into proper raster (e.g. GEOTiff), remember to tick the option named something like 'Add to layers after finished' (not sure if that's required)
- reproject the resulting raster to your desired spatial reference system (i.e. projection, eg. UTM), Raster->Change Projection; settings: source srs: WGS84, target srs: UTM with your zone, for your purposes 'Nearest neighbour' method of resampling should be good but you may try the remaining ones; again check 'Add to layers after finished'
- create a new shapefile: Layer->Create new layer->Shapefile; make sure it's polygon-shapefile and that the srs (projection) is set to your desired projection (UTM); don't worry about adding any attributes apart from the default FID
- create a new polygon in your new shapefile: select the layer, click 'Edit mode' (a button with a pen or something), then 'Add polygon', and then, using 'Rectangles, ovals digitizing' plugin draw a 'Square from the middle' (AFAIK there is no, more suitable for us, 'Square from the edge' option in the plugin yet) that will cover your desired area (if you're planning for example a 10240x10240 meters map the side of the square should be 10240m - could be tricky with QGIS, I took a shortcut and created the square shapefile in ArcGIS to be sure its side is as close to 10240m as you can get; in QGIS you can for example use the measuring tools, or maybe there are some plugins that I'm not aware of which serve that purpose; or find some other tool just for a purpose of creating a shapefile with a square polygon of the desired dimensions and importing it into QGIS - on thing to remember that such a shapefile should be natively in the target srs, in our case UTM). This will be our cake-cutter layer for cutting all the different layers we ever need (e.g. Sat mask; if you could acquire a land cover layer for the area you could then cut it with the cake-cutter and get a mask for Sheazan74's brilaint World Tools for generation of randomly placed vegetation that way); stop editing (the pen button) and save changes to the layer
- then, at last, clip your DEM raster with the cake-cutter layer: Raster->Clip raster; select the name of your source projected DEM raster and a name and destination for the clipped raster

You should receive a greyscale DEM raster corresponding to the desired area. You can resample it to the desired resolution (eg. 1024x1024) in any program for digital graphics (eg. IrfanView) and you should be able to load it either in visitor or in L3DT. One thing you need to check is to check the min and max pixel values which should correspond to minimal and maximal heights in the covered area and which you should put into right places in Visitor (pbl file) or L3DT - the form when importing the heightmap. But that's a different story.

I may be not precise in many points but I don't have time now for writing a comprehensive tutorial. Maybe some time later if there is some interest. It's not the prettiest method in the world but it works. If you have any questions feel free to PM me and I'll answer if I can.

[Homer Johnston]

I haven't fiddled with terrains in years, but back in ancient times I always used 3DEM to convert a GeoTIFF to a terragen .ter, and then loaded that into Wilbur & converted again to a 16 bit png. Took me a while to figure out how to do it, but it always worked just fine for me can't recall how I cropped an area, though.