Structure from Motion Point Clouds to Real World Coordinates

| 0 Comments | 0 TrackBacks
This post describes one method to convert a structure from motion point cloud (Photosynth and Bundler) into real world coordinates using a set of mapped controls. The technique hinges upon the use of a set of numbered controls that 1) appear in the structure from motion point cloud model and 2) are mapped by a survey instrument (high accuracy GPS or accurately coordinated total station). Using the Helmert transformation, the arbitrary coordinates of the modeled targets are transformed into the real world coordinate system of the mapped reference targets. With the exception of Microsoft Excel (which could probably be replaced by OpenOffice), the entire work flow employs free software. Thus the techniques should be accessible to everyone with a digital camera, a laptop computer, an internet connection, and access to some kind of survey grade mapping equipment.


Equipment:
Digital Camera
Hand held GPS receiver (optional, for geotagging the photos)
Numbered Targets (orange cones, bright plastic plates, cut piece of plastic, or other equivalently noticeable object)
Mapping Equipment (High accuracy GPS or total station)
Laptop Computer

Software:
Structure from motion image processor (Microsoft's Photosynth or Noah Snavely's Bundler)
Geotagging program optional (Friedemann Schmidt's GeoSetter)
Point cloud viewing and measurement software (Menci's ScanView)
Coordinate transformation software (Graticule 3D)
Text editor (Notepad++)
1. FIELDWORK
1.1.
Place numbered targets on the surface or object that is to be recorded.
Because the numbered targets have to be located in the point cloud, it
is advisable to use numbered targets of colors that contrast strongly
with the surface of object that is to be recorded. Because of the way
that the points will be measured later, the first numbered point should
be numbered 0. This is because during the measurement process described
under 2.10 ScanView
begins its automatic numbering at 0. If for some reason, it is
absolutely necessary to begin numbering at 1, the measurements made in ScanView
can be renumbered--but do not forget to do this. When placing the
numbered targets, be sure that they will be visible from a number of
different shooting angles. It is generally useful to make a sketch map
that illustrates the location of the numbered targets.

1.2.
Start a small hand held GPS and keep it running throughout the entire
duration of photo shoot. This GPS data will be used to geotag the
photographs. Note that in order for geotagging to solve properly, the
clocks of the GPS and the camera must be perfectly synchronized. This
calibration of the camera's clock should be performed prior to taking
any photographs.

1.3. Following general guidelines for structure
from motion photography, capture images of the surface or object that
is to be recorded. Ensure that between each of the photographs there
are low angles (<15°) and that overlap is high (>70%). Be sure to
take detailed photographs of the numbered targets. Consider doing this
by zooming in on or shooting them with a long lens. These detailed
shots are required to get good representation of the numbered targets
in the point cloud.

1.4. Map the numbered targets with a survey
instrument. This may be done with either a high accuracy GPS or an
accurately coordinated total station.

1.5. Collect the numbered targets. The fieldwork stage is complete.

2. LABWORK
2.1 Load the photographs onto the computer.

2.2.
If geotagging the images, download the GPS data from the hand held
receiver. Geotag the photographs and apply any additional information
to the EXIF headers (i.e. who took the photographs). GeoSetter is a free program that can be used to geotag the images.

2.3. Downsize the photographs to the appropriate dimensions for the given structure from motion processor that will be used. Photosynth downsamples images to 2 megapixels. Bundler has processed images as large as 3.8 megapixels. Determine the megapixel dimensions by multiplying the height by the width. Megapixel = one million pixels.

2.4. Submit the downsampled image collection to the structure from motion processor. If using Photosynth, simply load the images into the Photosynth interface and begin processing. If using Bundler, there is additional information on installation and image processing here.

2.5.
From the high accuracy GPS or total station, download to the computer
the files that contain the mapped location of the numbered targets.

2.6.
Generate a text file that contains the targets with their corresponding
numbers. Do not include a header row. The format of the text file
should be as follows:
ID X Y Z

2.7. Once the structure from motion processor is complete obtain the point cloud files. If the images were processed with Photosynth, then obtain the point cloud by using the free Photosynth Point Cloud Exporter. When using this tool, export the point cloud as an ASCII ply file. If the images were processed with Bundler,
then look in the bundle directory and copy the ply file with the
highest number. Past this file into the directory where other files
related to the model are stored (i.e. the directory that contains the
text file with the mapped coordinates of the numbered targets).

2.8. From the ply file, delete the header information. All that should remain are six columns of data in the following format:
X Y Z R G B

2.9. After removing the header information, save the file as a text file.

2.10. Using the free program ScanView,
open inspect the text file that contains the point cloud information.
Explore the point cloud until all, or most, of the numbered targets are
located. Look for the groups of points that represent the numbered
targets. Once the numbered targets are located, use the ScanView measure tool to obtain the XYZ values for the appropriate point. Do this for each of the numbered targets.

2.11.
In the Scan View Measure dialog box, select all of the measured points
and click the copy button. Open a text editor, like the free Notepad++,
and select paste. This will produce a tab delimited array according to
the following format:
ID X Y Z

2.12. Save this file that contains the modeled coordinates representing the location of the numbered targets.

2.13. Close the point cloud file from ScanView, and then open this file in Excel. The format of the file should be:
X Y Z R G B

2.14.
Add a new column before the X column and add three new columns before
the R column. If B = a blank column, then the file should have the
following format:
B X Y Z B B B R G B

2.15. Above the very
first line of the file, add a new row for each of the numbered targets.
For example, if there are 10 numbered targets then add 10 new rows. In
these new rows, add the information from the file that was saved in
step 2.6, the file that contains the modeled coordinates of the numbered
targets. Add the value 0 into each of the blank columns that are
between the Z and the R column. For each of the numbered targets, add
the value 255 into each of the R G and B columns respectively.

2.16.
Beginning with the first number higher than the highest numbered
target, use the fill functions in Excel to give numbers to each of the
the renaming records that make up the points forming the point cloud.

2.17.
For each of the three empty columns that are between the Z and the R
column, add the value 0, and copy this value to each of the records
that make up the point cloud. Once these edits are made, save two
versions of the file. Where = the name of the file, save
one version as "_proc" and save the other version as
"_RGB". Close both of the files.

2.18. Now the files are properly formatted the "_proc" file will be passed to Java Graticule 3D where the Helmert coordinate transformation will be performed. The "_RGB" will be used later.

2.19.
Open the free program Java Graticule 3D. Click on the menu
"Transformation" and select 3D. A new window will open. This is the
CoordTrans program. From the CoordTrans window, under
"Transformationsarten:" be sure that 3D is selected. Click on the drop
down menu, and select 9-Parametertransformation-3D (Mx, My, Mz, Rx, Ry,
Rz, Tx, Ty, Tz). Click on the "Dateil" menu, select "Import
Startsystem", and browse to the edited point cloud file
"_proc". If the file is large, it may take a few minutes
for the file to load. Be patient. Once the file has loaded, look at the
first few records and ensure that they represent the numbered targets
that were entered earlier.

2.20. Once the "Startsystem" file
"_proc" loads, click the "Datel" menu, select "Import
Zielsystem" and brows to the surveyed control point file created in
step 2.17. The file should load quickly. Ensure that the coordinates for
the mapped numbered targets are present and correct.

2.21. Click
the "Berechnung starten" button and wait for CoordTrans to perform the
transformation. Once the transformation is complete, click on the
Transformation tab at the bottom of the window. Compare the coordinates
of the modeled numbered targets against the coordinates of the mapped
numbered targets. The values should be very close to one another. If
they are, then click the "Datei" menu, select "Export Transformation",
browse to where the file should be saved, name the file properly, and
save the file. The suggested name of the file is "_trans"

2.22.
From Excel, open the exported file "_trans" and open the
file "_RGB". From the file, "_RGB" copy the
colums that contain the RGB information and paste those values into the
file "_trans". The file "_trans" should now
have the form of:
ID X Y Z OX OY OZ R G B

2.23. Add a header
row with the appropriate column names and save the file. If importing
into ArcGIS, save an additional copy of the file as an Excel file.
Close both of the these files. The file "_trans" contains
the properly transformed point cloud. This file can be imported into a
GIS by loading the table and displaying XY data. Remember that northing
is Y and easting is X.

The image below shows a a blue structure
from motion point cloud and a green total station point cloud. Using
the procedures outlined above, the blue structure from motion point
cloud was properly oriented in real world space.




Equipment:
Digital Camera
Hand held GPS receiver (optional, for geotagging the photos)
Numbered Targets (orange cones, bright plastic plates, cut piece of plastic, or other equivalently noticeable object)
Mapping Equipment (High accuracy GPS or total station)
Laptop Computer

Software:
Structure from motion image processor (Microsoft's Photosynth or Noah Snavely's Bundler)
Geotagging program optional (Friedemann Schmidt's GeoSetter)
Point cloud viewing and measurement software (Menci's ScanView)
Text editor (Notepad++)
1. FIELDWORK
1.1. Place numbered targets on the surface or object that is to be recorded. Because the numbered targets have to be located in the point cloud, it is advisable to use numbered targets of colors that contrast strongly with the surface of object that is to be recorded. Because of the way that the points will be measured later, the first numbered point should be numbered 0. This is because during the measurement process described under 2.10 ScanView begins its automatic numbering at 0. If for some reason, it is absolutely necessary to begin numbering at 1, the measurements made in ScanView can be renumbered--but do not forget to do this. When placing the numbered targets, be sure that they will be visible from a number of different shooting angles. It is generally useful to make a sketch map that illustrates the location of the numbered targets.

1.2. Start a small hand held GPS and keep it running throughout the entire duration of photo shoot. This GPS data will be used to geotag the photographs. Note that in order for geotagging to solve properly, the clocks of the GPS and the camera must be perfectly synchronized. This calibration of the camera's clock should be performed prior to taking any photographs.

1.3. Following general guidelines for structure from motion photography, capture images of the surface or object that is to be recorded. Ensure that between each of the photographs there are low angles (<15°) and that overlap is high (>70%). Be sure to take detailed photographs of the numbered targets. Consider doing this by zooming in on or shooting them with a long lens. These detailed shots are required to get good representation of the numbered targets in the point cloud.

1.4. Map the numbered targets with a survey instrument. This may be done with either a high accuracy GPS or an accurately coordinated total station.

1.5. Collect the numbered targets. The fieldwork stage is complete.

2. LABWORK
2.1 Load the photographs onto the computer.

2.2. If geotagging the images, download the GPS data from the hand held receiver. Geotag the photographs and apply any additional information to the EXIF headers (i.e. who took the photographs). GeoSetter is a free program that can be used to geotag the images.

2.3. Downsize the photographs to the appropriate dimensions for the given structure from motion processor that will be used. Photosynth downsamples images to 2 megapixels. Bundler has processed images as large as 3.8 megapixels. Determine the megapixel dimensions by multiplying the height by the width. Megapixel = one million pixels.

2.4. Submit the downsampled image collection to the structure from motion processor. If using Photosynth, simply load the images into the Photosynth interface and begin processing. If using Bundler, there is additional information on installation and image processing here.

2.5. From the high accuracy GPS or total station, download to the computer the files that contain the mapped location of the numbered targets.

2.6. Generate a text file that contains the targets with their corresponding numbers. Do not include a header row. The format of the text file should be as follows:
ID X Y Z

2.7. Once the structure from motion processor is complete obtain the point cloud files. If the images were processed with Photosynth, then obtain the point cloud by using the free Photosynth Point Cloud Exporter. When using this tool, export the point cloud as an ASCII ply file. If the images were processed with Bundler, then look in the bundle directory and copy the ply file with the highest number. Past this file into the directory where other files related to the model are stored (i.e. the directory that contains the text file with the mapped coordinates of the numbered targets).

2.8. From the ply file, delete the header information. All that should remain are six columns of data in the following format:
X Y Z R G B

2.9. After removing the header information, save the file as a text file.

2.10. Using the free program ScanView, open inspect the text file that contains the point cloud information. Explore the point cloud until all, or most, of the numbered targets are located. Look for the groups of points that represent the numbered targets. Once the numbered targets are located, use the ScanView measure tool to obtain the XYZ values for the appropriate point. Do this for each of the numbered targets.

2.11. In the Scan View Measure dialog box, select all of the measured points and click the copy button. Open a text editor, like the free Notepad++, and select paste. This will produce a tab delimited array according to the following format:
ID X Y Z

2.12. Save this file that contains the modeled coordinates representing the location of the numbered targets.

2.13. Close the point cloud file from ScanView, and then open this file in Excel. The format of the file should be:
X Y Z R G B

2.14. Add a new column before the X column and add three new columns before the R column. If B = a blank column, then the file should have the following format:
B X Y Z B B B R G B

2.15. Above the very first line of the file, add a new row for each of the numbered targets. For example, if there are 10 numbered targets then add 10 new rows. In these new rows, add the information from the file that was saved in step 2.6, the file that contains the modeled coordinates of the numbered targets. Add the value 0 into each of the blank columns that are between the Z and the R column. For each of the numbered targets, add the value 255 into each of the R G and B columns respectively.

2.16. Beginning with the first number higher than the highest numbered target, use the fill functions in Excel to give numbers to each of the the renaming records that make up the points forming the point cloud.

2.17. For each of the three empty columns that are between the Z and the R column, add the value 0, and copy this value to each of the records that make up the point cloud. Once these edits are made, save two versions of the file. Where <name> = the name of the file, save one version as "<name>_proc" and save the other version as "<name>_RGB". Close both of the files.

2.18. Now the files are properly formatted the "<name>_proc" file will be passed to Java Graticule 3D where the Helmert coordinate transformation will be performed. The "<name>_RGB" will be used later.

2.19. Open the free program Java Graticule 3D. Click on the menu "Transformation" and select 3D. A new window will open. This is the CoordTrans program. From the CoordTrans window, under "Transformationsarten:" be sure that 3D is selected. Click on the drop down menu, and select 9-Parametertransformation-3D (Mx, My, Mz, Rx, Ry, Rz, Tx, Ty, Tz). Click on the "Dateil" menu, select "Import Startsystem", and browse to the edited point cloud file "<name>_proc". If the file is large, it may take a few minutes for the file to load. Be patient. Once the file has loaded, look at the first few records and ensure that they represent the numbered targets that were entered earlier.

2.20. Once the "Startsystem" file "<name>_proc" loads, click the "Datel" menu, select "Import Zielsystem" and brows to the surveyed control point file created in step 2.17. The file should load quickly. Ensure that the coordinates for the mapped numbered targets are present and correct.

2.21. Click the "Berechnung starten" button and wait for CoordTrans to perform the transformation. Once the transformation is complete, click on the Transformation tab at the bottom of the window. Compare the coordinates of the modeled numbered targets against the coordinates of the mapped numbered targets. The values should be very close to one another. If they are, then click the "Datei" menu, select "Export Transformation", browse to where the file should be saved, name the file properly, and save the file. The suggested name of the file is "<name>_trans"

2.22. From Excel, open the exported file "<name>_trans" and open the file "<name>_RGB". From the file, "<name>_RGB" copy the colums that contain the RGB information and paste those values into the file "<name>_trans". The file "<name>_trans" should now have the form of:
ID X Y Z OX OY OZ R G B

2.23. Add a header row with the appropriate column names and save the file. If importing into ArcGIS, save an additional copy of the file as an Excel file. Close both of the these files. The file "<name>_trans" contains the properly transformed point cloud. This file can be imported into a GIS by loading the table and displaying XY data. Remember that northing is Y and easting is X.

The image below shows a a blue structure from motion point cloud and a green total station point cloud. Using the procedures outlined above, the blue structure from motion point cloud was properly oriented in real world space.




No TrackBacks

TrackBack URL: https://blogs.psu.edu/mt4/mt-tb.cgi/131927

Leave a comment

Search This Blog

Full Text  Tag

Recent Entries

Quick and Dirty 123D Catch model/movie of Saywite
AutoDesk provides the free tool 123D Catch that permits photo textured 3D reconstructions based on structure from motion. The 3D…
Hypr3D and Meshlab to scaled model
This video illustrates the steps involved in generating a scaled 3D model from a set of photographs. All of the…
Hypr3D a promising tools for sharing SfM models on the web
Generating image collections for building structure from motion models is simple and inexpensive. These are two obvious appeals of the…

Subscribe


Visit Global Heritage Network