June 5, 2013 GBKN GEO GIS NEN1878 SDO_GEOMETRY
The GBKN data comes in two formats: NEN1878 and Autocad DXF. Some programs can import and/or read Autocad DXF. Most other programs cannot import or read NEN1878 without custom modules. The NEN1878 format is a custom format which is - as far as I know - only found in The Netherlands. Thus, the data has to be converted to be useful for most applications.
The NEN1878 format is a very simple format:
- The data contains several objects;
- An object is either a point object, a line object, or a semantic object (text);
- The format is in plain text;
- There are several types of records:
- 4 types of records are meta-data for the file;
- 3 record types contain the data itself.
- Each file contains records, where each record 64 bytes long (not including \r\n);
- Some record types are split into sub-records;
- An object can consist of several records.
See this NEN1878 format description document (dutch) for a description of the format.
On a modern workstation, converting and loading 18GB of NEN1878 data takes just a few hours. I have seen other programs take several days with multiple computers to handle that amount of data.
To use it, make sure you have a working NodeJS. The NodeJS version used during development is 0.6 and 0.8. Your results may vary with NodeJS 0.10. Globally, these are the steps:
generate_init_sql.js, this will create a script to create all the tables and sptial-meta data in Oracle.
generate_post_sql.js, this will create a script to remove all invalid geometries and create indexes for Oracle Spatial.
gbkn_oracle_loader.jswith all the NEN1878 input files as arguments. For example:
gbkn_oracle_loader.js WEESP_N__7001.NEN. This will create several sqlldr files, each file will become a table in Oracle.
After all the files are converted, perform the following steps:
- Run the
- Run the
output/load_all.batfile. This will start sqlldr for all the sqlldr datafiles in the
- Run the
Now all your data should be read into Oracle in several tables. Review the mappings in the
mappings/ directory for the mapping of the GBKN types to Oracle types and other GBKN properties.
The program in itself might not be useful to you directly. It does contain code to convert a geometry to an SDO_GEOMETRY, which might be useful to you. Otherwise, the sqlldr-approach might also be useful in several other situations..