sábado, 23 de febrero de 2013

Bulk importing into GeoServer

Some of the algorithms contained in SEXTANTE do not actually do any analysis, but they just perform some common tasks such as management ones. Calling this algorithms from SEXTANTE is interesting, because you can save time by automating those task through the SEXTANTE modeler or the batch processing interface.

We will see in this entry how to bulk import layers into GeoServer using SEXTANTE, and how to add a bit of processing along the way, to ensure that the layers we import have optimal characteristics to provide maximum performance when responding to requests.

To import a single vector layer, double click on the Import vector into GeoServer

Set the parameters of the GeoServer instance that you want to import to (and make sure GeoServer is running) , the workspace where you want to import it, and the layer to import. Click OK and the layer will be imported.

The workspace must exist. If you want to create a new one, you can use the Create workspace algorithm before importing.

Importing to GeoServer from SEXTANTE is easy but, what if you have to import all the layers in a folder? That might take time. However, using the SEXTANTE batch processing interface can make the task much easier.

Go to the toolbox and, instead of double clicking on the name of the algorithm name, select the Import vector into GeoServer algorithm, right-click on it and choose the Run as batch process option. The following dialog will be shown.

Each row represents an execution of the algorithm. That is, in this case it represents one layer being imported into GeoServer. We have to fill the table, adding as many rows as needed, to perform the bulk import.

Select the Layer to import field in the top row. That correponds to the layer to import in the first execution of the algorithm. Click on the button in the field and you will see the typical file selection dialog. Browse to the folder where all the files you want to import are found, and select them all. Not just one of them, but all of them.

When you click on OK, SEXTANTE will understand that you want to use each selected file for a different run of the algorithm, and will add the rows that are needed to accomodate them.

Filling the other rows has to be done manually, but if you want to use the same values for all the rows in a given column (you will probably be importing to the same GeoServer instance and the same workspace), you can use the following trick: enter the value in the first row and then double-click in the column name. That will cause the value to be copied to all the other rows automatically.

Once the table is filled, just click on OK and the process will import all your layers into GeoServer. Open your GeoServer admin page to check that they have been imported.

As it happens with all SEXTANTE algorithms, this one is available in the GIS Python console as well. We can get some automation by calling it from a script. As an example, the following code imports all the layers that are  loaded in the current QGIS project .

layers = QGisLayers.getVectorLayers()

for layer in layers:


    sextante.runalg("gspg:importvectorintogeoserver","http://localhost:8080/geoserver/rest","admin","geoserver", layer,"workspace")

Let's now import raster layers instead of vector ones, but adding some extra processing before importing. What we are going to do is to make sure that the raster layer has a good format (in our case GeoTiff) in terms of performance, and a correct structure (we are going to ensure that it is internally tiled and has overviews).

The first thing to do is to export the raster layer as a GeoTiff using GDAL, and using the corresponding modifiers to add inner tiles of a size of 1024 x 1024 pixels. We do this with the GDAL Translate algorithm and the options shown in the figure below:

After that, we will run the Build pyramids algorithm, and add 4 levels of overviews, using the following configuration.

The last step will be importing into GeoServer, which is trivial if we already know how to import vector layers,since the interface of the algorithm to import raster ones is almost identical.

To make things better, we can put these three steps in a simple model, using the SEXTANTE modeler:

The URL and workspace fields have been harcoded in this case, and only the store name is shown as option when running the model.

That is the most convenient way of running this workflow, since the intermediate layer should not be loaded into QGIS but just generated (we can disable this when running the Translate algorithm, but as part of the model that is just defined once, and we do not have to worry about getting unnecesary layers in the QGIS canvas.)

Since models are just like any other algorithm, they can be run on the batch processing interface, so can do a bulk import of raster layers as well, including the preparation of the corresponding files before importing.

2 comentarios:

  1. Hola,
    Muy buena tu entrada, justo lo que necesito, pero no encuentro la herramienta import raser/ vector into Geoserver.
    Estoy usado Qgis Dufour no se si me podrias ayudar.
    Ana Belen Colen

  2. Hi,
    Same comment that above, I cant find the Import raster/vector to Geoserer tool in Qgis Dufour.
    Do I need to install a new plugging?