QGIS Server is an open source OGC data server which uses QGIS engine as backend. It becomes really awesome because a simple desktop qgis project file can be rendered as web services with exactly the same rendering, and without any mapfile or xml coding by hand.

QGIS Server provides a way to serve OGC web services like WMS, WCS and WFS resources from a QGIS project, but can also extend services like GetPrint which takes advantage of QGIS’s map composer power to generate high quality PDF outputs.

Oslandia decided to get strongly involved in QGIS server refactoring work and co organized a dedicated Code Sprint in Lyon.

We also want to warmly thank Orange (French Internet and Phone provider) for its financial supports for helping us ensure QGIS 3 is the next generation of bullet proof, fast and easy to use open source web map server.

Interoperability

When we talk about web map server, we already imagine how our maps are rendered and integrated into websites. However, there are a number of servers like Geo Server, Map Server or QGIS Server, so we have to ensure that the behavior is similar to avoid technology risk and thus having an interoperable product.

One way to achieve this goal is to use a tool provided by the OGC Compliance Program to run dedicated tests on the server : Teamengine (Test, Evaluation, And Measurement Engine). Once Teamengine is ready to be used, we just have to install the corresponding test suites for the desired version (WMTS 1.0, WFS 1.11, WMS 1.1.1, and so on). Then test suites are available through a web interface:

During our work, we focused on the WMS 1.3.0 OGC standard in order to test the following requests:

• GetCapabilities : returns supported operations, metadata and available layers
• GetFeatureInfo : retrieves data at a specific map location (geometry, attribute values, …)
• GetMap : returns an image of a map given layers to render, extent, …
• GetLegendGraphic : returns a legend for a map

Continuous integration

Our aim was not only to run these tests to have a global vision of QGIS Server’s status, but also to provide a continuous integration facility to always check the current status of the master branch of QGIS. So, the first step was to build a single Docker image with both QGIS Server and Teamengine thanks to an init system embedded into the Docker container. Moreover, some QGIS project files have to be prepared with specific data. In our case, for the WMS 1.3.0 standard, the test dataset is described and available here. No worries, these QGIS projects are already packaged and embedded within the image!

The resulting image is available on the Oslandia Docker hub:

$ docker pull oslandia/qgis-server-ogc-cite:latest

To run a container from this image on a Debian host:

$ docker run --rm --name qgis-server-ogc-cite --cap-add SYS_ADMIN -v /sys/fs/cgroup:/sys/fs/cgroup -d oslandia/qgis-server-ogc-cite:latest
$ docker exec -it qgis-server-ogc-cite /bin/ps -ax
  PID TTY      STAT   TIME COMMAND
    1 ?        Ss     0:00 /bin/systemd
   17 ?        Ss     0:00 /lib/systemd/systemd-journald
   33 ?        Ss     0:00 /usr/bin/dbus-daemon --system --address=systemd: --no
   40 ?        Ss     0:00 /bin/sh /home/user/wrapper_qgisserver.sh
   42 ?        Ss     0:00 /lib/systemd/systemd-logind
   49 ?        S      0:00 /bin/sh /usr/bin/xvfb-run /usr/bin/spawn-fcgi -f /usr
   95 ?        S      0:00 Xvfb :99 -screen 0 640x480x8 -nolisten tcp -auth /tmp
   97 ?        Sl     0:13 /usr/bin/java -Djava.util.logging.config.file=/srv/to
  130 ?        Ss     0:00 nginx: master process /usr/sbin/nginx -g daemon on; m
  133 ?        S      0:00 nginx: worker process
  154 ?        Sl     0:00 /usr/local/bin/qgis_mapserv.fcgi
  259 ?        Rs+    0:00 /bin/ps -ax

However, depending on your OS or Linux distribution, some additional parameters may be needed to run the Docker container with systemd. For example on Ubuntu 16.04, we need to indicate more parameters via the –tmpfs option to make it work:

$ docker run --rm --name qgis-server-ogc-cite --cap-add SYS_ADMIN -v /sys/fs/cgroup:/sys/fs/cgroup --tmpfs /run --tmpfs /run/lock --tmpfs /tmp -d oslandia/qgis-server-ogc-cite:latest

Once the container is running, you can reach QGIS Server thanks to the GetCapabilities request. And as outlined above, a specific QGIS project, fully prepared for the WMS 1.3.0 test suite has to be used through the MAP parameter:

$ docker inspect qgis-server-ogc-cite --format '{{.NetworkSettings.Networks.bridge.IPAddress}}'
172.17.0.2
$ curl "http://172.17.0.2/qgisserver?REQUEST=GetCapabilities&SERVICE=WMS&VERSION=1.3.0&MAP=/home/user/teamengine_wms_130.qgs"
<?xml version="1.0" encoding="utf-8"?>
<WMS_Capabilities version="1.3.0" xmlns:qgs="http://www.qgis.org/wms" xmlns="http://www.opengis.net/wms" xmlns:sld="http://www.opengis.net/sld" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wms http://schemas.opengis.net/wms/1.3.0/capabilities_1_3_0.xsd http://www.opengis.net/sld http://schemas.opengis.net/sld/1.1.0/sld_capabilities.xsd http://www.qgis.org/wms http://172.17.0.2/qgisserver?MAP=/home/user/teamengine_wms_130.qgs&SERVICE=WMS&REQUEST=GetSchemaExtension">
 <Service>
  <Name>WMS</Name>
  <Title></Title>
  <Abstract></Abstract>
  <KeywordList>
   <Keyword vocabulary="ISO">infoMapAccessService</Keyword>
  </KeywordList>
  <OnlineResource xlink:type="simple" xlink:href="" xmlns:xlink="http://www.w3.org/1999/xlink"/>
  <ContactInformation>
...
...
...
...

Then, several possibilities are available to actually run the test suite. One of them is to go on the Teamengine web page provided by the container through the port 8080 and create a login to connect to. Thus, the result will be a simple HTML report indicating the status (Failed or Passed) for each kind of tests. However, for the needs of continuous integration and automation, we used the REST API coming with the test suite. Note that the ampersand character ‘&’ has to be encoded as ‘%26′:

$ curl "http://172.17.0.2:8080/teamengine/rest/suites/wms/1.20/run?queryable=queryable&basic=basic&capabilities-url=http://172.17.0.2/qgisserver?REQUEST=GetCapabilities%26SERVICE=WMS%26VERSION=1.3.0%26MAP=/home/user/teamengine_wms_130.qgs"

And by doing so, we obtain a big XML report (several tens of thousands of line). But with a xsl file and a Q&D python script, we are finally able to retrieve a HTML report! To make it more readable, we not only added images generated by QGIS Server into the report for the GetMap and the GetLegendGraphic requests, but we also used an online instance of GeoServer to generate images for the same requests:

For the time being, we created a dedicated GitLab Pipeline to run nightly builds for the WMS 1.3.0 test suite thanks to a crontab trigger, and HTML reports are finally published here.

Tests results

There are several kinds of tests which can be activated for the WMS 1.3.0 standard:

• Basic (required for certification)
• Queryable
• Raster elevation
• Vector elevation
• Time

Thanks to the REST API, we just run the Basic and Queryable options. However, due to an issue within the test engine, we had to deactivate 5 tests related to the GetMap request in the XML configuration file. As these tests was causing a crash of Teamengine (the result is not properly digested), we may reasonably considered their status to Failed. The issue has been reported on the ets-wms13 github repository and everything should work fine after the next release.

Once deactivated, about 115 tests are launched and only 4 are failing! Even if these numbers may change a bit in the meantime because the server is under heavy development for the 3.0 release, this is very good news because this means that more than 90% of the tests passed. Certification is not so far!

How to use OGC testsuite in development phase

If you’re a QGIS server developer, then this part is made for you! Indeed, the docker image may also be used to run OGC tests on a local server. By doing so, the server is running on your host but OGC tests are running on the container.

For this purpose, the server needs to have read access on the QGIS project file embedded within the container and used during the OGC tests. To do that, we have to add an option to share the directory /home/user when the container is created:

$ docker run --rm --name qgis-server-ogc-cite --cap-add SYS_ADMIN -v /sys/fs/cgroup:/sys/fs/cgroup -v /home/user/ -d oslandia/qgis-server-ogc-cite:latest

Then we just have to mount the docker volume in read only (to avoid all sorts of dangerous handling):

$ export OGC_VOLUME=$(docker inspect --format '{{range .Mounts}}{{ if eq .Destination "/home/user" }}{{ .Source }}{{ end }}{{ end }}' qgis-server-ogc-cite)
$ echo $OGC_VOLUME
/var/lib/docker/volumes/51a9ddef47a5056cd96103d95496f95d65af5f523d07e465b44c5e27f612c821/_data
$ mkdir -p /tmp/ogc_volume
$ sudo mount --bind $OGC_VOLUME /tmp/ogc_volume
$ ls /tmp/ogc_volume
data-wms-1.3.0      ets-wms13   teamengine_wms_130.qgs  wrapper_qgisserver.sh
data-wms-1.3.0.zip  teamengine  testng

As we can see, the interesting QGIS project file is /tmp/ogc_volume/teamengine_wms_130.qgs. Assuming that your local server is available through http://192.168.1.10/qgisserver, we may run the GetCapabilities request using the QGIS project file located on the container:

$ curl "http://192.168.1.10/qgisserver?REQUEST=GetCapabilities&SERVICE=WMS&VERSION=1.3.0&MAP=/tmp/ogc_volume/teamengine_wms_130.qgs"

Note that your web server has to be properly configured to indicate the IP address of the host in the OnlineResource XML entities:

<OnlineResource xlink:href="http://192.168.1.10/qgisserver?MAP=/tmp/ogc_volume/teamengine_wms_130.qgs&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetLegendGraphic&LAYER=cite:Autos&FORMAT=image/png&STYLE=default&SLD_VERSION=1.1.0" xlink:type="simple" xmlns:xlink="http://www.w3.org/1999/xlink"/>

For the needs of continuous integration, the REST API is used to avoid manual operations. However, by doing so, some tests implying visual statements are not run. In the development phase, it seems to be a good practice to also run these tests. So, we have to go on the Teamengine webpage hosted by the container through http://172.17.0.2:8080/teamengine. Then, after registering and signing in, we arrive on the below page where you have to click on `Create a new session`:

Then, once we have selected the testsuite for the WMS 1.3.0 specification in the drop-down menu (others specifications such as WFS or WCS are also available in the menu, but not configured so it’s not runnable), we way start the new session and configure the capabilities setup by indicating the GetCapabilities request for your local server:

Finally, we just have to click on the OK button to run OGC tests, visually confirm some results and wait a little to get an HTML report:

Conclusion

Even if it’s a great step forward to have an objective and a measurable way to test QGIS Server, a lot of work still need to be done for testing WMS 1.1.0, WFS, WCS, SLD support and so on.

Moreover, thanks to this report, we are now able to improve the behavior of the server by doing bugfixes!

One last thing, we are currently working with QGIS.org to host the html reports somewhere on qgis.org websites so that they are available freely to the community.

And if you have some issues using the Docker image, do not hesitate to open an issue on the github project