For more than two years, Oslandia has been working with Mappy for its transition towards OpenSource. The first step is represented by the database migration project of the cartographic backoffice, from Oracle to PostGIS. This has just been finalized and is currently running in production.
Mappy offers routing and cartography services. Supporting several billions of requests per month, the LBS platform (Location Based Services) is at the heart of Mappy’s activity and serves as a base for new products such as Web To Store. Oslandia works together with Mappy to migrate this cartographic platform from a solution developped in-house, to a new solution based entirely on free and opensource softwares, in order to prepare forthcoming technical and operational challenges.
The migration
Data preparation and cartographic map creation backoffices have been migrated successfully.
This software stack was initially based on Oracle Spatial, SQL Server, and other non-standard internal tools to create the map. It only runs opensource components by now. Oracle and SQL Server have completely been replaced by PostgreSQL and PostGIS, which is the key component for spatial data storage and preprocessing. Mapnik, Python, Tornado, Varnish, MemCached, Debian are other Opensource components used in this architecture.
The migration to OpenSource softwares allowed Mappy to rationalize and optimize the architecture of their « map » component :
- Reduced data processing time
- API and data format Standardization
- Strong reduction of the technical debt
- Reduced amount of lines of code
- Cost reduction for the platform, and cost-effective scaling
- Technical skills improvements for the team and stronger motivation
- Full technical mastership of their tools
Oslandia helped Mappy to integrate the Opensource culture and methods into their teams, and provided them with the technical skills required to implement this high-load architecture. Oslandia’s competences in geographical information systems, and its high expertise on PostGIS let the project be achieved successfully.
Mappy benefited from the latest OpenSource technologies, at the state of the art, and was also able to contribute back to some free software, like Mapnik.
Migration steps
The migration has been achieved with several steps, so as to keep the project under control, validating the migrated components through early tests.
The initial platform was the following one :
The target platform, with all components migrated to OpenSource, is now the following one :
Thanks to the availability of OpenSource components, a prototype has been developped within two weeks at the beginning of the project, so as to validate architectural decisions. Then the real migration work has begun.
The choice concerning OpenSource components has been fast, as already said. The target architecture could therefore be setup quickly, and the development and migration processes have been setup once this first step validated.
This migration gave the opportunity for Mappy to contribute to Mapnik. A 25% performance gain was achieved when reading data from PostGIS. Mappy also opensourced Pycnik, a Python tool to generate styles for Mapnik.
PostGIS
PostGIS, the database supporting the whole infrastructure, allowed to reach very high performances and a rich functional level. The database – 75GB – benefits from all latest enhancements of PostgreSQL and PostGIS, such as streaming replication, PostGIS new geographical functions, recursive CTEs, JSON native support and much more.
Audrey Malherbe ( @AudreyMalherbe ), project manager at Mappy, stresses that choosing PostgreSQL and PostGIS was obvious:
« We wanted to dive into OpenSource, leaning on reknown and performant technologies like PostGIS. It was important for us to contribute to OpenSource, and Oslandia’s expertise and its implication in the community has allowed us to rush into this adventure with full confidence. »
Some facts
Most geeks like to see this kind of numbers :
- for 85% of all Mappy traffic
- 14 servers (4 PostGIS, 8 Tornik, 2 Varnish)
- 240 req/s peak without cache
- 2500 req/s peak with cache
- 94% Cache Hit Ratio
- 2M objects in cache warming
- 75 GB for PostGIS database
- 300 GB for the relief database
Next steps
More technical informations on this migration are available in the `Audrey’s presentation at FOSS4G 2013.
Mappy intends to go on with this OpenSource orientation and extend this migration method to the other services of the cartographic platform.
About Mappy
Routing and cartographic services specialist, Mappy is reknown as the leader for local search through maps, on internet, tablets, mobiles and GPS.
Mappy offers three kinds of search to its users : map search, which allows to visualize a neighborhood, to experience city immersion thanks to 360° views in 320 french cities, but also to open the door of several thousands of shops ; the routing service available for cars, public transportation, bicycle and pedestrian mode ; and product search, allowing to locate a specific product in a given geographical zone, to know its price and availability.
As a major actor of urban transportation, Mappy offers to advertisers a geolocated solution on the whole country, facilitating web-to-store applications and trafic generation to their retail centers.
Mappy counts as of now more than 10 millions users on Internet, tablets and mobile (Mappy and MappyGPS Free).
Mappy is a subsidiary of the Solocal Group.
