Open Historical Map/Dev
Introduction
Documentation of things for OHM development
OHM Github repo: https://github.com/organizations/OpenHistoricalMap
Key pieces of infrastructure: - Rails Port - Mod_tile
Server Setup
The server was mostly set up with packages according toː http://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/
Originally the older mapnik osm.xml stylesheet in /home/tim/src/mapnik-style/osm.xml was used
Then the ohm-carto carto css was deployed. This currently lives in /home/tim/ohm-carto/
The shapefiles I downloaded and put into ohm-carto/data.zip which need to be extracted into ohm-carto/data to use.
the carto node module was installed into this ohm-carto directory.
The mapnik.xml was created using the carto command:
:~/ohm-carto$ ./node_modules/carto/bin/carto project.mml > mapnik.xml
The default tiles site was disabled from Apache. Instead we just added the one line to the OHM VHOST:
LoadTileConfigFile /etc/renderd.conf
The renderd conf:
[renderd]
stats_file=/var/run/renderd/renderd.stats
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile
[mapnik]
plugins_dir=/usr/lib/mapnik/input
font_dir=/usr/share/fonts/truetype/
font_dir_recurse=true
[default]
XML=/home/tim/ohm-carto/mapnik.xml
#XML=/home/tim/src/mapnik-style/osm.xml
URI=/ohm_tiles/
DESCRIPTION=This is the standard osm mapnik style
HOST=openhistoricalmap.org
Replication / Planet
Osmosis was installed outside the packages and is installed in /usr/local/bin
OHM API -> Replication change files: home/tim/replication/minute/data
every 5 minutes
crontab for tim user
*/5 * * * * cd /home/tim/replication/minute/; osmosis -q --replicate-apidb authFile=/home/tim/osmosis/dbAuth.txt validateSchemaVersion=no --write-replication workingDirectory=data
Replication change files -> mapnik database and refresh tiles etc:
crontab for www-data user
*/5 * * * * /var/lib/mod_tile/openstreetmap-tiles-update-expire.sh
/var/lib/mod_tile:
configuration.txt - points to baseUrl=file://localhost/home/tim/replication/minute/data
openstreetmap-tiles-update-expire.sh - collation, import of changes and expiry of tiles
replag - a script to show the lag between imports, called by the expire.sh script
Planet can be generated via
osmosis --read-apidb authFile=/home/tim/osmosis/dbAuth.txt validateSchemaVersion=no --write-xml file="ohm_planet_20150303.osm.bz2"
as of 3 March 2015 it took around 2 1/2 hrs 7943383 ms
osm2pgsql custom things
Osm2pgsql must be compiled with lua support.
in the https://github.com/openhistoricalmap/osm2pgsql repository - there are two files of interest:
ohm_style.style - here we specify that start_date and end_date should be dates in the database, and to add an extra start_date_year and an end_date_year coluumn
ohm_tag_transform.lua - this is a Lua script to parse out and ensure that the dates are in YYYY-MM-DD formatting. It also parses out the year into the _year column value
example execution:
osm2pgsql --slim -S ohm_style.style --tag-transform-script ohm_tags_transform2.lua -d ohm_gis -C 16000 --number-processes 4 ohm_test.osm
Custom layers
OHM Readme
The 2008 and 2009 styles are meant to be used with the OHM osm2pgsql style and tag transforms which use start_date and end_date which are postgresql date types and start_date_year and end_date_year which are number types
import the sql commands to create the two 2008 and 2009 views and add them to geometry columns
psql -d ohm_gis_hack -f views_2008_2009.sql
Compile the carto stylesheets
1. install carto locally
npm install carto
2. compile new project files for each:
./node_modules/carto/bin/carto project_2008.mml > mapnik_2008.xml
./node_modules/carto/bin/carto project_2009.mml > mapnik_2009.xml
3. set up renderd configuation for each
[ohm_2008]
XML=/home/tim/ohm-carto/ohm-carto/mapnik_2008.xml
URI=/ohm_2008/
DESCRIPTION=A custom ohm map showing those things in 2008
HOST=openhistoricalmap.org
[ohm_2009]
XML=/home/tim/ohm-carto/ohm-carto/mapnik_2009.xml
URI=/ohm_2009/
DESCRIPTION=A custom ohm map showing those things in 2009
HOST=openhistoricalmap.org
troubleshooting
Renderd
Run renderd from the command line
sudo service renderd stop
renderd -f -c /etc/renderd.conf
see what errors you find