Overpass API >

Quick installation guide

We give here an installation guide for installing and starting the most common components. For other components, see the detailed installation guide. There is also an installation guide on the wiki which covers most errors that have ever been occured during installation and startup.

We do cover here various base systems, in particular at least Ubuntu Linux and FreeBSD.

Outline

We first give a complete list of commands you need for installation and startup: The comands are explained afterwards:

Install required software packages (on Ubuntu):

sudo apt-get install wget g++ make expat libexpat1-dev zlib1g-dev

Install the Overpass API software:

wget https://dev.overpass-api.de/releases/osm-3s_v0.7.55.tar.gz
gunzip <osm-3s_v0.7.55.tar.gz | tar xvf -
cd osm-3s_v0.7.55/
./configure
make
chmod 755 bin/*.sh cgi-bin/*

Clone the database or import a planet file:

mkdir -p db
bin/download_clone.sh --source=https://dev.overpass-api.de/api_drolbr/ --db-dir="db/" --meta=no

or

wget -O planet.osm.bz2 "https://ftp.heanet.ie/mirrors/openstreetmap.org/planet/planet-latest.osm.bz2"
bin/init_osm3s.sh planet.osm.bz2 "db/" "./" --meta

Activate minutely updates:

mkdir -p diffs
nohup bin/dispatcher --osm-base --meta --db-dir="db/" &
chmod 666 "db/osm3s_v0.7.55_osm_base"
nohup bin/fetch_osc.sh id "https://planet.openstreetmap.org/replication/minute/" "diffs/" &
nohup bin/apply_osc_to_db.sh "diffs/" auto --meta=yes &

Requirements

With a POSIX conforming operating system (this includes all kinds of Linux as well as FreeBSD, OpenBSD and several others), you have already fulfilled most base requirements.

Concerning hardware, I suggest at least 4 GB of RAM. The more RAM is available, the better, because caching of disk content in the RAM will significantly speed up Overpass API. The processor speed will have little relevance. For the hard disk, it depends on what you want to install. A full planet database with minutely updates should have at least 250 GB of hard disk space at disposal. Without minute diffs and meta data, 100 GB would already suffice.

To automatically download diff files, you need a command line download tool. I suggest wget. If it is not already installed, you can get it on e.g. Ubuntu with:

sudo apt-get install wget

If you want to use another program instead of wget, you can find an explanation in the detailed installation guide.

To compile the software, you need a C++ compiler and make. I suggest the GCC collection. If it is not already installed, you can get it on e.g. Ubuntu with:

sudo apt-get install g++ make

To compile the software, you also need the expat library and the zlib. If it is not already installed, you can get it on e.g. Ubuntu with:

sudo apt-get install expat libexpat1-dev zlib1g-dev

You can also include expat from sources. Please have a look in the detailed installation guide.

Software Installation

First, you need to download the tarball of the latest version:

wget https://dev.overpass-api.de/releases/osm-3s_v0.7.55.tar.gz
gunzip <osm-3s_v0.7.55.tar.gz | tar xvf -
cd osm-3s_v0.7.55/

You need to choose a directory where you put the executable files. You can later move them to a different directory. But the default choice of the installation program automake, /usr/bin, requires root permissions, although no root permissions are really necessary to run the program. I suggest subdirectories of the source code directory: this can be reached with "`pwd`". To configure this output directory and detect necessary adaptions of your system, run in the build subdirectory:

../src/configure --prefix="`pwd`"

Generate the executables:

make

Another system than Linux may require here some extra parameters. For example, FreeBSD needs -DNATIVE_LARGE_FILES, because it doesn't have a separate open64 function:

make CPPFLAGS="-DNATIVE_LARGE_FILES"

Fast Startup

Since version 0.6.98, the database can be cloned from an exisiting instance rather than created from scratch. This only takes 4 to 8 hours in comparison to 24 to 48 hours for an update from the planet file. Note that this feature is still rather experimental - please report any problems by eMail to me (roland at overpass-api.de). If you don't want the entire planet or prefer a manually planet import for some other reason, use the manual import instead.

Download a clone of the database at overpass-api.de with the command:

bin/download_clone.sh --source=https://dev.overpass-api.de/api_drolbr/ --db-dir="db/" --meta=no

or

nohup bin/download_clone.sh --source=https://dev.overpass-api.de/api_drolbr/ --db-dir="db/" --meta=no &

If you want meta data, use --meta=yes instead of --meta=no. This downloads about 30 GB (50 GB with meta data) in several compressed files and uncompresses them to a ready-to-use database.

Now you can proceed with minute updates.

Startup

The standard use case is to set up the database with the whole planet data and including meta data. If you haven't downloaded an OSM XML planet file yet, you can fetch one for example with:

wget -O planet.osm.bz2 "https://ftp.heanet.ie/mirrors/openstreetmap.org/planet/planet-latest.osm.bz2"

This file has a size of more than 20 GB. Thus, depending on your internet connection, it may take between 4 hours (fastest possible) and 22 hours (with 2 MBit) to download the file. If you are not working on your local machine, you may want the download to continue even if you logout. Use nohup for this:

nohup wget -O planet.osm.bz2 "https://ftp.heanet.ie/mirrors/openstreetmap.org/planet/planet-latest.osm.bz2" &

Once you have the file, you can start the import. The import again may take up to 48 hours:

bin/init_osm3s.sh planet.osm.bz2 "db/" "./" --meta

or

nohup bin/init_osm3s.sh planet.osm.bz2 "db/" "./" --meta &

You may need to adapt the parameters: The first parameter planet.osm.bz2 is the osm file to process, the second parameter "db/" is the directory where the database should go to, and the third parameter "./" is the base directory of the executables, i.e. there must exist update_database in the subdirectory bin of the location where the third parameter points to.

When this command is done, it writes Update complete. to the console (or to the file nohup.out if you have used nohup). At this point, the database can be used.

Minute Updates

But maybe you also want minutely updates. In this case, run the commands:

nohup bin/dispatcher --osm-base --meta --db-dir="db/" &
chmod 666 "db/osm3s_v0.7.55_osm_base"
nohup bin/fetch_osc.sh id "https://planet.openstreetmap.org/replication/minute/" "diffs/" &
nohup bin/apply_osc_to_db.sh "diffs/" id --meta=yes &

These commands don't make sense without nohup, because the programs become daemons and never terminate. Once again, you need to replace parameters: you always need to replace id by the replicate id to start from. If you have cloned the database then the right id is the content of the file db/replicate_id. Otherwise, search on https://planet.openstreetmap.org/replication/minute/ with your browser for the last replication diff that has been created before the planet creation date.

The other parameters need only to be adapted if you have chosen a different directory in a previous step: "db/" is the directory of the database, "https://planet.openstreetmap.org/replication/minute/" is the replicate diff's remote source, and "diffs/" is the directory where the minute diffs are stored until they have been applied.

Congratulations! Now you have a database mirror that can serve the entire world and is always only a few minutes behind the OSM main database.