Go Back

Setting up the import

January 29, 2018

This step requires a little more familiarity with JavaScript, and around 4 to 10 hours of time depending on the volume of data.

You need to run the Log Importer tool with the correct parameters. Some of them must be present, while others are optional.

Sample command:

import_logs.py --url=https://demo.piwik.pro --login=demo@piwik.pro --token-auth=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX --enable-static --enable-bots --show-progress --idsite=15 --recorders=2 --recorder-max-payload-size=50 sample.log
--url=https://demo.piwik.pro

– This is a mandatory parameter which points to the location of your Piwik instance

--login=demo@piwik.pro

– login of a user with superuser rights

--token-auth=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

– authentication token of this user

--idsite=1

– defines the Site ID of the website

--recorders=2

– specifies the number of threads – Log Analytics on Piwik PRO Cloud works best when this setting is set to 2

--recorder-max-payload-size

– maximum number of log entries to record in one tracking request – the suggested value is 200

There are many other options that can be added to this script, which are described in the script itself:
https://github.com/piwik/piwik-log-analytics/blob/master/import_logs.py#L448

Once the log importer tool finishes parsing and uploading logs to your Piwik instance, you will have to wait for the archiving process to populate the Piwik reports with new data.

The time needed for this process depends on the amount of the data you’ve uploaded, and in rare cases may even take a couple of days (for example, uploading years of historical data), but usually it is a matter of around an hour.

If you plan to import logs on a regular basis it is advised to setup a scheduled job using a tool such as CRON.