Go Back

Archiving

January 30, 2018

Piwik PRO can receive data in couple of various methods (Javascript Tracking code, Image tracker, Server log replay, backend trackers, SDK’s etc.)
No matter what method you’ll choose the data will land in on the Piwik server in a log form called raw data. In order to populate multiple Piwik PRO reports, the raw data needs to be processed, some data has to be extrapolated, some has to be summed or trends have to be found. That raw data processing is called archiving.
Without the archiving only the Live reports in Piwik will display the data, rest of the reports need to wait until the process finishes before it will show new data.

Live reports usually state that in their name but just in case, here’s the full list of reports and widgets working without the archiving:

  • Visitors in real-time
  • Live concurrent visits
  • Real-time visitor count
  • Real-time map
  • Visitor log

Above functionalities are great for testing purposes if you need to know if the tracking works (It will show the visits in Piwik PRO Analytics immediately). They also prove a valuable resource when you need to know what is happening on your website right now, but they lack a lot of processed information.

As for the rest of the reports, they will update with new data as soon as the next archiving process finishes. The process is scheduled to run regularly and the interval between them is decided based on the amount of traffic that hits the Piwik PRO server (ex. most of the Piwik PRO Cloud instances are archived every hour).
Next Archiving process will not start if the previous one haven’t finished. If that happens the next process will be skipped and the one in the following hour will try again.

In very rare cases the process can even take couple or more hours, however it should never be longer than 24 hours as that would mean that the data is coming to the server faster than it is archived, causing extending gap in the data. In such case the our admin team would have to step in to suggest a best approach to quicken the process. The most common solution in such situations is setting multiple archiving threads.