What is a store-and-forward data logger?

A data logger is a self-contained instrument that sits next to its sensors in the field and keeps running without a person or a network. Campbell Scientific, which makes them, describes data loggers as devices that scan a wide variety of measurement sensors, perform programmed calculations, convert data to other units, and store it in memory. They can also transmit the data for analysis and reporting, and control external devices.

The part that matters for unreliable networks is the order of those steps. The logger measures and stores first. Sending is a separate, later step, and it may happen minutes, days or weeks after the measurement, depending on the link. This is the same store-and-forward pattern that email and delay-tolerant networks use, described in how store-and-forward messaging works, applied to measurements instead of messages.

In Campbell Scientific’s CRBasic language, a program declares output tables. The DataTable instruction defines each table’s name, the condition that triggers a new record, and its size in records. Each time the trigger condition is true, the logger writes a new row, a record, with the processed values from that interval.

Three details show the kinds of decisions every logging system has to make.

• What happens when memory is full. By default, table memory is organised as a ring: when it is full, the oldest data are overwritten by the newest. A FillStop statement changes that, so the table stops storing when it is full and stores nothing more until it is reset. A ring keeps recent data and loses old; fill-and-stop keeps the beginning of a record and loses what came after.
• How time is stored. If a table records on a fixed interval, the logger does not need to store a time stamp with every row. The CRBasic documentation says the logger then stores time only periodically and calculates each record’s time stamp at retrieval, from the number of records and the last successful storage.
• Where the data lives. A table can also be saved to a removable memory card with CardOut, so the records can be carried away physically.

Storage is sized for the gap, not the average. If a site can go without contact for a week, the tables need to hold more than a week of records, or the ring will overwrite data nobody has collected.

Campbell Scientific’s communications range covers both on-site and telemetry peripherals, and its loggers can be reached in several ways. Its LoggerNet software, which runs on a PC, retrieves data from loggers over direct connections, Ethernet, short-haul and phone modems (land-line or cellular), and UHF, VHF or spread-spectrum radio. LoggerNet is built for scheduled data retrieval across large logger networks, and its server stores retrieved data in a cache before writing it out in formats such as CSV and XML.

Retrieval can take a few shapes:

1. Scheduled collection. Software at the base polls each logger on a timetable over whatever backhaul the site has.
2. On-site collection. A person visits with a laptop or swaps a memory card. The data travels physically, like a data mule.
3. Mixed. A site can send a small subset over a low-bandwidth link and keep the full record for collection on site.

In each case the logger’s memory is the safety net. A failed poll means the records wait for the next one, as long as the table has room.

• Size each table for the longest gap you expect, plus a margin, at the rate the trigger fires. Campbell’s documentation recommends a fixed size for tables written only when a condition is met, rather than letting them claim memory as if the condition were always true.
• Pick ring or fill-and-stop deliberately. Ring suits continuous monitoring where recent data matters most. Fill-and-stop suits a test or campaign where the start must be preserved.
• Keep the clock right. Time stamps come from the logger’s clock. LoggerNet can check or set a logger’s clock when it connects, and a schedule for doing so limits how much clock drift can creep into the record.

A phone app that records readings offline and uploads them later is a data logger in software. The same choices apply: a bounded table, a rule for when it is full, trustworthy time stamps, and a record that stays on the device until the destination has it. The general version of this is covered in how to collect device data without internet.