Table of Contents

• 1  Cleaning files individually

• 2  Cleaning files by batch

  • 2.1  Start/stop-time log file

  • 2.2  Reading SST log files with the Reader class

• 3  Test with real data

How to discard invalid sequences in actigraphy recordings before analysis

Quite frequently, actigraphy recordings contain sequences that do not correspond to the participant’s activities but rather to artefacts, such as bringing the actimeter to the participant’s home or removing the actimeter from the participant to read the acquired data. Such meaningless data happen either at the beginning, when the record starts before the actimeter is actually worn by the participant or at the end, when the actimeter is removed by the participants before the end of the data acquisition. Or even at both ends…

In any case, it is necessary, before any analysis of the data, to discard these invalid sequences of activity counts and pyActigraphy provides the adequate tools for that.

import pyActigraphy
import os

import plotly.graph_objs as go

# retrieve path to example files
fpath = os.path.join(os.path.dirname(pyActigraphy.__file__),'tests/data/')

# Read test file
raw = pyActigraphy.io.read_raw_awd(fpath+'example_01.AWD')

# Check the start time of the actigraphy recording
raw.start_time

Timestamp('1918-01-23 13:58:00')

# Check the duration of the recording
raw.duration()

Timedelta('12 days 18:41:00')

go.Figure(data=[go.Scatter(x=raw.data.index, y=raw.data)], layout=go.Layout())

The initial recording starts the 23rd of January, around 13:00 and lasts for 12 days, 18 hours and 41 minutes. Both at the beginning and at the end of the recording, the recorded activity does not seem to be valid: - at the beginning, this mixture of zero and non-zero activity counts with no cyclic pattern is most likely due to the fact the acquisition started before the participant actually wore the actimeter but the actimeter was nontheless manipulated for demonstration purposes - towards the end of the recording, the activity counts suddenly fall to zero, when the participant removed the actimeter, until the last epochs, when the actimeter was most likely manipulated for reading the acquired data.

Cleaning files individually

With pyActigraphy, it is quite easy to mask these invalid sequences by specifiying a start and stop times for the analysis of the recordings.

In our example above, to discard the first hour of data acquisition and keep only 9 days of recordings:

raw_cropped = pyActigraphy.io.read_raw_awd(
    fpath+'example_01.AWD',
    start_time='1918-01-24 08:00:00',
    period='9 days')

raw_cropped.start_time

Timestamp('1918-01-24 08:00:00')

raw_cropped.duration()

Timedelta('9 days 00:01:00')

go.Figure(data=[go.Scatter(x=raw_cropped.data.index, y=raw_cropped.data)], layout=go.Layout())

Let us now verify the impact of discarding invalid activity counts on the usual rest-activity rhythm variables:

raw.IS()

0.527656245354158

raw_cropped.IS()

0.8290854646594275

The difference is quite substantial, isn’t it?

Cleaning files by batch

It is certainly managable to invalidate activity sequences in a few files by specifying a start and a stop time. However, it is less so when one wants to apply this procedure to hundreds or thousands of recordings. Fortunately, pyActigraphy allows users to specify a start and a stop time to actigraphy recordings by batch.

All it needs is a start/stop-time (SST) log file.

A Start/stop-time log file consists in a list of participant’s ID associated with two datetimes corresponding to the requested start and end of the actigraphy recording, respectively. These datetimes will overwrite the actual start and stop times of the recordings.

---

Start/stop-time log file must be formatted as the following:

Subject_id	Start_time	Stop_time	Remarks
name_in_header	YYYY-MM-DD HH:MM	YYYY-MM-DD HH:MM	WhateverTextYouWant

Start/stop-time log file

As an example, let us first create an SSTLog object in order to inspect it:

from pyActigraphy.log import SSTLog

sstlog_ods = pyActigraphy.log.read_sst_log(fpath+'example_sstlog.ods')

/Users/ghammad/Work/pyActigraphy/pyActigraphy/log/baselog.py:157: FutureWarning:

Inferring datetime64[ns] from data containing strings is deprecated and will be removed in a future version. To retain the old behavior explicitly pass Series(data, dtype={value.dtype})

sstlog_csv = pyActigraphy.log.read_sst_log(fpath+'example_sstlog.csv')

sstlog_xls = pyActigraphy.log.read_sst_log(fpath+'example_sstlog.xlsx')

To access to the data contained in the log file:

sstlog_ods.log

	Start_time	Stop_time	Duration
Subject_id
example_01	1918-01-24 08:00:00	1918-02-02 08:00:00	9 days
example_03	1918-01-26 14:14:00	1918-02-05 14:14:00	10 days
bad_example	1970-01-01 00:00:00	1970-02-01 00:00:00	31 days

sstlog_csv.log

	Start_time	Stop_time	Duration
Subject_id
example_01	1918-01-24 08:00:00	1918-02-02 08:00:00	9 days
example_03	1918-01-26 14:14:00	1918-02-05 14:14:00	10 days
bad_example	1970-01-01 00:00:00	1970-02-01 00:00:00	31 days

sstlog_xls.log

	Start_time	Stop_time	Duration
Subject_id
example_01	1918-01-24 08:00:00	1918-02-02 08:00:00	9 days
example_03	1918-01-26 14:14:00	1918-02-05 14:14:00	10 days
bad_example	1970-01-01 00:00:00	1970-02-01 00:00:00	31 days

Please, notice that the duration of the time interval between the start and stop times is automatically calculated and therefore does not need to be entered in the log file.

A summary function is available. It returns the min/max/median/mean of the durations found in the log file:

sstlog_ods.summary()

count                             3
mean               16 days 16:00:00
std      12 days 10:09:15.560846451
min                 9 days 00:00:00
25%                 9 days 12:00:00
50%                10 days 00:00:00
75%                20 days 12:00:00
max                31 days 00:00:00
Name: Duration, dtype: object

sstlog_csv.summary()

count                             3
mean               16 days 16:00:00
std      12 days 10:09:15.560846451
min                 9 days 00:00:00
25%                 9 days 12:00:00
50%                10 days 00:00:00
75%                20 days 12:00:00
max                31 days 00:00:00
Name: Duration, dtype: object

sstlog_xls.summary()

count                             3
mean               16 days 16:00:00
std      12 days 10:09:15.560846451
min                 9 days 00:00:00
25%                 9 days 12:00:00
50%                10 days 00:00:00
75%                20 days 12:00:00
max                31 days 00:00:00
Name: Duration, dtype: object

Reading SST log files with the Reader class

As it is possible to read actigraphy files by batch with the Reader class, this class has been extended to read a SST log file.

As an illustration, let us first read a small batch of files:

rawReader = pyActigraphy.io.read_raw(input_path=fpath+'example_0[0-3]*.AWD', reader_type='AWD')

Check how many files were found and read:

len(rawReader.readers)

4

To read a SST log files containing the start and stop times for these recordings:

rawReader.read_sst_log(fpath+'example_sstlog.csv')

Now, verify the data found in this log file by accessing and displaying the log associated to the reader object:

rawReader.sst_log.log

	Start_time	Stop_time	Duration
Subject_id
example_01	1918-01-24 08:00:00	1918-02-02 08:00:00	9 days
example_03	1918-01-26 14:14:00	1918-02-05 14:14:00	10 days
bad_example	1970-01-01 00:00:00	1970-02-01 00:00:00	31 days

Then, let us check the IS values of these recordings:

rawReader.IS()

{'example_02': 0.5237758757150337,
 'example_03': 0.3720781873681576,
 'example_01': 0.527656245354158,
 'example_01_mask': 0.44565929945478583}

There are quite low. Indeed, the SST log has been read by the reader object but the start and stop times have not been applied to the actigraphy recordings yet.

To do so:

rawReader.apply_sst(verbose=True)

Could not find an entry in SST log file for example_02
Found an entry in SST log file for example_03
Found an entry in SST log file for example_01
Could not find an entry in SST log file for example_01_mask

By using the verbose option, one can easily check if there is an entry in the SST log file for each actigraphy recording.

Now, the IS values display a significant change for the recordings found in the SST log file:

rawReader.IS()

{'example_02': 0.5237758757150337,
 'example_03': 0.8722002132735109,
 'example_01': 0.8290854646594275,
 'example_01_mask': 0.44565929945478583}

The actigraphy recordings have been truncated according to the start and stop times specified in the SST log file.

Et voilà! Easy, isn’t it?