LYS import now works with any timestamp column in raw data, as long as it starts with timestampadd_states() no longer creates duplicate observations at identical start/endpoints. The default behavior is now that the start is inclusive and end is exclusive. Adjust the new bounds argument for more options.extract_metric() give more helpful error messages when using a base dataset without start/end columns.
mean_daily() can output the number of days that factor into the calculations by setting .n = TRUE.
summary_table(), summary_overview(), and summary_metrics() are a great set of new functions that give quick summaries across the whole dataset. Care has to be taken with these functions, as they are high level and will not perform well with a higly fragmented/irregular dataset.
sample_groups() is a new function that makes it easy to reduce the number of groups either by random sampling, flexible ordering, or based on a condition.
style_time() is a new convenience function that takes datetimes, times, or numeric input and outputs a clean time format (e.g., "03:45"). This is primarily used to style times in tables or plots.
format_coordinates() is a new convenience function that takes a coordinates and formats them nicely for plots and tables.
Datetime2Time() now allows for circular time through a sine-conversion of time of day - this is especially useful for averaging of times that (can) cross midnight, such as bed-times.
Circular2Time() is a new function that back-converts circular time columns to hms time. See the package {circular} for details on how circular columns work.
remove_partial_data() allows to specify a minimum duration of available data. Simply supply a negative duration to the threshold.missing argument. E.g., "-20 hours" will only keep groups with at least 20 hours of data. While this was easy before in case of groups of known total duration, 24 hours (e.g., simply set the threshold to 4 hours to get to 20 hours of data), it was not possible for groups of unknown total duration.
import functions now support a version argument. If there are multiple known formats for one supported device, the version can be changed. As of now, this is the case with the VEET device, which changed its format slightly with v2.1.17. This argument is also used for the Actiwatch Spectrum, which requires several adjustments for a German locale (beyond simple adjustment of locale settings). #65
supported_versions() is a new function that provides an overview which device versions are supported.
add_Date_col() gained the as.count argument. If set to true, it will output the number of days since start. The basis are calendar days.
add_states() now
start and end variables to be of class Interval - this makes the function ready to work with output from sc2interal() or sleep_int2Brown().force.tz = TRUE, i.e., the timestamp in the states dataset is forced to the timezone of the receiving time-series dataset. That is useful, e.g., when you know that the timestamp is correct, but was imported ad UTC by default.Added many new resources to the documentation webpage (accessible through the nav menu), including an interactive online course for LightLogR.
filter_Datetime() & filter_Date():
length is provided, but not start nor end, the functions now respect grouping, i.e., the length will be taken from the first record (or last in the case of from_start = FALSE) within each group.length being the first argument taken, then start and endgg_states() now
ymin and ymax argumentsextract.metrics argument.Datetime column through the Datetime.colname argument.gg_photoperiod() now
ymin and ymax argumentsDatetime column through the Datetime.colname argument.The plotly package was moved to suggested dependency, as it only covers an edge case.
The janitor package is no longer a dependency, as a simpler version to find duplicates during import was implemented.
VEET devices import much faster now, thanks to an efficient way to construct the data table. Thanks to @ThomasKraft for raising this issue! #66
gg_states() replaced the function gg_state() for more consistent naming with other states functions.
gg_day() and gg_days() now have the y.axis variable and the geom as first two arguments, putting the most often used arguments to the front.
New device import: MiEye from Circadian Health Innovations. There are two known datetime formats for the device: ymd HMS, and dmy HMS. Both are parsed.
More flexible import for ActLumus devices: Data can start at any line. For computational efficiency, it will determine the correct starting row in the first file provided (filenames[1]) and use it for all files provided for import.
log_zero_inflated() and exp_zero_inflated() have an updated reference.
Brown_check() now also takes factor vectors for state. This affects the Brown upstream functions that use Brown_check().
gg_heatmap() has gained a facetting variable to remove facetting altogether. Default is TRUE.
Standard y.axis.label of visualization functions is now Melanopic EDI (lx). Affects gg_day(), gg_days(), and gg_heatmap().
gg_day(): standard x.axis.label is Local time (HH:MM).
aggregate_Date() and aggregate_Datetime() now contain a warning for ... about partial matching of argument names.
sleep_int2Brown() will sensibly fill in values for columns in the evening, should the state dataset contain more than Interval and Sleep columns.
added a Newsletter section to the package page
added a test-coverage workflow and badge
extract_metric() throws no error if the original data is already grouped by the identifying.colname
gap_table() no longer throws an error if the package gt is not installed. gt is now added as a dependency.
dose() is a new metric function to calculate light dose (or any other kind of dosage)
add_Date_col() is a new convenience function to add a Date column to the dataset, optionally showing the weekday.
Datetime2Time() is a new convenience function that is used in other functions that average over datetimes, which is often more sensible over times.
reworked the README file to reflect some of the features LightLogR has gained over time
summarize_numeric() now calculates total_duration correctly, even when the prefix is removed.
added the sample.data.irregular internal dataset
removed LYS wearable sample file, due to package size limitations
add_Time_col() replaces create_Time_data(), also, the new column is called Time by default instead of Time.data
extract_clusters() has the option to show the cluster condition in the output with add.label = TRUE, e.g., MEDI>500|d≥30min|i≤5min for clusters of melanopic EDI larger than 500, at least 30 minutes long (d), allowing interruptions of up to 5 minutes at a time (i).
add_clusters() now drops empty groups, which has led to warnings before
Added many more unit tests - 888 and counting!
Removed a nasty bug in the internal functions that could lead to a shift in how dominant epochs are assigned to durations when groups are dropped due to singular observations
gapless_datetimes(): fixed a bug that prohibited the use of durations. Had downstream effects on basically all gap functions
extract_gaps() now warns correctly if there are implicit gaps with a non-default epoch
import has the option to remove data before a specified date. Default is not.before = 2001-01-01. Some devices fall back to a time stamp in the year 2000 after the battery drained completely. This makes the import problematic in terms of gap searching.
extract_states() has the option not to group by the extracted state.
extract_clusters() and extract_states() do not drop empty groups, which is important for summaries. extract_clusters() does it by default, extract_states() does not.
summarize_numeric() has the option to show zero-instances of groups. Helpful to make certain groups with zero instances are not dropped, especially in a chain with mean_daily()
mean_daily(): Automatic conversion to weekdays from dates. Further it has the option to replace NA with zeros before calculating mean daily values
This is a huge update for LightLogR, bringing many new features and twenty-two new functions
spectral_reconstruction():reconstruct a spectral power distribution (SPD) from sensor channels that provide (normalized) counts, and a calibration matrix. Examples for such devices include the the ActLumus from Condor instruments, or the VEET from Meta Reality labs (after normalization of counts, e.g., through normalize_counts())
alphaopic.action.spectra: New dataset containing alphaopic action spectra (CIE S026) plus the photopic action spectrum in 1-nm wavelength steps.
spectral_integration(): integrate over all or just parts of the spectrum, including the option to weigh the spectrum with an action spectrum (e.g., from alphaopic.action.spectra)
remove_partial_data(): remove groups with a below than user-specified amount of data
extract_gaps(): provides a start and end times, as well as durations for all gaps in the dataset.
has_gaps(), has_irregulars(); provide a logical feedback on whether a dataset has (implicit) gaps or irregular data.
gap_table() provides a comprehensive summary of available and missing data.
durations(): calculate the groupwise duration of a dataset, based on datapoints, the dominant interval, and missing data
mean_daily() and mean_daily_metric(): give a three-row summary of weekday, weekend, and mean daily (numeric) values. mean_daily_metric() skips the prior metric calculation for duration-based metrics, and directly calculates the mean daily value.
extract_clusters(), add_clusters(): find clusters of a user-specified condition and either summarize them or add them to a dataset.
extract_states(), add_states(): provides a summary of every state in the dataset or add them to a dataset.
extract_metric(): add a calculation to extracted data, such as from extract_state() or extract_clusters().
summarize_numeric()/summarise_numeric(): calculate means across numeric values, ideal to summarize results from extract_state(), extract_gaps(), or extract_clusters.
Brown_cut(): divide light exposure variables into sections ≤1lx, ≤10lx, and ≥250lx according to Brown et al. 2022
log_zero_inflated() and exp_zero_inflated(): apply or reverse a logarithmic transformation after adding a small value to a vector so as to provide zero values in logarithmic transformation, which is especially important for light exposure.
gg_gaps(): visualize gaps and shows instances of irregular data.
gg_states(): is an addon-function to gg_day() or gg_days(), which adds a state or cluster indicator to the plot
gg_heatmap(): visualize a condensed version of time series patterns, optionally as double plots.
Import support for the Clouclip device.
number_states() added the option to just output a count number without the original state
gg_days(): jco_color = TRUE is now the default
import_Dataset() no longer changes a pre-existing Id column (if it is not called Id). The function is also more informative for the daylight savings time handling in files with more than one Id.
gg_photoperiod() does no longer throw an error when the main plots y.axis is not based on a MEDI column.
gapless_Datetimes() now ignores groups with only a single measurement, instead of throwing an error. This function is the basis for all calculations regarding gaps.
changed the behavior of all metric functions that calculate an in-between value (such as the duration of light between 250 and 1000 lux). Up until now, the function would use inclusive bounds on both sides, i.e. the value of 250 lux and 1000 lux would be included in the calculation. This is now changed to right exclusive bounds, i.e. the value of 250 lux will still be included in the calculation, whereas 1000 lux will not. While of little practical difference in a realistic dataset (where exact values matching the threshold are likely not present), it is relevant when calculating, e.g., the time spent in various levels of light or any other variable. The sum of those times should always add up to the total time. With inclusive bounds on both sides, the sum could theoretically be larger, with right exclusive bounds it cannot.
Metrics intradaily_variability() and interdaily_stability() now use the population variance (divide by N), instead of the sample variance (divide by N-1). The legacy behavior can still be accessed by setting the argument use.samplevar = TRUE. #55
Metric IS now correctly uses the overall mean in relation to the variance, instead of the mean of hourly averages across days. #56
passed 300 unit tests for LightLogR 🎉
normalize_counts() was added as a low-level helper function and the accompanying dataset gain.ratio.tables to facilitate calculating normalized sensor values when comparing across different sensors, e.g. to assess daylighting conditions based on UV, IR, and photopic sensing ranges. See documentation for more infos.
Update to the import function of GENEActiv devices, based on input from the author of the GGIR package. The timezone tz argument in LightLogR now is just set on the timestamp provided by the GGIR export, instead of shifting the datetime. This requires the correct setting of desiredtz/configtz arguments in GGIR during preprocessing.
refinement and cross-referencing in tutorials on photoperiod
fixing a bug in the photoperiod family of functions when using a timezone with a large offset to the coordinates where photoperiod is calculated and crosses a date.
added a suite of functions to deal with photoperiod: photoperiod() and solar_noon() to calculate dusk, dawn, and noon times of the day. extract_photoperiod() and add_photoperiod() utilize datasets imported with LightLogR to calculate and deal with photoperiods in the context of your own datasets. gg_photoperiod() brings this functionality to the visualization tools of LightLogR in an easy and powerful way.
added the function number_states() that relabels states based on their non-consecutive appearance. This is especially useful when labelling photoperiod states, as the function will allow for an easy classifier of "day 1", "day 2", ..., and "night 1", "night 2", .... These can be used to, e.g., calculate metrics for individual photoperiod sections throughout the observed time frame.
added a tutorial on the new functions Photoperiod. This also details how to calculate metrics based on photoperiod (#39).
implemented further changes in the paper.md based on the JOSS Reviews
implemented changes based on the JOSS Reviews
added a Code of Conduct and a Contributing file for the project
updated the license to MIT: LightLogR is now permissively licensed
import functions will now give a warning message about identical observations in the provided data files, stop the import process and return a tibble with the duplicate rows. Through the remove_duplicates parameter, the user can decide to automatically remove these duplicates during import. Note: identical observations refers to identical rows when disregarding the filename.
added support for OcuWEAR devices
added support for MotionWatch 8 devices #32
added support for LIMO devices
added support for GENEActiv devices, when data was preprocessed with the GGIR package. The function import$GENEActiv_GGIR() takes the GGIR output and imports it with LightLogR naming schemes. #27
release on CRAN!
changed the supported.devices list to a function supported_devices() instead, so the documentation automatically updates with the list of supported devices. Similarly, ll_import_expr is now ll_import_expr().
added support for the Meta VEET device for visual experience measurements
added support for the Kronowise device
added support for the MPI melanopiQ Circadian Eye (Prototype)
rewrote the import function for Actiwatch_Spectrum, as the sample file the original was based off, had specific formatting to German standards. Now, the German version can still be called through Actiwatch_Spectrum_de, wheras the main function refers to the english/international format.
updated the landing page for the website with a list of supported devices and a table of metrics
small changes to documentation
Changes to the tutorial articles on the website
Integration of a community survey on the website and Github Readme.
bright_dark_period() now maintains the date when looping the data.
Added articles on Import & Cleaning, Metrics, and Visualizations to the website.
Added the option for more print rows of observation intervals during import.
Added the option to set a length for the dataset starting from the end in filter_Datetime() and family.
Added the function aggregate_Date() to aggregate long datasets to one day per group.
New function gg_doubleplot() for ... well, double plots.
added import functions for nanoLambdaand LightWatcher devices
new Logo!
fixed bug in interval2state() that would dismiss the first state if it starts before the actual data
fixed a bug in interval2state() that would add other columns then the State column present in the interval dataset to the output dataset, but leave them empty. Added an example that shows how to add multiple columns to the output dataset correctly.
in aggregate_Datetime(), added the option to set the dominant.epoch, i.e., the most common interval, as the unit parameter, to effectively deal with irregular data.
Added the functions dst_change_summary() and dst_change_handler() to detect and deal with Daylight Savings. The functionality is also integrated into the import functions, so that a user can automatically apply it during the import process.
Added Steffen Hartmeyer as a collaborator, who added a number of light metrics from the lightdosimetry package.
Added the import_adjustment() function for more flexibility when importing light logger data that does not conform to the standard format. This goes hand in hand with the ll_import_expr list that contains specific expressions for all supported devices.
lots of bug fixes and improvements
Bugfix for LiDo import
Added import support for new devices: LiDo, DeLux, and Speccy
Removed minor inconsistencies in naming conventions. Also, all imported columns will have syntactic naming now
Added an option to all gap functions, to extend the gapless Datetime range to full days.
Exports the up to now internal function count_difftime() that is the basis for dominant_epoch(). But whereas the latter gets only the most common epoch, count_difftime() returns a table with the counts of all epochs. This is useful in conjunction with gap_finder(), to check the distribution of data intervals.
Added the gg_days() function to visualize multiple days of data in a single plot. Alongside come two helper functions, Datetime_limits() and Datetime_breaks(), to set the limits and breaks of the x-axis.
Added the filter_Datetime_multiple() function to filter for multiple Datetime ranges depending on certain conditions, e.g. different filter cutoffs for different participants. It wraps around filter_Datetime() or filter_Date().
Reworked the internals of the light logger data import functions. They now use a more straightforward function factory approach. For users the only visible change it that device specific functions now have the form import$device() instead of the old import.device().
Added the symlog_trans() function from a post on stack overflow. This function leads to a better visualization of light logger data, as a logarithmic transformation is necessary, but values of 0 are common. The function was integrated as a default for gg_day() and will likely be the basis of upcoming visualization functions.
Added the aggregate_Datetime() function to aggregate data to a given time interval.
Added the gg_overview() function to get a sense for the timeframe of measurement data.
Added the family of regularize functions to find and deal with implicit missing data. These functions include dominant_epoch(), gapless_Datetimes(), gap_handler(), and gap_finder().
A ton of updates to documentation, unit tests, and bug fixes.
Added Unit tests and documentation for all new functions.
To filter_Datetime() and filter_Date() added the option to filter for group specific dates.
Added the family of functions around States and Reference to import, process, and add states to light logger data, like sleep/wake times, wear times, or other data. This family includes import_Statechanges(), sc2interval(), ìnterval2state(), data2reference(), sleep_int2Brown(), Brown_check(), Brown_rec(), and Brown2reference().
Added the Article/Vignette "What´s in a Day" to demonstrate the LightLogR workflow.
Added the convenience function create_Timedata() to create a Time-of-Day column in datasets.
Added the family of filter_Datetime(), filter_Date() and filter_Time() functions to easily filter datasets.
Added unit tests for the first functions.
Added several helper functions to work with states like sleep or wear times.
Added an automatic ID creation at import and streamlined the import functions.
Added the function join_datasets to combine imported datasets with sensible constraints.
Added major grid marks for the y-axis.
Added a message when using start or end dates to make it clear, that only the Date portion of the input will be used.
Changed the behavior, when there is already a Day.data column present in the data. It will only create a new column if none is present, otherwise it will use the existing column for faceting (after factorization)
Added the option to create an interactive plot by feeding the plot to the [plotly] package.
NEWS.md file to track changes to the package.