Version 1.43.0

This version adds optional web logging to record user access. A new table log has been added to the bigsdb_auth database that records IP address, username, and page called with a timestamp. In addition, the REST API logging to the bigsdb_rest database has been improved by the addition of client and username information for when authenticated access is used.

To enable logging, you need to set the following in bigsdb.conf:

web_log_to_db=1
rest_log_to_db=1

Logging requires writing to the database on each page access so there is a very small performance penalty to enabling this. The tables are however unlogged (i.e. data are not written to the PosgreSQL write-ahead log) which makes them considerably faster than ordinary tables but data in them will be lost in the event of a database crash or unclean shutdown.

As every page access is recorded the log tables will grow in size over time. It is recommended that they are pruned regularly to remove records older than a specified period of time - this may also be required by GDPR! The easiest way to do this is to set up a scheduled CRON job by adding the following to /etc/crontab:

0  *  *  *  *  postgres psql -c "DELETE FROM log WHERE timestamp < NOW() - INTERVAL '7 days'" bigsdb_rest > /dev/null
10 *  *  *  *  postgres psql -c "DELETE FROM log WHERE timestamp < NOW() - INTERVAL '7 days'" bigsdb_auth > /dev/null

Additionally, there is a fix to the locus stats function in the sequence typing database necessitated by the introduction of schemes that can include locus presence/absence in their definitions. You therefore need to update the bigsdb_rest, bigsdb_auth, and any sequence definition database using the rest_v1.43.sql, auth_v1.43.sql, and seqdefdb_v1.43.sql scripts respectively. See https://github.com/kjolley/BIGSdb/blob/develop/Upgrading_README.txt.

Note that in the unlikely event that you have defined schemes that utilize locus presence/absence (where profiles can include the 'P' designation to indicate 'presence'), rather than just sequence variation, you should update the locus stats within the typing database with the following commands, otherwise the reported minimum allele length may be wrong.

DELETE FROM locus_stats;
INSERT INTO locus_stats(locus,datestamp,allele_count,min_length,max_length) 
SELECT loci.id,MAX(sequences.datestamp),COUNT(sequences.allele_id),MIN(LENGTH(sequence)),MAX(LENGTH(sequence)) 
FROM loci LEFT JOIN sequences ON loci.id=sequences.locus 
WHERE allele_id NOT IN ('N','0','P') OR allele_id IS NULL 
GROUP BY loci.id;

Full Changelog: v_1.42.7...v_1.43.0