New in version 6.1.0: See the new features.

Sculptor Audit service

An optional audit service has been implemented for kfserver/kfservnt or direct sage/srep stand-alone applications.

This service can be used to track down specifield keyed files or fields that have been modified, added or deleted. It can be used as a history of the keyed file changes, and know which user, program or machine has changed a keyed file. It doesn’t need the modification any source code, but to add more meaning into the audit information it can be adapted to the application (adding some specific application information) with a the new audit functions added to the language.

The audit service is implemented as a background specialized kfserver thread that uses a shared queue to receive the audit commands. That thread is the only one that really writes the audit* and transaction information. That allows a normal client not to have to wait for the audit information to be written on disk. It just “posts” a command in the queue and continues. so that the audit will not act as a bottleneck.

How it works

The audit service uses keyed files to configure the files and fields to audit. And then it inserts all the audit information into three keyed files:

  • transactions: used as the main entry point. Each transaction ideally refers to an “atomic” busines unit. See audit transactions for a deeper explanation

  • auditrecords: related to a transaction, a record is created for each audited file operation (the operation can be any configured combination of insert, write or deletion).

  • auditrefields: stores the new and old values (whatever is aplicable) of an audited field (refers back to an auditrecord)

As all the audit keyed files are closely linked (related), a special SQL Engine Schema has been created to make it easy to extract information.

Setting up the audit service

The setting up of the whole audit service can be done with the suplied $SCULPTOR/sbin/auditmgt program. This is the recomended way of configuring it, but can also be configured directly modifying JSON files and keyed files.

There JSON config files are used to configure the Audit service behaviour and with the filetable and filetablefields keyed files we specify the fields and files to be audited.

The file and fields to be audited will be checked only when opening each keyed file. So after opening a posible audited keyed file any chnage in configuration afecting that keyed file will not be active until reopening the keyed file.

On the filetable keyed file you specify the keyed files you want to audit and using the fileFlags field you can control the file to not be audited at all, choose the operations to audit (insert, update or delete) or if you want to specify in a field by field basis which field is going to be audited (“Only audit selected fields” option).

When you specify the “Only audit selected fields” option for a keyed file, is the only case where the filetablefields is used. In this case you can specify the fields you want to audit, which operations to audit or if you want to ignore changes to this field from being audited as record modifications (e.g.: record fields used like timestamps)

Where the audit information will be stored

An audit database can be specified, by default will be called “default”. This will provide the ability to have different audit databases inside a single sculptor system. The audit keyed files location will be (where auditdb is the chosen audit database name):

$SCULPTOR/audit/auditdb

For each audit database, there will be a number of keyed files.

The location can be changed using the Sculptor System File Relocation service (scsfr) using the management program $SCULPTOR/scsfmgt.q

Archive

The audit_switch_database feature makes possible to archive the audit information which may be huge in size. So that it can be backed up elsewhere, analised or just deleted if no longer needed.

The operation is done transparently and there is no need to stop the aplications or the audit service. As a result of the switch operation the previous audit data files will be copied in a correct/stable state, while the online or current audit keyed files will be “reset” maintaining all the needed records (active transactions…)