10. Maintenance

10.1. TACTIC Backup and Restore

Like any other data producing service, TACTIC requires a backup regimen. The data in that TACTIC produces and manages must be backed up in accordance with the backup policy of the departments that manage the TACTIC service.

TACTIC managers should create a backup policy for TACTIC data. A backup policy is extremely important, as data loss within the TACTIC service is catastrophic.

TACTIC manages data in three parallel scopes:

  • versioned assets Versioned assets are all the files that are checked in and out of the tactic service.
  • asset metadata Asset metadata are all the data produced by user interaction with the tactic service.
  • project metadata Project metadata are all data produced by the creation of a project within tactic the tactic service.

Tactic data is all of equal importance. The data is all interrelated, and thus all data must be treated with equal importance in the backup regimen.

The frequency of backup of tactic data depends on the policies of the TACTIC managers. An examination of the frequency of use of the tactic service, combined with the down time potentially required to back up tactic data should be considered.

the only consideration that tactic managers should take into account when creating a backup policy for tactic data is that tactic data produced is synchronized. This is accomplished via a transaction system within tactic. This transaction system relies on co-services to store transacted data. During the backup process, tactic and its co-services should be stopped to prevent any loss of synchronization to these co-services.

*Versioned assets*Versioned assets are all the files that are checked in and out of the tactic service. Typically the tactic service works in conjunction with a co-service such as a network file storage service to accomplish its tasks of asset management. Thus the task of backing up tactic of versioned assets onto these systems usually overlaps with the file management policies of the IT department. Most enterprise class network file storage systems, and third party file backup systems can accomplish the task of backing up tactic assets. Redundant raid storage, snapshots, and offsite backups can all be utilized to accomplish version and asset backups.

It is highly recommended that you back up your TACTIC server using a regular schedule. In the event of a hardware failure, you will be able to restore your TACTIC server fully from the latest backup if you have taken the following three backup steps:

  1. Back up the assets directory
  2. Dump the TACTIC database
  3. Back up the TACTIC program directory

Note

You may want to stop the tactic service while you do your backup process. To do so, run the following commands:

service tactic stop

service tactic start

The assets directory is the repository where TACTIC stores all of the asset files. By backing up this directory, you can restore all of your asset files in the event of a database crash into a clean TACTIC-managed directory structure.

Note

You may need to contact your TACTIC server administrator if your assets directory has been redirected to another location.

Windows

The assets directory on a Windows install is located by default in:

C:\assets

Linux

The assets directory on a Linux install is located by default in:

/home/apache/assets

To restore the TACTIC assets directory, you have to restore your backup to the current assets location.

To dump the TACTIC data base you need to log in to the PostgreSQL database and perform a database dump.

Note

For both Linux and Windows the database file is in the assets directory, although you may redirect the location to your backup location. Also, the TACTIC default for the database is to have no password. If you have added a password, you may need to enter it into your postgres commands with the -P tag.

Windows

pg_dumpall -U postgres -c > c:\assets\tacticDatabase.sql

Linux

pg_dumpall -U postgres -c > /home/apache/assets/tacticDatabase.sql

To restore the TACTIC database run the command:

psql -U postgres < tacticDatabase.sql

This last step, although not completely necessary, is recommended because it takes a snapshot of your TACTIC source code and project settings at the time of backup.

Windows

The folders to back up are:

C:\Program Files\southpaw\tactic

C:\Program Files\southpaw\projects

Linux

The folders to back up are:

/home/apache/tactic

/home/apache/projects

To restore the TACTIC program files, you have to restore the tactic and projects directories to their original locations.

10.2. Database Backup Automation

Overview

The TACTIC database needs backing up as part of a routine maintenance on the TACTIC system.

Procedure

  1. Log into the TACTIC server as root
  2. Make a backup of the existing /etc/crontab -the original crontab should look something like this:

    SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin
    MAILTO=root
    HOME=/
    
    # run-parts
    01 * * * * root run-parts /etc/cron.hourly
    02 4 * * * root run-parts /etc/cron.daily
    22 4 * * 0 root run-parts /etc/cron.weekly
    42 4 1 * * root run-parts /etc/cron.monthly
  3. The following line is an example of code that will write the date out to a file and run on the 15th minute of every hour:

    15 * * * * root date > /root/out

    Modify this example code to run in the next few minutes and add it to the crontab. To see if the cron script ran, after the time has passed, verify that the script wrote out the time to the file` /root/out`. The following line is an example of how to backup the TACTIC database every day at 3:45am yum

    45 3 * * * root /usr/bin/pg_dumpall -U postgres > /tmp/my_tactic_db_my_date

    Note

    Be careful as this example will overwrite the backup every day . Another more clever way is the name the file with the date:

    45 3 * * * root /usr/bin/pg_dumpall -U postgres > /root/my_tactic_db_`date +\%Y-\%m-\%0e_\%H:\%m:\%S`

    or

    45 3 * * * root my_date=date +\%Y-\%m-\%0e_\%H:\%M:\%S; /usr/bin/pg_dumpall -U postgres > /tmp/my_tactic_db_${my_date}

    Important

    *From crontab manpage:*The “sixth” field (the rest of the line) specifies the command to be run. The entire command portion of the line, up to a newline or % char- acter, will be executed by /bin/sh or by the shell specified in the SHELL variable of the cronfile. Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input.

By using Task Scheduler can schedule tasks such as to automate DB backup to run at a time that is most convenient. Task Scheduler starts each time Windows is started, and runs as a background process. With Task Scheduler, you can: Schedule a task to run daily, weekly, monthly.

To add a scheduled task:

  1. Create a windows batch script to back up the database
  2. Click Start→Control Panel→Scheduled Tasks→Add scheduled Task
  3. Create the task.

    image 4. Use the provided python script to dump the database.

    image 5. Run the task according to the schedule required.

    image 6. image 7. At the concluding stage of the scheduled tasks wizard, be sure to check the "Open advanced properties for this task when I click Finish"

    image

    Edit the command line to read "<path to python>\python dbbackup.py", and click "OK"

10.3. TACTIC logs

In a default installation tactic stores its logs in a single file. For proper maintenance, this file must be rotated by the host operating system. This action must be set in the host operating system. As tactic has no facility to control the log files it makes as of yet, it may be necessary to configure an external tool to rotate logs.

The host operating system must have a facility to rotate text logs on a regular chronological basis.

Logrotate is used as a tool to rotate text files on a chronological basis. The tool allows automatic rotation, compression, removal and mailing of log files.

Directives

This is a partial list of logrotate directives.

  • missingok: If the log file is missing, go on to the next log file without issuing an error message.
  • copytruncate: Truncate the original log file to zero size in place after creating a copy, instead of moving the old log file and optionally creating a new one
  • rotate 7 : Log files are rotated 7 times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather then rotated.
  • compress: Old versions of log files are compressed with gzip to save disk space.
  • notifempty: Do not rotate the log if it is empty
  • sharedscripts postrotate /etc/init.d/lighttpd reload endscript:

Default configuration

Logrotate’s main configuration file /etc/logrotate.conf

Applies the main configuration settings from this file to all log rotation settings, unless overridden by individual directives on a per-log basis.

# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
rotate 1
}

Example Service Configuration

This is an example logrotate configuration for HTTP. The configuration file is `/etc/logrotate.d/httpd `

/var/log/httpd/*.log {
 weekly
 rotate 52
 compress
  missingok
  notifempty
  sharedscripts
  postrotate
      /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true    endscript
}

On the example linux system, service or server specific configurations are stored in /etc/logrotate.d directory. For example here is sample apache logrotate configuration file:

In this example, HTTP logs are not deleted until they are at least a year old, and are rotated on a weekly basis. There are other configuration options available, but they will not be discussed in this tutorial. Please look online for more information on these other options. Depending on the requirement, each log file may be handled at different intervals.

The lines between postrotate and endscript (both of which must appear on lines by themselves) are executed after the log file is rotated. These directives may only appear inside a log file definition. In our case we are reloading lighttpd.

Example TACTIC configuration

The configuration goal is get logrotate to get the tactic log files rotated for our needs and requirements.

This is an example logrotate configuration for HTTP. The configuration file is `/etc/logrotate.d/tactic `

Remember to restart the logrotate services after any additions to logrotate configuration.

/home/apache/tacticTemp/log/stdout.log {
    notifempty
    daily
    rotate 365
    missingok
    copytruncate
    create 0600 apache apache
}