OS/Shell

The official command line interface for interacting with Airbrake.

The OS/Shell integration is done via the airbrake CLI using the capture command.

CLI Capture Command#

The CLI Capture command is described briefly in the CLI doc with instructions for how to install the CLI and run it manually. This doc expands on this feature set to give more OS-level and DevOps use cases for the capture.

For more use DevOps-focused cases and details for how to implement them, see Shell Capture for IT/DevOps Use

Aliases#

To make things easier when running commands manually, add the ID/KEY exports to your .bash_profile (or equivalent), along with an alias, such as:

alias aa="airbrake capture ansible-playbook"

It will be a process change to use aa instead of ansible to run your ansible commands, but if this becomes established practice it will be easier than typing ansible-playbook when running your ansible commands

Instrumenting a Shell Script#

Many shell scripts have the following pattern:

AWS=/usr/local/bin/aws
$AWS some aws command
some other command
some third command
$AWS some other aws command

Where you may not really care to wrap the entire script, but rather just parts of it. This could be done simply by changing the line which defines the AWS command:

AWS=/usr/local/bin/airbrake capture /usr/local/bin/aws

As always, ensure the $AB_PROJECT_ID and $AB_PROJECT_KEY variables properly defined/available to the script

Wrapping Systemd Services#

Wrapping a systemd service is as easy as prepending the airbrake capture to the command as above in the ExecStart line of the service file. An example of a ruby service (resque):

ExecStart=/usr/local/bin/airbrake \
  --project-id $AB_PROJECT_ID \
  --project-key $AB_PROJECT_KEY \
  --environment production capture --no-stdout -- /bin/bash -l -c "/usr/local/bin/bundle exec rake resque:worker COUNT=1 QUEUE=$QUEUE TERM_CHILD=1 RESQUE_TERM_TIMEOUT=9 LOGFILE=/var/log/resque_worker.log"

If you use ansible, it may be useful to have this only in certain server types (eg, some redises but not all), so you can modify your template as such:

{% if inventory_hostname in groups.ab_redis_resque %}
ExecStart=/usr/local/bin/airbrake capture --project-id {{ airbrake_cli_project_id }} \
  --project-key {{ airbrake_cli_project_key }} \
  --environment {{ env }} -- {{ redis_path }} /etc/redis/redis_{{ redis_port }}.conf
{% else %}
ExecStart={{ redis_path }} /etc/redis/redis_{{ redis_port }}.conf
{% endif %}

Capturing Server Cronjobs#

Again, this is as simple as changing the entry in /etc/crontab or /etc/cron.d/* to the following:

0 0 * * 6 postgres /usr/local/bin/airbrake capture --project-id $AB_PROJECT_ID --project-key $AB_PROJECT_KEY --no-stdout /var/lib/postgresql/backup_full.sh >> /var/log/postgresql/pg_backup.log 2>&1
Note that the redirects are fine, as they are redirecting the output of the `airbrake` command, as the output from `backup_full.sh` gets laundered through the `airbrake` command and captured for the notification, before being output again.  Essentially, `airbrake` acts like the `tee` command.