Using CircleCI and Splunk AppInspect*

Recently worked on writing and open sourcing https://github.com/splunk/TA-osquery. One of my goals was to make sure my code was in compliance with Splunk’s best practices at every stage of development. No one wants to get first wind of all of the issues in his or her app while they're submitting to Splunkbase.com, am I right?

Because a few of us were going to be working on that repo/app, automated testing was the way to go. I thought it would be great to share the experience with you, in case you’d also like to start automatically testing your Splunk content (App, TA, DA, SA).

Pan camera over to CircleCI, a CI/CD cloud service that allows you to “automate your pipeline from commit to deploy”. First I spent a few minutes going through the “getting started” process and ended with an example job running on in repo. The next step was to figure out what tool to use to test Splunk automatically. Fortunately, Splunk provides AppInspect as a way for customers to automatically validate their applications. Its description:

“Splunk AppInspect evaluates your Splunk app against a set of Splunk-defined criteria so that you can be assured of its quality and robustness.”

The next task was to build a job to automatically run AppInspect on every pull request for our TA-osquery. This was the end result:

Here, you can see that CircleCI executes three jobs: grab appinspect, install appinspect, and run appinspect. Let’s unpack these a bit:

• The job grab appinspect is straightforward. Using curl grabs its latest build and untars it under the ~/appinspect-latest directory.
• Next, install appinspect uses pip to install the necessary tools and virtualenv. We use virtualenv to install appinspect and its dependencies.
• Finally, run appinspect runs appinspect and returns an error if your app does not pass. It also prepares the repo code to ensure it does not return obvious errors.

Now, let’s look at a full example configuration from the TA-OSquery repo:

splunk/TA-osquery

Note the validate-content job where we clone, build, and run Splunk Appinspect. There is also a publish-github-release job, which takes the inspected tarball (TA-osquery.tar.gz) and creates a Github release using ghr that includes it. Also, under the workflows section, we define a very simple process:

• All pull request (CI jobs) run the validate-content job
• The publish-github-release job only runs on tags that start with v (eg. v1.0) and it requires the validate-content job to run successfully.

This allows us to only generate a release when we tag one in our code:

Mind you, this does require you to set an environmental variable in CircleCI named GITHUB_TOKEN with a Github token. The Github token must have access to post releases to your repo.

On the Github side, while not required, I highly suggest you create the following branch protection rule for all your branches (*****):

Require at least 1 reviewer for each pull request and always require that the validate-content job finishes successfully before allowing merging.

To summarize, in order to automatically test Splunk content in Github, you need to:

1. Sign-up and configure CircleCI with Github
2. Modify TA-osquery job and save to .circleci/config.yml in your repo
3. Configure branch protection Github

Using pre-commit hooks, specifically, those for linting configuration files and flake8 for python scripts, will help with the code hygiene of any project. Also, if you see yourself editing the CircleCI config file, it is good practice to use their cli tool to validate the file