Skip to content

Commit 5ff8ec6

Browse files
committed
CI: Initial unit test setup
* Adds PHPUnit to the `dev` requirements in the `composer.json` file. - As the PHPCS PHPUnit framework isn't compatible with PHPUnit 7 until PHPCS 3.2.3, I've chosen to explicitly only support PHPUnit 4, 5, 6 via Composer to prevent having to make extra allowances for PHPUnit 7 in the Travis script. - Includes raising the minimum PHPCS requirement from PHPCS `3.0.2` to `3.1.0` as PHPCS `3.0.2` does not yet contain the PHPUnit `bootstrap.php` file for cross-version PHPUnit compatibility. - Includes adding a convenience script to easily run the unit tests. * Adds a `phpunit.xml.dist` configuration file. * Adds a `phpunit-bootstrap.php` file to sort out the autoloading for the unit tests and to prevent PHPCS from trying to run unit tests for other installed external standards. * Adds an abstract base test case for the security sniffs which handles setting the configuration variables for the tests. The configuration variables are set based on the test case file name. See the explanation in the file docblock. * Adds running the unit tests to the Travis configuration. As the full unit test matrix can take a little while to run, I've set this up in a way that on pushes only a quick check is being run and that the full build is only run on PRs and merges to `master`. The updated build matrix takes compatibility of PHPCS with various PHP versions into account. Includes: * Adding the test related files to the `.gitattributes` `export-ignore` list so they don't get shipped in the release packages. * Adding typical PHPUnit overload/cache files to the `.gitignore` file.
1 parent 1359bce commit 5ff8ec6

File tree

8 files changed

+269
-6
lines changed

8 files changed

+269
-6
lines changed

.gitattributes

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@
88
/.gitattributes export-ignore
99
/.gitignore export-ignore
1010
/.travis.yml export-ignore
11+
/phpunit.xml.dist export-ignore
12+
/phpunit-bootstrap.php export-ignore
13+
/Security/Tests/ export-ignore
1114

1215
#
1316
# Auto detect text files and perform LF normalization

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
11
/vendor
22
composer.lock
3+
/phpunit.xml
4+
/.phpunit.result.cache

.travis.yml

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,30 @@ cache:
1313

1414
php:
1515
- 5.4
16-
- 7.4
17-
- "nightly"
16+
- 5.5
17+
- 5.6
18+
- 7.0
19+
- 7.1
20+
- 7.2
21+
22+
env:
23+
jobs:
24+
# `master`
25+
- PHPCS_VERSION="dev-master" LINT=1
26+
# Lowest supported PHPCS version.
27+
- PHPCS_VERSION="3.1.0"
1828

1929
# Define the stages used.
30+
# For non-PRs, only the sniff and quicktest stages are run.
31+
# For pull requests and merges, the full script is run (skipping quicktest).
32+
# Note: for pull requests, "master" is the base branch name.
33+
# See: https://docs.travis-ci.com/user/conditions-v1
2034
stages:
2135
- name: sniff
36+
- name: quicktest
37+
if: type = push AND branch NOT IN (master)
2238
- name: test
39+
if: branch IN (master)
2340

2441
jobs:
2542
fast_finish: true
@@ -48,6 +65,37 @@ jobs:
4865
- diff -B ./example_base_ruleset.xml <(xmllint --format "./example_base_ruleset.xml")
4966
- diff -B ./example_drupal7_ruleset.xml <(xmllint --format "./example_drupal7_ruleset.xml")
5067

68+
#### QUICK TEST STAGE ####
69+
# This is a much quicker test which only runs the unit tests and linting against the low/high
70+
# supported PHP/PHPCS combinations.
71+
- stage: quicktest
72+
php: 7.4
73+
env: PHPCS_VERSION="dev-master" LINT=1
74+
- php: 7.2
75+
# PHP 7.3 is only supported since PHPCS 3.3.1, PHP 7.4 since PHPCS 3.5.0, so running low against PHP 7.2.
76+
env: PHPCS_VERSION="3.1.0"
77+
78+
- php: 5.4
79+
env: PHPCS_VERSION="dev-master" LINT=1
80+
- php: 5.4
81+
env: PHPCS_VERSION="3.1.0"
82+
83+
#### TEST STAGE ####
84+
# Additional builds to prevent issues with PHPCS versions incompatible with certain PHP versions.
85+
# PHP 7.3 is only supported since PHPCS 3.3.1, PHP 7.4 since PHPCS 3.5.0.
86+
- stage: test
87+
php: 7.4
88+
env: PHPCS_VERSION="dev-master" LINT=1
89+
- php: 7.4
90+
env: PHPCS_VERSION="3.5.0"
91+
- php: 7.3
92+
env: PHPCS_VERSION="dev-master" LINT=1
93+
- php: 7.3
94+
env: PHPCS_VERSION="3.3.1"
95+
96+
- php: "nightly"
97+
env: PHPCS_VERSION="n/a" LINT=1
98+
5199
allow_failures:
52100
# Allow failures for unstable builds.
53101
- php: "nightly"
@@ -58,9 +106,35 @@ before_install:
58106

59107
- export XMLLINT_INDENT=" "
60108

109+
# On stable PHPCS versions, allow for PHP deprecation notices.
110+
# Unit tests don't need to fail on those for stable releases where those issues won't get fixed anymore.
111+
- |
112+
if [[ "$TRAVIS_BUILD_STAGE_NAME" != "Sniff" && $PHPCS_BRANCH != "dev-master" && "$PHPCS_VERSION" != "n/a" ]]; then
113+
echo 'error_reporting = E_ALL & ~E_DEPRECATED' >> ~/.phpenv/versions/$(phpenv version-name)/etc/conf.d/travis.ini
114+
fi
115+
116+
install:
117+
# Set up test environment using Composer.
118+
- |
119+
if [[ $PHPCS_VERSION != "n/a" ]]; then
120+
composer require --no-update --no-scripts squizlabs/php_codesniffer:${PHPCS_VERSION}
121+
fi
122+
- |
123+
if [[ "$TRAVIS_BUILD_STAGE_NAME" == "Sniff" || $PHPCS_VERSION == "n/a" ]]; then
124+
# The sniff stage doesn't run the unit tests, so no need for PHPUnit.
125+
# The build on nightly also doesn't run the tests (yet).
126+
composer remove --dev phpunit/phpunit --no-update --no-scripts
127+
fi
128+
61129
# --prefer-dist will allow for optimal use of the travis caching ability.
62130
- composer install --prefer-dist --no-suggest
63131

64132
script:
65133
# Lint PHP files against parse errors.
66-
- composer lint
134+
- if [[ "$LINT" == "1" ]]; then composer lint; fi
135+
136+
# Run the unit tests.
137+
- |
138+
if [[ $PHPCS_VERSION != "n/a" ]]; then
139+
composer test
140+
fi

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ phpcs-security-audit in its beginning was backed by Pheromone (later on named Fl
3131
Install
3232
-------
3333

34-
Requires [PHP CodeSniffer](http://pear.php.net/package/PHP_CodeSniffer/) version 3.x with PHP 5.4 or higher.
34+
Requires [PHP CodeSniffer](http://pear.php.net/package/PHP_CodeSniffer/) version 3.1.0 or higher with PHP 5.4 or higher.
3535

3636
The easiest way to install is using [Composer](https://getcomposer.org/):
3737
```bash
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
<?php
2+
/**
3+
* An abstract class that all Security sniff unit tests must extend.
4+
*
5+
* A sniff unit test checks a .inc file for expected violations of a single
6+
* coding standard. Expected errors and warnings that are not found, as well as
7+
* unexpected warnings and errors, are considered test failures.
8+
*
9+
* This class will take care of setting the configuration variables in PHP_CodeSniffer
10+
* needed to test all relevant configuration combinations for each sniff in
11+
* the Security standard.
12+
*
13+
* The configuration variables set are based on the file name of a test case file.
14+
*
15+
* Naming conventions for the test case files:
16+
* SniffNameUnitTest[.CmsFramework][.ParanoiaMode].inc
17+
*
18+
* Both `[.CmsFramework]` as well as `[.ParanoiaMode]` are optional.
19+
* If neither is set, the defaults of no CmsFramework and Paranoia level 0 will be used.
20+
*
21+
* Separate test case files for different paranoia levels and different frameworks are
22+
* only needed if the sniff behaves differently based on these settings.
23+
*
24+
* - If the sniff behaviour is the same all round, just having one plain `SniffNameUnitTest.inc`
25+
* test case file will be sufficient.
26+
* - If the sniff behaviour is only dependent on one of the two configuration settings,
27+
* the other can be left out.
28+
* Examples:
29+
* - Sniff behaviour only depends on `ParanoiaMode`: `SniffNameUnitTest.[01].inc`.
30+
* - Sniff behaviour only depends on `CmsFramework`: `SniffNameUnitTest.[CmsFramework].inc`.
31+
*/
32+
33+
namespace PHPCS_SecurityAudit\Security\Tests;
34+
35+
use PHP_CodeSniffer\Tests\Standards\AbstractSniffUnitTest;
36+
37+
abstract class AbstractSecurityTestCase extends AbstractSniffUnitTest
38+
{
39+
40+
/**
41+
* Get a list of CLI values to set before the file is tested.
42+
*
43+
* @param string $filename The name of the file being tested.
44+
* @param \PHP_CodeSniffer\Config $config The config data for the run.
45+
*
46+
* @return void
47+
*/
48+
public function setCliValues($filename, $config)
49+
{
50+
// Set paranoia level.
51+
$paranoia = substr($filename, (strlen($filename) - 5), 1);
52+
if ($paranoia === '1') {
53+
$config->setConfigData('ParanoiaMode', 1, true);
54+
} else {
55+
$config->setConfigData('ParanoiaMode', 0, true);
56+
}
57+
58+
// Set the CMS Framework if necessary.
59+
$firstDot = strpos($filename, '.');
60+
$firstOffset = ($firstDot + 1);
61+
$secondDot = strpos($filename, '.', $firstOffset);
62+
63+
$extendedExtension = '';
64+
if ($secondDot !== false) {
65+
$extendedExtension = substr($filename, $firstOffset, ($secondDot - $firstOffset));
66+
}
67+
68+
switch ($extendedExtension) {
69+
case 'Drupal7':
70+
$config->setConfigData('CmsFramework', 'Drupal7', true);
71+
break;
72+
73+
case 'Drupal8':
74+
$config->setConfigData('CmsFramework', 'Drupal8', true);
75+
break;
76+
77+
case 'Symfony2':
78+
$config->setConfigData('CmsFramework', 'Symfony2', true);
79+
break;
80+
81+
default:
82+
$config->setConfigData('CmsFramework', null, true);
83+
break;
84+
}
85+
}
86+
}

composer.json

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,22 @@
1111
],
1212
"require": {
1313
"php": ">=5.4",
14-
"squizlabs/php_codesniffer": "^3.0.2",
14+
"squizlabs/php_codesniffer": "^3.1.0",
1515
"dealerdirect/phpcodesniffer-composer-installer": "^0.4.1 || ^0.5 || ^0.6"
1616
},
1717
"require-dev" : {
1818
"php-parallel-lint/php-parallel-lint": "^1.0",
19-
"php-parallel-lint/php-console-highlighter": "^0.4"
19+
"php-parallel-lint/php-console-highlighter": "^0.4",
20+
"phpunit/phpunit": "^4.5 || ^5.0 || ^6.0"
2021
},
2122
"minimum-stability": "dev",
2223
"prefer-stable": true,
2324
"scripts" : {
2425
"lint": [
2526
"@php ./vendor/php-parallel-lint/php-parallel-lint/parallel-lint . -e php --exclude vendor --exclude .git"
27+
],
28+
"test": [
29+
"@php ./vendor/phpunit/phpunit/phpunit --filter Security ./vendor/squizlabs/php_codesniffer/tests/AllTests.php"
2630
]
2731
}
2832
}

phpunit-bootstrap.php

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
<?php
2+
/**
3+
* Bootstrap file for running the tests.
4+
*
5+
* - Load the PHPCS PHPUnit bootstrap file providing cross-version PHPUnit support.
6+
* {@link https://github.com/squizlabs/PHP_CodeSniffer/pull/1384}
7+
* - Allows for a `PHPCS_DIR` environment variable to be set to point to a different
8+
* PHPCS install than the one in the `vendor` directory to allow for testing with
9+
* a git clone of PHPCS in a local develop environment.
10+
* - Prevent attempting to run unit tests of other external PHPCS standards installed.
11+
*/
12+
13+
if (\defined('PHP_CODESNIFFER_IN_TESTS') === false) {
14+
\define('PHP_CODESNIFFER_IN_TESTS', true);
15+
}
16+
17+
$ds = \DIRECTORY_SEPARATOR;
18+
19+
/*
20+
* Load the necessary PHPCS files.
21+
*/
22+
// Get the PHPCS dir from an environment variable.
23+
$phpcsDir = \getenv('PHPCS_DIR');
24+
$composerPHPCSPath = __DIR__ . $ds . 'vendor' . $ds . 'squizlabs' . $ds . 'php_codesniffer';
25+
26+
if ($phpcsDir === false && \is_dir($composerPHPCSPath)) {
27+
// PHPCS installed via Composer.
28+
$phpcsDir = $composerPHPCSPath;
29+
} elseif ($phpcsDir !== false) {
30+
/*
31+
* PHPCS in a custom directory.
32+
* For this to work, the `PHPCS_DIR` variable needs to be set in a custom `phpunit.xml` file.
33+
*/
34+
$phpcsDir = \realpath($phpcsDir);
35+
}
36+
37+
// Try and load the PHPCS autoloader.
38+
if ($phpcsDir !== false
39+
&& \file_exists($phpcsDir . $ds . 'autoload.php')
40+
&& \file_exists($phpcsDir . $ds . 'tests' . $ds . 'bootstrap.php')
41+
) {
42+
require_once $phpcsDir . $ds . 'autoload.php';
43+
require_once $phpcsDir . $ds . 'tests' . $ds . 'bootstrap.php'; // PHPUnit 6.x+ support.
44+
} else {
45+
echo 'Uh oh... can\'t find PHPCS.
46+
47+
If you use Composer, please run `composer install`.
48+
Otherwise, make sure you set a `PHPCS_DIR` environment variable in your phpunit.xml file
49+
pointing to the PHPCS directory.
50+
';
51+
52+
die(1);
53+
}
54+
55+
/*
56+
* Set the PHPCS_IGNORE_TEST environment variable to ignore tests from other standards.
57+
*/
58+
$securityStandards = [
59+
'Security' => true,
60+
];
61+
62+
$allStandards = PHP_CodeSniffer\Util\Standards::getInstalledStandards();
63+
$allStandards[] = 'Generic';
64+
65+
$standardsToIgnore = [];
66+
foreach ($allStandards as $standard) {
67+
if (isset($securityStandards[$standard]) === true) {
68+
continue;
69+
}
70+
71+
$standardsToIgnore[] = $standard;
72+
}
73+
74+
$standardsToIgnoreString = \implode(',', $standardsToIgnore);
75+
\putenv("PHPCS_IGNORE_TESTS={$standardsToIgnoreString}");
76+
77+
// Clean up.
78+
unset($ds, $phpcsDir, $composerPHPCSPath, $allStandards, $standardsToIgnore, $standard, $standardsToIgnoreString);

phpunit.xml.dist

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<phpunit
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/6.5/phpunit.xsd"
5+
backupGlobals="true"
6+
bootstrap="./phpunit-bootstrap.php"
7+
beStrictAboutTestsThatDoNotTestAnything="false"
8+
colors="true"
9+
forceCoversAnnotation="true">
10+
11+
<testsuites>
12+
<testsuite name="Security">
13+
<directory suffix="UnitTest.php">./Security/Tests/</directory>
14+
</testsuite>
15+
</testsuites>
16+
</phpunit>

0 commit comments

Comments
 (0)