Prev Next

Chapter 15. Code Coverage Analysis

You have learned how to use unit tests to test your code. But how do you test your tests? How do you find code that is not yet tested -- or, in other words, not yet covered by a test? How do you measure testing completeness? All these questions are answered by a practice called Code Coverage Analysis. Code Coverage Analysis gives you an insight into what parts of the production code are executed when the tests are run.

PHPUnit's Code Coverage Analysis utilizes the statement coverage functionality provided by the Xdebug extension. An example of what statement coverage means is that if there is a method with 100 lines of code, and only 75 of these lines are actually executed when tests are being run, then the method is considered to have a code coverage of 75 percent.

Let us generate a code coverage report for the BankAccount class from Example 13.3.

phpunit --coverage-html ./report BankAccountTest
PHPUnit 3.4.2 by Sebastian Bergmann.

...

Time: 0 seconds

OK (3 tests, 3 assertions)

Generating report, this may take a moment.

Figure 15.1 shows an excerpt from a Code Coverage report. Lines of code that were executed while running the tests are highlighted green, lines of code that are executable but were not executed are highlighted red, and "dead code" is highlighted orange. The number left to the actual line of code indicates how many tests cover that line.

Figure 15.1. Code Coverage for setBalance()

Code Coverage for setBalance()


Clicking on the line number of a covered line will open a panel (see Figure 15.2) that shows the test cases that cover this line.

Figure 15.2. Panel with information on covering tests

Panel with information on covering tests


The code coverage report for our BankAccount example shows that we do not have any tests yet that call the setBalance(), depositMoney(), and withdrawMoney() methods with legal values. Example 15.1 shows a test that can be added to the BankAccountTest test case class to completely cover the BankAccount class.

Example 15.1: Test missing to achieve complete code coverage

<?php
require_once 'PHPUnit/Framework.php';
require_once 'BankAccount.php';
 
class BankAccountTest extends PHPUnit_Framework_TestCase
{
    // ...
 
    public function testDepositWithdrawMoney()
    {
        $this->assertEquals(0, $this->ba->getBalance());
        $this->ba->depositMoney(1);
        $this->assertEquals(1, $this->ba->getBalance());
        $this->ba->withdrawMoney(1);
        $this->assertEquals(0, $this->ba->getBalance());
    }
}
?>


Figure 15.3 shows the code coverage of the setBalance() method with the additional test.

Figure 15.3. Code Coverage for setBalance() with additional test

Code Coverage for setBalance() with additional test


Specifying Covered Methods

The @covers annotation (see Table B.1) can be used in the test code to specify which method(s) a test method wants to test. If provided, only the code coverage information for the specified method(s) will be considered. Example 15.2 shows an example.

Example 15.2: Tests that specify which method they want to cover

<?php
require_once 'PHPUnit/Framework.php';
require_once 'BankAccount.php';
 
class BankAccountTest extends PHPUnit_Framework_TestCase
{
    protected $ba;
 
    protected function setUp()
    {
        $this->ba = new BankAccount;
    }
 
    /**
     * @covers BankAccount::getBalance
     */
    public function testBalanceIsInitiallyZero()
    {
        $this->assertEquals(0, $this->ba->getBalance());
    }
 
    /**
     * @covers BankAccount::withdrawMoney
     */
    public function testBalanceCannotBecomeNegative()
    {
        try {
            $this->ba->withdrawMoney(1);
        }
 
        catch (BankAccountException $e) {
            $this->assertEquals(0, $this->ba->getBalance());
 
            return;
        }
 
        $this->fail();
    }
 
    /**
     * @covers BankAccount::depositMoney
     */
    public function testBalanceCannotBecomeNegative2()
    {
        try {
            $this->ba->depositMoney(-1);
        }
 
        catch (BankAccountException $e) {
            $this->assertEquals(0, $this->ba->getBalance());
 
            return;
        }
 
        $this->fail();
    }
 
    /**
     * @covers BankAccount::getBalance
     * @covers BankAccount::depositMoney
     * @covers BankAccount::withdrawMoney
     */
 
    public function testDepositWithdrawMoney()
    {
        $this->assertEquals(0, $this->ba->getBalance());
        $this->ba->depositMoney(1);
        $this->assertEquals(1, $this->ba->getBalance());
        $this->ba->withdrawMoney(1);
        $this->assertEquals(0, $this->ba->getBalance());
    }
}
?>


Ignoring Code Blocks

Sometimes you have blocks of code that you cannot test and that you may want to ignore during code coverage analysis. PHPUnit lets you do this using the @codeCoverageIgnoreStart and @codeCoverageIgnoreEnd annotations as shown in Example 15.3.

Example 15.3: Using the @codeCoverageIgnoreStart and @codeCoverageIgnoreEnd annotations

<?php
class Sample
{
    // ...
 
    public function doSomething()
    {
        if (0) {
            // @codeCoverageIgnoreStart
            $this->doSomethingElse();
            // @codeCoverageIgnoreEnd
        }
    }
 
    // ...
}
?>


The lines of code between the @codeCoverageIgnoreStart and @codeCoverageIgnoreEnd annotations are counted as executed (if they are executable) and will not be highlighted.

Including and Excluding Files

By default, all sourcecode files that contain at least one line of code that has been executed (and only these files) are included in the report. You can configure the sourcecode files that are included in the report using the PHPUnit_Util_Filter API (see Table 15.1).

Table 15.1. The PHPUnit_Util_Filter API

MethodMeaning
void addDirectoryToFilter(string $directory)Adds all files from a directory that are suffixed with .php to the blacklist (recursively).
void addDirectoryToFilter(string $directory, string $suffix)Adds all files from a directory that are suffixed with $suffix to the blacklist (recursively).
void addFileToFilter(string $filename)Adds a file to the blacklist.
void removeDirectoryFromFilter(string $directory)Removes all files from a directory that are suffixed with .php from the blacklist (recursively).
void removeDirectoryFromFilter(string $directory, string $suffix)Removes all files from a directory that are suffixed with $suffix from the blacklist (recursively).
void removeFileFromFilter(string $filename)Removes a file from the blacklist.
void addDirectoryToWhitelist(string $directory)Adds all files from a directory that are suffixed with .php to the whitelist (recursively).
void addDirectoryToWhitelist(string $directory, string $suffix)Adds all files from a directory that are suffixed with $suffix to the whitelist (recursively).
void addFileToWhitelist(string $filename)Adds a file to the whitelist.
void removeDirectoryFromWhitelist(string $directory)Removes all files from a directory that are suffixed with .php from the whitelist (recursively).
void removeDirectoryFromWhitelist(string $directory, string $suffix)Removes all files from a directory that are suffixed with $suffix from the whitelist (recursively).
void removeFileFromWhitelist(string $filename)Removes a file from the whitelist.


The blacklist is pre-filled with all sourcecode files of PHPUnit itself and the tests. When the whitelist is empty (default), blacklisting is used. When the whitelist is not empty, whitelisting is used. When whitelisting is used, each file on the whitelist is added to the code coverage report regardless of whether or not it was executed.

Prev Next
1. Automating Tests
2. PHPUnit's Goals
3. Installing PHPUnit
4. Writing Tests for PHPUnit
Test Dependencies
Data Providers
Testing Exceptions
Testing PHP Errors
5. The Command-Line Test Runner
6. Fixtures
More setUp() than tearDown()
Variations
Sharing Fixture
Global State
7. Organizing Tests
Composing a Test Suite Using the Filesystem
Composing a Test Suite Using XML Configuration
Using the TestSuite Class
8. TestCase Extensions
Testing Output
9. Database Testing
Data Sets
Flat XML Data Set
XML Data Set
CSV Data Set
Replacement Data Set
Operations
Database Testing Best Practices
10. Incomplete and Skipped Tests
Incomplete Tests
Skipping Tests
11. Test Doubles
Stubs
Mock Objects
Stubbing and Mocking Web Services
Mocking the Filesystem
12. Testing Practices
During Development
During Debugging
13. Test-Driven Development
BankAccount Example
14. Behaviour-Driven Development
BowlingGame Example
15. Code Coverage Analysis
Specifying Covered Methods
Ignoring Code Blocks
Including and Excluding Files
16. Other Uses for Tests
Agile Documentation
Cross-Team Tests
17. Skeleton Generator
Generating a Test Case Class Skeleton
Generating a Class Skeleton from a Test Case Class
18. PHPUnit and Selenium
Selenium RC
PHPUnit_Extensions_SeleniumTestCase
19. Logging
Test Results (XML)
Test Results (TAP)
Test Results (JSON)
Code Coverage (XML)
20. Build Automation
Apache Ant
Apache Maven
Phing
21. Continuous Integration
Atlassian Bamboo
CruiseControl
phpUnderControl
22. PHPUnit API
Overview
PHPUnit_Framework_Assert
assertArrayHasKey()
assertClassHasAttribute()
assertClassHasStaticAttribute()
assertContains()
assertContainsOnly()
assertEqualXMLStructure()
assertEquals()
assertFalse()
assertFileEquals()
assertFileExists()
assertGreaterThan()
assertGreaterThanOrEqual()
assertLessThan()
assertLessThanOrEqual()
assertNotNull()
assertObjectHasAttribute()
assertRegExp()
assertSame()
assertSelectCount()
assertSelectEquals()
assertSelectRegExp()
assertStringEndsWith()
assertStringEqualsFile()
assertStringStartsWith()
assertTag()
assertThat()
assertTrue()
assertType()
assertXmlFileEqualsXmlFile()
assertXmlStringEqualsXmlFile()
assertXmlStringEqualsXmlString()
PHPUnit_Framework_Test
PHPUnit_Framework_TestCase
PHPUnit_Framework_TestSuite
PHPUnit_Framework_TestResult
Package Structure
23. Extending PHPUnit
Subclass PHPUnit_Framework_TestCase
Assert Classes
Subclass PHPUnit_Extensions_TestDecorator
Implement PHPUnit_Framework_Test
Subclass PHPUnit_Framework_TestResult
Implement PHPUnit_Framework_TestListener
New Test Runner
A. Assertions
B. Annotations
@assert
@backupGlobals
@backupStaticAttributes
@covers
@dataProvider
@depends
@expectedException
@group
@outputBuffering
@runTestsInSeparateProcesses
@runInSeparateProcess
@test
@testdox
@ticket
C. The XML Configuration File
PHPUnit
Test Suites
Groups
Including and Excluding Files for Code Coverage
Logging
Test Listeners
Setting PHP INI settings, Constants and Global Variables
Configuring Browsers for Selenium RC
D. Index
E. Bibliography
F. Copyright