PHPUnit and Laravel Config Caching

This week at work, I ran into a fun issue with a colleague while running some unit tests that I wanted to document here.

I was writing some new unit tests for a legacy codebase that's been around a long time. For the sake of portability, I decided to use an in-memory SQLite database. After writing the tests and finishing the other work for the ticket, I opened a PR and went about my other tasks.

My colleague pulled down my ticket for testing and ran the tests. What should have been a very simple test was failing! What was weirder, is that the failing action was happening during one of the database migrations. The error isn't important but for the sake of completeness, but it was a foreign key constraint error on the database.

After trying to debug stuff with them, we also noticed that his local database had been entirely nuked. Considering the test was running with the RefreshDatabase trait, I was starting to get an idea of what was going wrong.

What happened?

The empty database was the big clue that led me down the rabbit hole. I knew that running the database migrations is a slightly different process for in-memory vs. a real database, so I dug into that trait first.

This method in the call stack stuck out to me:

protected function usingInMemoryDatabase()
{
	return config('database.connections')[
		config('database.default')
	]['database'] == ':memory:';
}

If the config value is for the default database is :memory: then this would return true. Since PHPUnit overrides your .env when the tests are ran, this should be the correct value. Dumping it however, we get a different result.

For the config() function to be returning a different result than what is in a configuration file already, that must mean a config cache file exists. You can read more about how Laravel potentially includes that file here.

In our case, there was a config cache present, so when Laravel was booting to run the tests, that config was being used instead of PHPUnit's environment variables.

To tie it back to the error we were seeing, since the local MySQL database was being used instead of the in-memory database, a migration with some outdated seeding code was throwing the foreign key constraint error. This is also a good reminder that MySQL and SQLite are not the same! If you are expecting foreign key constraint failures to be enforced in your tests, use the appropriate database for both your test suite, and when running your application.

REad More about SQLite Quirks

The Solution

If this case, I decided it was best to try and fix the problem for the user running the tests. I modified the base TestCase.php as follows.

<?php

namespace Tests;

use Illuminate\Support\Facades\Hash;
use Illuminate\Contracts\Console\Kernel;
use Illuminate\Contracts\Foundation\Application;
use Illuminate\Foundation\Testing\TestCase as BaseTestCase;

abstract class TestCase extends BaseTestCase
{
    /**
     * Creates the application.
     *
     * @return \Illuminate\Foundation\Application
     */
    public function createApplication()
    {
        $app = require __DIR__ . '/../bootstrap/app.php';

        $app->make(Kernel::class)->bootstrap();

        Hash::driver('bcrypt')->setRounds(4);

        $this->clearCache($app); // Added this line.

        return $app;
    }

    protected function clearCache(Application $app)
    {
        // We don't have a cached config, so continue running the test suite.
        if (!$app->configurationIsCached()) {
            return;
        }

        $commands = ['clear-compiled', 'cache:clear', 'view:clear', 'config:clear', 'route:clear'];
        foreach ($commands as $command) {
            \Illuminate\Support\Facades\Artisan::call($command);
        }
        // Since the config is already loaded in memory at this point, 
        // we need to bail so refresh migrations are not ran on our
        // local database.
        throw new \Exception('Your configuration values were cached and have now been cleared. Please rerun the test suite.');
    }
}

If no config cache file is found, then we're golden! Continue running the test suite. If we do find a config file, we run a few Artisan commands to clear all of those files and then throw an Exception. The exception is necessary since the config is already loaded into memory at this point. If the tests continued to run, all of the requested config values would be the old ones. While it may be possible to reset the config and continue running the tests, the utility of that is miniscule. Instead, we inform the user of the action that was taken and ask them to run the test suite again.

In an ideal world, you would never have a configuration cached locally as it's main purpose is to speed up expensive disk operations in production, but alas, we don't live in an ideal world! 😅

I want to give a shoutout to Joel Clermont for helping me debug what was going on and being a great rubber 🦆!

Posted in:

Laravel