Migrating Data and Merging Models in Laravel

On one of my side projects, Let Them Eat 🍰 , I recently needed to do some migrations to combine / merge two models. I originally had optimized a bit too much, and later realized things would be a lot simpler if I only had one model.

There wasn't a ton of info about this online, so I'm going to do my best to try and explain what I did. For this migration, I was using Laravel 8.

So turns out, merging models is kind of a pain! In my case, I was merging a SlackUser::class model with the default User::class model that ships with Laravel. From a data perspective, it wasn't too bad. I needed to add a few columns to the User table that were previously on the SlackUser table. The issues arose when I realized there were a lot of places in my code that were reliant on accessing each model off of a relationship of the other. So lots of calls to $user->slackUser and $slackUser->user intermingled across the app, dependent on what I was doing at the time.

Since my ultimate goal was to completely delete any reference to a SlackUser, I had to take a careful approach when modifying the database.

First, I added the extra columns I needed to the users table.

Schema::table('users', function (Blueprint $table) {
  $table->string('slack_user_id')->after('id')->nullable();
  $table->foreignId('team_id')->after('slack_user_id')->nullable()->constrained()->onDelete('cascade');
  $table->boolean('is_owner')->after('team_id')->default(false);
  $table->string('avatar_url')->after('email')->nullable();
  $table->string('timezone')->after('avatar_url')->nullable();
  $table->boolean('is_onboarded')->after('timezone')->default(false);
  $table->softDeletes();
});

I continued in much the same process for other tables that needed to be modified.

After the tables were modified, I queried all of the SlackUsers and looped over them to create new User models if they didn't have one already. In my app, a User model was only created if the person logged into the webapp, otherwise they would happily live on as only a SlackUser. Now, everyone gets a user model, and I don't really care if they log in or not!

Some advice on the Shifty Coders Slack recommended not to rely on Eloquent here. This makes sense as in a future release, I'll be completely removing the SlackUser model, so if I relied on Eloquent for the migration, it would throw an error if I ever deleted that class. Here's what the migration looked like:

DB::table('slack_users')->get()->map(
	function ($slackUser) {
		$userId = $slackUser->user_id;
		$slackUser = Arr::except((array) $slackUser, ['id', 'created_at', 'updated_at', 'user_id']);
		$user = User::findOrNew($userId);
		if (!$user->exists) {
			$user->name = $slackUser['slack_user_id'];
			$user->password = Hash::make(random_bytes(20));
		}
		$user->fill($slackUser);
		$user->save();
	}
);

This is all pretty straightforward. For each SlackUser, check if they have a User model already, and if not, create a new User and give them a random password. My app doesn't actually use passwords for authentication, instead relying solely on Slack Oauth, so the password field is irrelevant. In the future I may want to allow for other auth methods, so I left it for the sake of simplicity.

After all of the users were migrated, I could then go about the business of updating other models that used the SlackUser as a foreign key. In my case, I had messed up in the original migrations and not enforced those foreign keys, but if they are enforced in your app, you'll need to drop the foreign key before you go about migrating all of this data around.

$table->dropForeign(['slack_user_id']);

Here is what the migration to change the foreign keys on my Cake model looked like.

DB::table('cakes')->get()->map(
	function ($cake) {
		$giver = DB::table('slack_users')->select('slack_user_id')->where('id', '=', $cake->giver_id)->get()->first();
		$giver = User::where('slack_user_id', '=', $giver->slack_user_id)->withTrashed()->first();
		$target = DB::table('slack_users')->select('slack_user_id')->where('id', '=', $cake->target_id)->get()->first();
		$target = User::where('slack_user_id', '=', $target->slack_user_id)->withTrashed()->first();
		DB::table('cakes')->where('id', $cake->id)->update(['giver_id' => $giver->id, 'target_id' => $target->id]);
	}
);

Again, notice that I'm not using Eloquent to access the SlackUser model, instead relying on the DB:class facade. I'm free to delete the SlackUser::class at anytime now!

All of this code was added to the up() method of my migration, and I carefully reversed all of the column changes for the down() method. One thing I did not do in the down method, was remigrate any data. I figured if the deploy went so bad that I needed to do that, then I would be better off restoring the database entirely from a backup instead. The down() changes I made were purely so I could migrate up/down for tests, which weren't reliant on any database values anyway.

That brings me to another pain point: tests! 90% of my tests had to be updated since they mostly relied on the SlackUser. I started with one test file at a time, running the entire file first, then each failing test. I generally changed any instance of SlackUser to User first and then saw what broke. The first few were painful as there were references to relationships that needed to be updated. Often times while doing this, I would catch something that needed to be updated in my migration as well.

Eventually, most of the methods from SlackUser were migrated to User and all of the relationships were updated. Views were the last thing to be checked and I even managed to add a few missing tests based on failures I found while manually browsing the site locally.

// One of my newly added tests!
/** @test */
public function a_user_can_view_the_perk_redemption_page() //phpcs:ignore
{
	$this->withoutExceptionHandling();
	$user = factory(User::class)->create([
	  'is_owner' => true
	]);

	// Make fake users for the manager selection.
	factory(User::class, 5)->create([
	  'team_id' => $user->team->id
	]);

	$this->actingAs($user);

	$user->team->perks()->create([
	  'title' => 'an image perk',
	  'cost' => 100,
	  'image_url' => 'https://perk-image.com/perk.jpg'
	]);

	$this->assertCount(1, Perk::all());

	$res = $this->get(route('redeem-perk-create', Perk::first()));
	$res->assertOk();

	// Check for all of the names on the page. (manager selection for perk redemption)
	$res->assertSee([...User::all()->map->name]);
}

In the end, the Github PR had 57 changed files! A huge undertaking by any standard. I would venture a guess that those 57 files represent 80-90% of all of the code I had written for the app.

Overall, I'm happy I did this, as the logic surrounding users is much simpler to understand. I also got the opportunity to try out some new things and learn a bit more about the built in DB facade. I'm still kinda intimidated by SQL in general, but I'm getting more courageous every time I tackle one of these projects. Backups are still really important though! 😉

Conclusion

If you have any questions, feel free to reach out to me on Twitter or leave a comment!

Posted in:

Laravel