We actually had a good 1st Q and a lousy 2nd Q. They adjusted on both their O and D, and BYU, in typical fashion (which is also true for most people), felt that if they were winning, they didn't need to do anything different than they were doing. Improving when you're winning is the most difficult thing to do. Bronco knows that, at least intellectually, but it's not clear that Anae does. It's really tough to get young kids to see that and buy in. But that's what elite opponents do--they continue to find and exploit additional weaknesses.
Taysom's contribution was more than yards and points. It deflated the team; demoralized them. T was so well-integrated that even if Stew had been as effective as Taysom on the field (and he wasn't at first), that the team was rudderless. It took four games against mediocre-at-best opponents to find enough identity to go forward such that when we lost JSwag, we were still OK.
But yeah, the biggest issue was the defense. Even if Howell did make some adjustments in the second half (the run D held their LB/RB to only 2.75 ypc in the second half), he couldn't overcome the soft mental status of the D. The LB/RB he was enough of a distraction that it threw BYU off its game. That and the sparkly uniforms. Evidence of weak mental prep. That's why I'm more optimistic about this year: Wintrich and Bronco will have the D with a better mental state.
The D was abused and exploited more and more and more as the season went along, culminating with Boise, then the competition eased up and the numbers got better, but they ultimately melted against the middling offense of Memphis.
But, to the OP's point, I don't think USU losing a bunch of their D will have much of an effect on this game per se, but it might have an effect on their season. This will be a good test to see if the new-found success of Logan is soft and shallow or deep. We'll know that way before the last game of the season.