Hmmm...having lived in the hearts of PAC10, BIG12, and now SEC country, I think the worst is the PAC10. The BIG12 and SEC fans have been very respectful, but don't really care about BYU. PAC10 people seemed to either love or hate us. Less in-between. I think the PAC10 was the worst from a BYU fan perspective, but it was my favorite place to live.
A funny side story about SEC rivalry--I live in Tuscaloosa, AL right now. The LSU game a couple of weeks ago was unlike any atmosphere I've been in. Anyway, my wife is in a local shop the day of that game and some LSU fans walk in wearing their purple and yellow gear. The woman helping her who owned the store said, "Hey, no LSU fans in here!" My wife gave a friendly laugh and so did the LSU fans. Then the store owner said, "I'm not joking, get the h--- out of my store!" To top it off, after the LSU fans left, the owner said, "I hope they get hit by a truck."