Yeah that's what I thought about the extra characters. I much prefer the Like a Dragon system of "the rest of your team is tagging along in the background" and magically gaining XP, even if it doesn't make much sense. I'll stick to the 2 parties of 4. By the time I finish the stories of the first 4 characters they should be in their 50s, and then moving to the new team I would expect to end up at the same point, having everyone around the same level by the time the finale comes up. Another advantage for me personally at least, is its doable for me to remember what's going on in 4 stories at a time. 8, not so much.
The game can be quite long, I agree! I've only played a handful of side quests, and some of them seem tricky to do without a guide.
I have unlocked and done one side story si far. It was a nice low-stakes story (only the first half so far). I like that kind of stuff mixed in with the "the world is about to end" stuff that's typical of a lot of JRPGs. Not as much of that here so far.
By the way, I skipped the first game. I read reviews of both and after reading that the second had more party interaction, I decided to just start there. If they make a third game hopefully it will be even more focused on the team going on an adventure together.