Ubisoft went with the industry standard for fighters in P2P and it never quite worked despite their efforts, for some reason xbox got it worse than PC and PS4, things have improved a fair bit from the start. As was said above, dedicated servers will be coming in the next few weeks and that should hopefully fix a lot of the issues.
Xbox user
The game seems to have issues sometimes when you quit the game. I seem to notice longer search times for games, latency symbols, and problems getting games with friends after logging back in and seeing the error message that your session has ended. This all seems to go away by shutting down the game.
Random Thoughts:
The latency symbols tend to pop up when this issue occurs. I think these may be due to other users or myself having the above problem. You could try a custom match vs ai and see if you still have the symbols. Verify your network connection (ping and packet loss) if the symbols are present in the custom match. If your connection is ok then try to reboot and see if the problem goes away.