I went from a 3 to an X. If sporty driving/handling is more important to you, then I would say stay with the Y. At my stage of life, I enjoy the comfort of the X. Also, I really like having a display directly in front of me.
If you have not done so already, you should test drive the X. Either way, enjoy the drive!