I've only ridden in the S once so I can't really comment much on S vs ES350. The S is definitely cushier than the 3 with the longer wheelbase being the primary contributing factor. Also keep in mind the tires on your ES350 may play a big role as well depending on tire make/type, sidewall height, etc. If your ES had oem 17" tires, the sidewalls are slightly taller on those compared to oem 19" model Y tires so that also makes a difference.
Are you getting used to the Y's ride or still feel it's not to your comfort needs?
My wife is driving the Y. I drive my 2007 ES350 even when she does not need it. I loved winter tires on ES as they felt cushier but louder. Yes, ES tires are stock size with taller sidewalls. Can I lower tire pressures on Y to get an idea of potential improvement?