They don't decide whether to start driving before everyone is buckled up, but the driver does (over here it's an infraction of the traffic code to drive onto a public road if someone is not buckled up, and that's a decision the driver makes). That decision is not something the other passengers can make.Then the other adult passengers should care (& hence be fined) too
Of course the other passengers should care, but they are not responsible in a legal sense. The driver is.