I'd rather go & I'd rather you went
This expression means 'would prefer to', and is followed by the infinitive without to. We often use the contraction 'd rather.
I'd rather stay home. (Would you rather stay home or go out?)
I'd rather not go out.
We can use would rather to say that a person would prefer somebody to do something. We use a special structure with a past tense.
would rather + subject + past tense
I'd rather YOU stayed home.
I'd rather YOU didn't go out.