"It's high time (that) we left " means, we are still in the room, but in our mind's eye, we see ourselves in the past, as having already left, and that's why "left" is used. "left" refers to the past, which is where the thoughts are: We should have left before now; We shouldn't have stayed until now.