It is not necessary to wait until the very end of the day to use your facial cleanser. If you are in for the rest of the evening, go ahead and begin your skin care regimen. This will help you avoid forgetting or making excuses (I'm too tired, etc.).
You should use a facial cleanser even if you do not wear anything on your face. During the course of the day, your skin accumulates dust, dirt and more. It's important to make sure you wash your face each and every night.