Fat being 3500 cals/pound, to lose one pound of fat in a week requires a 500 cal/day deficit, whether you get that from exercising, reduction of food, or a combo of both. 5 pounds would require a whopping 2500 cal/day deficit ! You'd have to pretty much each nothing AND exercise.
3 pounds requires a deficit of 1500 calories per day which is attainable but would certainly require a lot of effort. If your body requires 2000 cals/per day to maintain, you could achieve the 1500 cal/day deficit by eating 1000 cals/day and burning off another 500 through exercise. I would not be able to do this for more than a week !
Unless you are very very active, any loss in excess of 3 pounds is likely water, and perhaps a bit of muscle if you are not getting sufficient exercise.
That said, there is a HUGE psychological benefit to seeing your weight drop quickly. And even water loss alone will make you feel thinner. But one small slip in your diet or your body's own mysterious controlling mechanisms can cause you to retain water and make it appear that you have gained weight. Oh how many times have I gone through this ! It's why I don't want to weigh myself ! It's so depressing when you have done everything right and you feel thinner and yet the scale shows otherwise !