If you are taking a tablet, try taking only half a dose (split the tablet). I only ever took half of a tablet. To reassure you, I just graduated from college in December, so I spent a few months in school while also on Phen. My memory is as fine as it ever was, but I have temporarily lost the ability to spell sometimes. I think that has a lot more to do with school, work, kids, etc... all at the same time than anything to do with Phen.
It's coming back, but the last week of my last quarter, I turned in a paper where I had spelled enforce, inforce. What the ????
Yes, it's tough with all the side effects, but it is very worth it, in my opinion. If not this, then what is your back up plan? I'd hate for you to give up altogether. If you are that worried about your memory, take plenty of notes or perhaps use a voice recorder during classes. I'm no spring chicken, I did it and I believe you can do it too!