Closing out your oldest credit card immediately shortens credit history and negatively affects your credit score. Length of credit history accounts for approximately 15% of your credit score, and lenders naturally tend to view borrowers with short credit histories as riskier than those who have responsibly held credit for a longer period of time.
So, you will want to keep that first credit card around forever, and remember to use it from time to time. Some credit card issuers, such as Citibank, will automatically close down an account after a period of inactivity.