I'm the son that Payne mentioned above who attends a non-ivy "prestigious" university. When looking at graduate programs, you need to know what you're getting yourself into. Are you looking for a professional degree (e.g., MPA, JD, MSW, MD), a research-based master's program, or direct admission for a PhD? You need to have a clear vision of what you're looking for and how the degree will help you--not only to increase your competitiveness in your statement of purpose, but so that you don't waste thousands (or hundreds of thousands) and years of your life on the wrong program. If you're not sure yet, do yourself a favor after undergrad: get a job while you figure your plans out and don't waste years and thousands chasing prestige to no end.

The kind of degree program is also very important. You might be looking for a law degree, for instance. In that case, your undergraduate institution isn't really that important--it's about a sky-high GPA and a high LSAT score. If you're looking for another type of professional or academic degree, you might find that the Ivies don't offer the best program in that field. And if you're looking for a research degree (particularly a PhD), high academic performance and test scores are expected, but you're more or less precluded from admission without research experience, great recommendations from faculty (who have supervised you in research-related activities), and have attended a school of recognized academic standing (which a decent number of state schools are).

I wonder in part if you're even sure what the Ivy League is--yes, it's a group of selective schools, but it's ultimately a sports conference with a shared history. Plenty of fantastic schools--Stanford, MIT, UChicago, Emory, Vanderbilt, Duke, Northwestern--are not part of the Ivy League.