Rolan, you know by now that I tend to go against the flow.....

Yes, in some places it matters whether a person went to an Ivy League school, but there are many "Poison-Ivy" league schools out there that are quite good as well. One of my sons goes to a state university and the other goes to a prestigious university in the top 15 or so nationally. (My grad school ranks well up there, too, just behind some Ivies.) He is thinking of trying to transfer to an Ivy League school, but largely because the one he is interested in has a program that appeals to him. He does have the grades and the SAT scores.
In the meantime, you are going to have to buckle down and make those grades, even if you aren't sure what you what to do. Ivies expect high grades at the minimum. And remember that they are more demanding. Plus, they also cost a great deal, being private schools. Even my son's prestigious school costs over $50,000 a year! Most grad schools don't offer financial aid.
I hope I am not bursting your bubble. I'm not saying, "Don't shoot for that goal." I am saying, "Just be realistic going in to the situation." Okay? I'm sure you are quite capable. Can you see each course as a challenge and work hard at it?