How does a joint major affects grad school?

So I am a Finance, Math-Econ joint major. And my idea is that after i get my bachelors, I would get a job in finance while continuing my eduction in math. But before you say that I'm crazy, I would only need to take 2 more classes than if I were a finance-math double major or finance-econ double major. But the problem I see is that in the joint major, i would have taken less elective classes in math than a full math major. So I was wondering how well I could survive in grad school with the core classes and little math elective compared to other math majors who have had probably more classes than me. So I guess I feel a little nervous since I don't know what grad school is like. So could someone tell me how my bachelors education will affect my masters and possible doctorate education in math.