That is a great time for killing interest in mathematics; unfortunately particularly once you get into long division and the hints of algebra there's a focus on the rules and procedures of things, and forgetting the point of things. People who really grok mathematics don't get to do much teaching at those levels, to everyone's loss.

Perhaps coincidentally it was just after elementary school that my interest in playing music diminished. The only objectively observable event I can tie this to is going to a new school where I was demoted from First Violin to Second Violin, but I remember thinking the rather less shallow reason that while I was all right in playing most of the notes on schedule, I was just following procedures (I'm gifted in procedures) rather than playing music, if you get that difference.