I would be surprised if the breeder didn't want the kitten back, when researching breeders for my Nyah and then my new little Kovucula, that was very clear from the outset. These kitties mean a lot to their breeders, and we had to "interview" for both of them. I didn't mind one bit, it reinforced I was getting a quality kitten from a quality breeder. In fact, when I started looking for my second, I had a tough time because I had spoken to a few (most on this board) and would have loved to give my business to all of them. Before I get to off track, I really hope your breeder helps you guys out.
I will tell you these guys love as hard as they play, and are just incredible cats overall. Yes, the energy is something to behold, and a few things get broken along the way, but you discover a sense of humor you never knew you had when your platinum rimmed mid century modern shot glass from a perfect set of 6 goes flying into a million pieces. Somehow a perfectly timed head butt and tractor sounding purr right after makes you really believe they are saying sorry. BUT this is a 20 year commitment, and a decision to bring the kitten back to the breeder is a much better one than being very stressed out for a long time and the cat ending up in a shelter. No one is happy to see a new kitten not have a forever home, of course, and I know that includes you. But no one wants a kitty in a less than ideal situation either, so you will find support in whatever decision is the right one for the kitten.