Oops! Southern manners?

I volunteered in William's class Wednesday morning. First, there was a substitute, so it was not a "normal" day, and I was there, so it was extra exciting for William. I don't know if he is as silly as he was Wednesday all the time or if it was the uniqueness of the day, but I caught William being abrupt with his answers to the sub. It struck me he was not saying "yes, ma'am" or "no ma'am" and I felt I had fallen down on my training. First, I know it is just not done in New Jersey, but I was surprised at the tone. It may have been a bit of showing off for me, but yesterday at lunch, Joe and I practiced with him some more. The boys hear "no, sir" and "yes,sir" because I use it with them, but as I am the only girl in the house, they don't hear the ma'am much.