Should I have called the Sheriff?

I was building a deck at my parents house today. At around 3pm I heard screaming coming from the neighbors house. I new this was the Dad screaming at a kid. I had actually heard this in the past. This went on and off for the next 2 hours. I finally called my sister to see what I should do. She said to call the cops. As soon as I started to look for the non emergency number I heard the kid and Dad playing in their backyard together. So I never called. This jerk started out as a foster parent and has adopted the 4 year old and is the guardian of this boys little brother/sister. I couldn't imagine growing up with this jerk yelling like I heard today.