Family friendly spots for fishing?

We would like to do some fishing with the kids (6 and 4), but we have never done it and don't have any fishing gear. Any recommendation where and how to start? I wouldn't like to spend too much and we know we have to get the fishing licence.