I faced the same dilemma several years ago. To be honest, I would not expect to get much fishing done around moving water if you have a four-year old in tow. You pretty much have to keep your eyes on them at all times if there are slick rocks and any degree of current involved. If you don't mind skipping rocks and turning them over looking for insects/crayfish/salamanders with them, then you will fare better. You might be able to make a cast here and there and catch a few fish. If you have another adult along, you might get to do a little more fishing. My nine-year old son is just now getting to the point that he can wade a trout stream with me and cast to the easier spots. He can even catch a few fish. My five and three-year old daughters are still very much in the splashing and rock throwing stages. They have no respect for or fear of moving water at this age, which can be a real problem.
Oh, welcome to the board by the way.