I think without conclusive evidence you risk causing a rift in the family that may not be easily mended.
Although I am unfamiliar with the relationship you have with your brother.
What do you hope to achieve by voicing your beliefs?
What would you do once you told him?...I.e , go to the police?
How would things transpire if you turned out to be wrong?

Just a few things I feel might be worth considering.
It's a tough situation no doubt, that requires some thought.