First, make sure your OS is correctly associating the file types with the desired program. Go to Start and type "set your default programs" (without the quotes). Click on the button under the Control Panel category with that name. On the menu to the left, find Windows Media Player -> Choose defaults for this program. Make sure they're all checked off. You can also do this for file associations by going to Start -> Default Programs -> Associate a file type or protocol with a program. These two do the same thing but from different directions.

If the above doesn't work, then you'll have to set Windows Media Player for your browser. The steps for this depends on the browser you're using.