I know a lot of people have done this.
At first, they are worry about losing all benefits but turns out that they don't. They are just gonna lose a bit, depends on how much they work. Also, they realize losing that little bit of benefits is nothing from a long term plan perspective. As they work, the amount they earn from a part time gives them more money and flexibility in long terms, and they like it more this way. So it's not everyone's cup of tea.
You can always go back to full SSDI if your part time doesn't work out. Hope this helps.