These sound like folks with a lot of time on their hands. There would be nothing wrong with just calling and saying that something "came up" and that you have to bow out for now. You could use the work excuse you describe. Don't feel bad. Actually, you don't owe them any excuse, but I know it can feel awkward. Give yourself credit for at least checking the group out. Check other things out, and you'll eventually find something that clicks with you.