Comments like that are usually ignored by guests anyway!

Flowers or a potted plant are a "token" gift. This happened to me many years ago Invited to former neighbours' wedding anniversary lunch, they just wanted everyone's company. Needless to say, guests turned up with gift vouchers/
flowers/plants/wine. My aunt had made a beautiful card in lieu of a gift. Feeling embarrassed, my mother apologised for the "lack of gift". Former neighbour commented that she wished other guests had taken notice, as they were serious about the message.

No, it's not too late.