If you're okay with the added acidity in a sauce, that would be fine, but for baking I would recommend looking for a recipe that calls for yogurt or buttermilk.
As far as I know, coconut milk does not curdle like dairy or other nutmilks might in the presence of acid or heat. It might separate, but should re-emulsify well enough.
However, baking usually ...
I once overloaded a little too much chicken stock in my homemade tomato soup. I found a good large tin of coconut milk made all the difference. plus it was creamy and highly delicious. Problem solved (with an added cup or two or water).😊Salt down to a minimum.