Put sugar, buttermilk, butter, and baking soda into large pot (it gets really frothy and expands so don't put it in a small sauce pan). Cook over medium to medium-high heat and bring to a boil. Stirring constantly, keep boil going for about 7 minutes until syrup has changed from white to golden brown, and started thickening. Remove from heat and stir in vanilla.
You can store the leftovers in the fridge and microwave them to get the syrup back to the right consistency.