Kaiser's older brother and sister left the house while he was still young, and his parents divorced when he was nine years old. Kaiser was active in Milwaukee's music scene, starting at the age of twelve, and was a member of more than twelve bands, leading two, before reaching nineteen years of age.