He was married three times, firstly in 1359 to Blanche of Lancaster, the daughter of the aforementioned Henry of Grosmont, secondly in 1371 to Constance of Castile, and thirdly in 1396 to Kathryn Swynsford.
It is Gaunt's relationship, and eventual marriage to Kathryn Swynsford that he is mostly remembered. This relationship resulted in four children who would bear the name of Beaufort. These children and their descendants would be prominent players in the Wars of the Roses.