That is not a very complete or accurate statement of what the Supreme Court has said about the constitutional source of an implied constitutional right to personal privacy. The justices have never reached a lasting consensus about where this mysterious right is located in the Constitution, and some, like Justice Thomas, are unable to locate any such right anywhere in the Constitution. As the Court noted in Roe v. Wade,
In varying contexts, the Court or individual Justices have, indeed, found at least the roots of that right [to personal privacy] in the First Amendment, Stanley v. Georgia, 394 U. S. 557, 564 (1969); in the Fourth and Fifth Amendments, Terry v. Ohio, 392 U. S. 1, 8-9 (1968), Katz v. United States, 389 U. S. 347, 350 (1967), Boyd v. United States, 116 U. S. 616 (1886), see Olmstead v. United States, 277 U. S. 438, 478 (1928) (Brandeis, J., dissenting); in the penumbras of the Bill of Rights, Griswold v. Connecticut, 381 U.S. at 484-485; in the Ninth Amendment, id. at 486 (Goldberg, J., concurring); or in the concept of liberty guaranteed by the first section of the Fourteenth Amendment, see Meyer v. Nebraska, 262 U. S. 390, 399 (1923). (emphasis added) 410 U.S. 113, 152 (1973).
The Court in Meyer and Roe associated the implied right to privacy with the liberty guaranteed as part of due process, but in the other decisions listed it did not. And I don't think the word "liberty", as used in the Due Process Clause of the Fourteenth Amendments, has a meaning nearly as broad and vague as what you suggest, or as the majority said it did in Meyer and Roe (or, for that matter, in Obergefell.)
In his dissenting opinion in Obergefell, Justice Thomas discussed the historical meaning of due process, from the time of Magna Carta through 1868, when the Fourteenth Amendment was ratified. He presents compelling evidence that the liberty a person might not be deprived of without due process of law had throughout those centuries been, and still was in 1868, chiefly understood to mean physical liberty. The concern was about persons being seized or detained by agents of government without fair legal process--being informed of the charges against you, given a hearing, and so on.