Over the years the hotel was destroyed and re-built several times over. In 1821, when Captain William Becknell blazed the path of what would become known as the Santa Fe Trail, he stayed at a la fonda where the trail terminated at the town’s central plaza. As more and more pioneers traveled the Santa Fe Trail, the La Fonda became a popular destination for trappers, traders, mountain men, solders, politicians and the like. Soon after New Mexico became a U.S. Territory in 1848, the inn was purchased by Anglo-American owners who changed its name to the U.S. Hotel. The gambling Hall continued to be a major feature, however, providing entertainment for military officers and the occasional professional gambler. Fortunes were made and lost here, and one unfortunate person lost his life in 1857 at the end of a rope strung up in the hotel's backyard by a lynch mob.

Ten years later, in 1867, the Honorable John P. Slough, Chief Justice of the Territorial Supreme Court, was shot to death in the hotel lobby. Slough was in a dispute with Captain Rynerson, a member of the Territorial Legislature representing Dona Ana County, when he Slough called Rynerson a liar and a thief. The offended Rynerson then shot Slough, who died of his wounds. Though Rynerson was tried, he was later acquitted.

Around this same time, the hotel was sold again and became The Exchange Hotel, the name under which it operated for nearly six decades. Sometime during this period several tunnels were constructed underneath the hotel that lead to the courthouse.

More than a century ago, a distraught salesman who lost his company’s money in a card game, leaped to his death down a deep well that was located just outside the gambling hall of the Exchange Hotel.

The current La Fonda was built in 1922 on the site of the previous inns. In 1925 it was acquired by the Atchison, Topeka Santa Fe Railroad which leased it to Fred Harvey. For more than 40 years, from 1926 to 1968, La Fonda was one of the famous Harvey Houses, a renowned chain of fine hotels.

Today, the La Fonda Hotel  is said to host not only travelers visiting Santa Fe, but also several ghosts. Some people believe that the Honorable Judge Slough continues to walk its hallways. However, more often reported, is the ghost of the distraught salesman who jumped into the well after losing all of his company’s money. The hotel’s dining room, called the La Plazuela, is situated directly over the old well and both guests and staff alike have reported the sight of a ghostly figure that walks to the center of the room, then seemingly jumps into the floor and disappears.