Colter was hired to guide them to the mouth of the Big Horn River. Once again, the mountain man turned back leading the party into present day Montana, where they built Fort Raymond on the Yellowstone River, a short distance above the mouth of the Bighorn River.

In October, 1807, Lisa sent Colter out to meet with the the winter Indian camps, alerting them to the presence of the Missouri Fur Company and desire to trade. Though his exact route is uncertain, Colter traveled alone with only his rifle and pack, covering an estimated 500 miles. During the winter, with the help of Indian guides, he was thought to have crossed the Wind River Mountains, the Teton Range, and was probably the first white man to see Jackson's Hole and Yellowstone Lake.  Arriving back at Fort Raymond in the spring of 1808, he described the thermal wonders of Yellowstone to the rest of the party, and though most were skeptical of his descriptions, Yellowstone soon became known as  "Colter's Hell."

In 1808, Colter teamed up with another former member of the Lewis and Clark Expedition named John Potts and the two began to trap in the region near Three Forks, Montana. Both were wounded in a fight with Blackfoot warriors as they led a party of Crow Indians to Fort Raymond. The next year, the men were once again attacked by the Blackfoot and Potts was killed. Colter was captured and amazed when the Indians allowed set him free. However, before doing so, they stripped him naked and took all his possessions. Telling Colter to run, the mountain man quickly realized he was the object of a "human hunt." A very swift runner, Colter eluded most of the group but one man was gaining on him. Turning and facing the Indian, Colter killed him with his own spear, took his blanket, and by hiding in the river under a pile of logs, was able to escape. For the next eleven days, he walked 200 miles back to Fort Raymond with only the blanket for warmth and survived on bark and roots to eat. When he stumbled into the stockade he was almost dead. However, the brave mountain man was nursed back to health.