Fort Brown (1846-1944) - Established in 1846 as the first U.S. military post in Texas. Brigadier General Zachary Taylor arrived at the site in March, 1846 to occupy the territory that was claimed by both the United States and Mexico. The initial fort was an earthen fortification constructed by Taylor's troops and called "Fort Texas." When Taylor marched a large portion of his troops to Point Isabel (now Port Isabel), he left a garrison, under the command of Major Jacob Brown, to guard the fort. Taking advantage of the situation, the Mexican Army attacked the fort, sending part of their force to engage the main portion of Taylor's force as it returned from Point Isabel.

Taylor defeated the Mexicans at the Battle of Palo Alto on May 8, 1856 and at Resaca de la Palma the next day. These engagements were the first battles of the Mexican-American War, and the most important battles of the war fought on U.S. soil. Following these defeats, the Mexican Army fled back across the Rio Grande River. The fort was renamed Fort Brown in honor of Major Brown, who was killed in its defense.

Federal troops left the fort with Texas' secession from the Union during the Civil War. It was then occupied by Confederate Colonel John "Rip" Ford and his troops until 1863 when they were finally driven out by Union forces under General Nathaniel P. Banks, who then camped in tents erected at the fort site. However, in 1864, Confederate forces under General J. S. Slaughter and Colonel Ford reoccupied the area and held the . post until the end of the war.

In 1867, a permanent fort was constructed under the supervision of Capt. William A. Wainwright. The post remained active into the 21st century, often manned by regiments of Buffalo Soldiers.

On August 13 and 14, 1906 the Brownsville Raid occurred, in which several unknown individuals raided the city, indiscriminately shooting up the town, killing one man and wounding another.

The townspeople quickly blamed the black soldiers stationed at nearby Fort Brown and, when the Army investigated the matter, they came to the same conclusion. The Secretary of War, William H. Taft, then discharged all 168 black soldiers "without honor." It would be another sixty years before a second investigation was held and the black soldiers had their honor restored. However, by then, only two of the original 168 men were still alive. Historians today believe that the real culprits who shot up the town, used the same caliber ammunition as the soldiers, in order to frame them.