The regiment was organized on February 10, 1863, at
New Orleans as the 4th Regiment Louisiana Native Guard Infantry. The first 3 Native Guard Union regiments of Black troops had been formed in late 1862 after Union forces under General
Benjamin F. Butler captured the city. New Orleans had a sizeable population of
free people of color, and free Black men had served in the Louisiana militia since the French colonial period. When the Native Guard regiments were first organized, some of the officers were Black men, but General
Nathaniel P. Banks, Butler's successor as commander of Union forces in New Orleans, sought to remove them from their positions. He wrote in February as the 4th Regiment was being organized that the
1st,
2nd, and
3rd Native Guard regiments had "negro company officers, whom I am replacing, as vacancies occur, by white ones, being entirely satisfied that the appointment of colored officers is detrimental to the service." and that "The officers of the Fourth Regiment will be white men." Prejudices such as those held by Banks prevented Black soldiers from being commissioned as officers in the US Army until after the Civil War, all of the officers of the 4th Native Guard were white veterans from other Union regiments. The 4th Native Guard was initially assigned to guard the defenses of New Orleans and
Baton Rouge. The Regiment then took part in the
siege of Port Hudson beginning in May 1863, as Union forces sought to dislodge the Confederates from their strongpoints along the
Mississippi River. The Confederates at Port Hudson surrendered on July 9, this victory along with the capture of
Vicksburg, Mississippi a few days earlier, secured control of the entire Mississippi river for the Union. ==Garrison duty==