On April 18, 1887, Harlem Hospital was opened. The physical plant consisted of a leased, three-story wooden building, located at the foot of East 120th Street and the East River in New York City, with 54 beds. From the beginning, Harlem Hospital's responsibility was to provide medical care for the poor, specifically to those living in the rapidly growing districts north of Central Park. Although this area had not yet felt the impact of the first of many waves of African-American immigrants from the South, the hospital's 54 beds soon were grossly inadequate to meet the needs of the rapidly growing community. The dispensary was moved to a wooden building on a vacant lot near Harlem Hospital. The wooden building at one time had been used as the Out-Patient building for Gouverneur Hospital.

In 1900 land was acquired by the City of New York on the East Side of Lenox Avenue from 136th Street to 137th Street for a hospital of 100 beds. In 1903 additional land was purchased for the future erection of a Nurses' Home, a wing to the hospital, a power house, and stables. The new hospital, with a bed capacity of 150 and located on the East Side of Lenox Avenue, was opened on April 13, 1907.