The cornerstone of our church was laid in 1854. The mission was cared for by the Madison Parish until 1881, when it was attached to Morristown. In 1882 Whippany and Morris Plains were detached from Morristown and established as a separate parish. Although the new pastor initially set up headquarters in Whippany, he had moved his residence to the growing community at Morris Plains by 1885. In 1909 Bishop O'Connor separated Whippany from Morris Plains and established it as a separate parish.

At the centennial of the first church, a new church and school were constructed on Whippany Road in 1954. The church was renovated in 1982. The Sisters of St. Joseph of Newark staffed the school until its withdrawal in 1982.

Pastors:

1. Rev. Cornelius C. Clifford (1909-1939)
   1. https://www.nytimes.com/1938/12/05/archives/rev-c-c-clifford-priest-in-jersey-his-tiny-church-at-whippany-with.html
   2. https://centuryarchives.org/caba/bio.php?PersonID=588
   3. https://thericatholic.com/history.html
2. Rev. Francis P. Kowalczyk (born in Passaic, NJ, Sept. 24, 1896; son of Peter Kowalczyk and Mary Patoczak. Attended St. Joseph's parochial school, Passaic. Studied at St. Mary's High School, Orchard Lake, Mich., from which he graduated. Studied for two years at Seton Hall College, South Orange, where he completed a philosophy course. Studied theology at Immaculate Conception Seminary, South Orange. Ordained to holy priesthood at St. Patrick's Cathedral, Newark, June 10, 1922. His first assignment was as curate at Our Lady of Rosary Church, Passaic, in which post he remained for seventeen years. On Jan. 1, 1939, appointed pastor of Our Lady of Mercy Church, Whippany. In 1942 transferred as pastor of Our Lady of Rosary Church, Passaic, N. J.; died 3/8/1968)
3. Rev. Michael J. Zarrillo (1942-1965)
4. Rev. Thomas J. Boyle (1965-1972)
5. Rev. David W. McCarthy (1972-1976)
6. Pastoral Team - Rev. James Dolan coord. (1976-1978)
7. Pastoral Team - Rev. James Durkan coord. (1978-1979)
8. Pastoral Team - Rev. Patrick Browne coord. (1979-1980)
9. Rev. John A. DeMattia (1980-2006)
10. Rev. Sean McDonnell (2006-2017)
11. Rev. Roberto Amador (2017-2023)
12. Rev. Michal Rybinski (2023-2024 Administrator, 2024- Pastor)