These four men decided that five miles was too far for the children to walk to Sunday School, and they erected a brush harbor between two Sweet Gum trees and two cedar posts. This arrangement was situated about 2 1/2 miles northwest of where the present church structure is now. The area was called the Quarter at that time, and they had their first service under the brush harbor on the first Sunday in May, 1885.
The first deacons were: Deacon Julius Brevard, Deacon George Butler and Deacon Abram Brunson, Chairman. Brother Sony Brown Belton was elected Sunday School Superintendent. Sister Effie Brunson gave the church the name of Mount Olive Baptist Church. Rev. Gaither was elected as the first pastor. Rev. R. D. Gaither served as pastor until he moved across the river making it too far of a distance for him to travel causing him to give up the pastor ship.
During that time, the third church burned one Christmas, and five months later they moved into the new church building on the first Sunday in May. When Rev. Adamson had an accident and died, the church elected, Rev. J. L. Perkins as pastor and he served for 4 years.
Rev. P.M. McCain replaced Rev. Perkins as pastor and served for 2 years until his death. The church then elected Rev. David Jones. While Rev. Jones was pastor, the church decided to build a new and more modern building. The present sanctuary was completed in September 1974 across the road from the old church which was finally torn down after several years.