The East Side School District purchased 7.993 acres on February 25, 1949 from Johnston deForest for the sum of $12,000 for the construction of a new school. Construction began in 1952 on the 23,300 square foot building which would cost $395,000. This one story brick steel frame building, located at 75 Goose Hill Road, opened on September 9, 1953 with an anticipated enrollment of 148 students. It included 11 classrooms, an auditorium/gymnasium, and a cafeteria. The dedication ceremonies were postponed as the trucking strike prevented the completion of landscaping and road work, but the building was complete and ready for classes. On April 6, 1955, 2.999 adjoining acres were donated to the school according to the deed by Priscilla deForest Williams. This purchase now made the total area 10.992 acres. This school was kindergarten through 8th grade until the high school opened in 1962. Then the 7th and 8th graders went to the high school. In 1957 there were 220 K-8 students. In January 1961, a successful first scheduled lesson was shown on closed circuit television.

1963-1965 Addition:
 
On June 20, 1963, the Board of Education voted to establish a committee to advise the board on needed school construction and report their findings by September 1st.  In December, the residents voted and approved a 17,000 square foot addition for $521,000 to East Side School. It included: 7 classrooms, another gym, new kitchen, locker areas, enlarged music, and a library. Anticipated capacity after this addition would be 300 students. Ralph Collyer was the architectural consultant.

Closing of East Side School:

The district experienced continued declining enrollments in 1977 and decided in the spring to enter into a contract with BOCES. They reviewed our school facilities during the summer and fall with regard to the possible closing of one of the three elementary schools. BOCES confirmed that enrollment at the elementary level would be lower in subsequent years. Based upon this information, the board decided to close one elementary school. Public hearings were held, presentations were made, and petitions were signed early in 1978, but on January 26th, the board officially voted 6-1 to close East Side School on June 25, 1978. Boundary lines were reorganized and East Side students were assigned to attend either Lloyd Harbor School or West Side School. The majority of the students and staff went to Lloyd Harbor School.

Uses of the building and advertisement for sale:

From 1978 to 1982 different parts of the building were rented to various organizations.

In 1982 the board began advertising the sale of this building which by this time was in deplorable condition due to deterioration and increased vandalism. A committee was formed to serve as an advisory panel and meet with the board and potential purchasers of the school in the summer of 1983. Inquiries and offers were made by developers during the summer. The possibilities included: apartments for senior citizens, condominiums for seniors, long term rental lease by a cancer lab, and finally a builder suggested to demolish the building and develop 4 to 5 house lots.

Voter approval was required for the final decision. After considering all the offers the board decided not to sell the building but lease instead in order to retain the school for possible future use.

From 1983 to 1993 parts of the building were again leased to various organizations which ended when Goosehill Primary School opened in 1993.

In 1985 a residential vote was made to call the entire building “The Community Center”. It officially opened on November 19th. The Cold Spring Harbor Library leased some space there from 1985 to 2006. In December 1985, district office moved from the 3rd East Side School to this location where it remains in 2010.