Getting rid of the decrepit building, a former country club built in the 1980s, has mitigating factors and will require special handling, officials said.