The Colony Museum in Atascadero likely won't reopen until early 2020 after safety and health threats forced it to close in October.
"You'll see some pretty hard core people out there," Atascadero Historical Society Vice Pres. Thomas Lewis said. "A lot of drug, alcohol, mental issues."
Once a haven for history, the Colony Museum is now under siege.
"We had a drug deal went down bad out here in the parking lot and when the police car pulled in, the drug dealer ran in here to hide in the house until the police left," Lewis said. "It was scary for the docent in the house."
The 70-year-old docent was unharmed but due to safety concerns, the museum was closed.
But that incident is not the only trouble at the museum. The museum's century old fountain was smashed in what Lewis believes was an attempt to roll the bowl top down the hill into the creek.
The back patio is a popular place for using drugs.
On Tuesday, a homeless woman could be seen changing her baby's diaper in the back seat of her car parked in the museum parking lot.
"That's a regular occurrence," Lewis said about the woman.
"We get constant reports of the transient activity down there," Atascadero Police Lt. Jason Carr said.
Lewis doesn't fault local law enforcement for the ongoing issues at and around the museum. Lewis believes city leaders are trying to help but says state legislation limiting arrests for drug use allows the problem to persist.
"Now when we run across some one in possession of narcotics, it's simply a citation when before we would've taken them down to county jail and had them removed," Carr said.
To help curb crime in the area, the city public works department removed shrubbery and trees behind the museum where people slept and concealed their drug use.
Police plan to install lights and cameras in the area to help deter crime and keep closer watch.
But Lewis believes support from the community is necessary to make a change.
"The city's made a huge effort but Atascadero is a small community in general, with 30,000 people," Lewis said. "How much effort can they put in here without giving up responsibilities somewhere else. they can be in all places at once."