Bank Won't Include Value of Outbuildings for Financing Approval?

RBC got an appraisal done on the acreage we are wanting to buy. But they won't finance the value of the outbuildings (approximately $50,000 value). Do all banks do this or who should we talk to to get the financing needed? I think Affinity will, any others?