What's the problem with deporting illegal immigrants?

Genuinely asking 🙈 on the one hand, I feel like if you're caught in any country illegally then you have to leave. On the other, I wonder if I'm naive to issues with the process, implementation, and execution.

Edit: I really appreciate the varied, thoughtful answers everyone has given — thank you!