I'm still unclear what you initially did when you brought him home so I'd suggest you start by trying a reintroduction to the house and a rebonding experience with you. Put him back in his room for at least 2 wks and follow the instructions in the "acclimating your new savannah" attachment in my previous post. The handfeeding is key, I think, to establishing trust. In his room he should have a crate or a large box that he can go into when he doesn't want to interact with you.
Patience on your part is key. Very slow baby steps starting at the beginning again is, I think, key to your success. He's only been with you for 4 weeks which isn't a lot of time, particularly for an abused cat. He has lost his natural defense mechanisms (teeth/claws) and it sounds to me as though he has very good reason not to trust people. If he is a higher generation savannah, it contributes to the time it takes to establish trust since they are known for liking change, even if it's good change.
We have a member here who worked with her cat an entire year before she had a big breakthrough. I have an F2 who wasn't well socialized when I got him and it took me 2 yrs to fully gain his trust. Patience, patience, patience.
You are doing a wonderful thing by giving this cat a chance at a normal life. It's easy to get discouraged but we are here to help you, hold your virtual hand, talk you through the challenges. This place is full of wonderful people who have been there and done what you are attempting so keep posting and benefit from their knowledge and experience.