Borough fell behind in the very first minute as David Pratt struck for the visitors. The home side hit back through Jamie Taylor but then Andy Sandell restored the Bluebirds lead until Taylor netted again before half-time.
Eastbourne took the lead for the first time in the match shortly after the break as Charles Harris struck and then Sergio Torres wrapped up the points late on.
The win sees Howell's side move up to ninth, and he said afterwards: "It was an entertaining game, definitely for the neutrals, but a bit hairy for me and the staff! Both teams played well, there was lots of goals and it was a good advert for the league.
"It was a topsy-turvy game. Both teams wanted to attack and it left the defences exposed.
"We've been good recently in keeping clean sheets but our attacking play was good today and I'm really pleased they scored four goals at home.
Howell joined Eastbourne in the summer and was relieved to get his first home win today. "I was getting a bit worried about it!" he said.
"We're a work in progress and we're a long way from where we need to be but we're making steps in the right direction."