The 4-0 Roots Hall loss leaves the club 20th in the National League as the Shrimpers battle to avoid a third relegation in three seasons.
The former Hull City manager, 62, took over in April but couldn’t stop their slide out of the EFL.
The Essex club have also announced assistant Craig Fagan has left.
Southend say their search for a new manager begins tonight - and didn’t confirmed who would take the job in the interim.
Brown becomes the third National League manager to depart so far the campaign following Danny Searle’s exit from Aldershot Town and Barnet’s decision to sack Harry Kewell.
Southend released a brief statement less than an hour after their demoralising defeat to Chesterfield.
It read: "Southend United can confirm that Phil Brown and Craig Fagan have been relieved of their duties effective immediately.
"The search for a new manager starts today.
"The club will be making no further comment at this time."