if you quit Pd, the destructors are not properly called for externals. this is a rather serious issue reported on the sf-tracker soome years(?) ago, but a fix never made it into Pd proper.
Also, in this case I don't think this is the problem as the verbose output shows that the free function was called properly on both client and server, it is just that the server in its current state is apparently unable to free the port if it still has clients connected to it.
Ico