It sounds like you’re looking in a good direction with the idea that there could be a networking issue. Have you checked any of your network interface statistics to see if there are any errors? Do you have flow control enabled on all of your PC NICs? What type of switch(es) are you using? Are you using FastEthernet, Gigabit or combination of both?
Another thing to consider playing with is the NIC interrupt moderation settings. You could try disabling to see if it helps.
