Acoustic detection and tracking of small (swimmer-size) targets while minimizing the incidents of false alerts can be challenging in a shallow, multipath, high-clutter harbor environment. One common problem involves track intermittency where periods or locations of high clutter or multipath interference inhibit detection and cause tracks to fail and restart. This paper describes the results of applying track segment association and related algorithms from airborne and ground radar applications [e.g., Zhang and Bar-Shalom, IEEE Trans. Aerosp. Electron. Syst. 47(3), 1899–1914 (2011)] to an active sonar network for underwater port surveillance [Edelson et al., J.Acoust. Soc. Am. 129(4), 2598 (2011), Stein et al., J. Acoust. Soc. Am. 121(5), 3084 (2007)] using real-world swimmer and synthetic target data.