Welcome to the forum!
Or to remove it completely, select it (by clicking on it with the Selection tool) and then Delete.
Regarding
.....i want to add detail and clean up the map and.....
You can add detail, but you won't be able to edit the PNG almost at all, with Inkscape. You might be able to change colors (over the whole PNG at once), using the Layer Blend Modes, or maybe some filters.
But mostly it can't be edited with Inkscape. However, you could edit in a raster editor, and re-import back to Inkscape.