if you enter the "install from zip file"-section, the upcoming window is not kind of a source. It's more a file browser where you are able to browse to the folder you downloaded/stored the zip-file to. Those aren't sources like the ones you added in Kodi. Those are just folders that exist on your machine and therefore they can't be deleted by opening the context menu.
You might be able to delete specific folders on your system using Kodis file explorer. Therefore you need to add one of the parent folders as a source in Kodis file explorer. You have to enable "Allow file renaming and deletion" in settings->media settings->general. Then move over to Kodis file explorer, navigate to one of the subfolders from the sources-folder you added before and bring up the context menu and delete that folder. After you did all your deletions you just have to remove the source again.
Note: removing/deleting a real source inside Kodi doesn't mean, that the folder, where the source points to, is completely deleted from the HDD. It's just not visible anymore inside Kodi.