If you go to Xcode > File > Swift Packages, you can see options to add a new Swift package, update them, reset caches, and resolve package versions. However, you do not see an option to remove a particular Swift package.
To remove a package, click on the project on the left and then click on your app under Project on the right. After this, click on Swift Packages on the extreme right. Now, you should see all your Swift packages. You can click on a specific package and hit the - button at the bottom to remove it.
