You can only remove packages which are not required by another package.
Note: This only removes the reference to the package in the manifest. The package itself, including its contents, remains intact, even for local packages.
When you remove a package from a Project, any Editor or run-time functionality that it implemented is no longer available in that Project.
To remove an installed package:
Open the Package Manager window.
If the package scope is set to Built-in packages, select either In Project or All packages from the package scope drop-down menu.
Select the package you want to remove from the list of packages. The details view now displays that package’s information.
Click the Remove button.
When the progress bar finishes, the package disappears from the list.
If you want to restore a removed package, follow the instructions for installing a package.