Remove the files associated to the submodule
rm -rf .git/modules/path/to/sub-module-1
Remove any references to submodule in config and .gitmodules manually.
vim .git/config
vim .gitmodules
delete folder sub-module-1 folder and stage this change first
rm -rf path/to/sub-module-1
git add path/to/sub-module-1
then, you will add sub-module-1 again successfully.
git submodule add url path/to/sub-module-1