How can I remove all "name" files in all of my subdirectories on Linux

Remove all *.swp files underneath the current directory, use the find command in one of the following forms:

• find . -name \*.swp -type f -delete
  
  The -delete option means find will directly delete the matching files. This is the best match to OP's actual question.
  
  Using -type f means find will only process files.
  
  

• find . -name \*.swp -type f -exec rm -f {} \;
find . -name \*.swp -type f -exec rm -f {} +
  
  Option -exec allows find to execute an arbitrary command per file. The first variant will run the command once per file, and the second will run as few commands as possible by replacing {} with as many parameters as possible.
  
  

• find . -name \*.swp -type f -print0 | xargs -0 rm -f
  
  Piping the output to xargs is used form more complex per-file commands than is possible with -exec. The option -print0 tells find to separate matches with ASCII NULL instead of a newline, and -0 tells xargs to expect NULL-separated input. This makes the pipe construct safe for filenames containing whitespace.