I don't know if it will be a problem, but one possible issue with that approach is that there may be two copies of a file in the second directory without any in the main "reference" directory. You'd then delete both copies.
However, you could avoid that by first doing a duplicates search in the second directory only, and deleting all but one of every file there (which the duplicate finder will do easily enough). Now you know there are no duplicates in the second directory by itself.
Then you can do the duplicates search across the two directories and, as MrC says, ungroup the results and delete anything from the second directory that was found to be a duplicate. (You now know those have to be duplicates of things in the reference directory because you just ensured there are no duplicates in the second directory by itself.)
Then, anything which is left in the second directory must be a new file that needs to be moved to the "reference" directory.