I use this to filter images with tag "Wallpaper", If there are hundreds of images, it took a very long time.
Select "Wallpaper" FILTER FILTERFLAGS=hidenomatch DESELECT
There a no details of your filter for us to see what it does, but if it involves opening the files and reading data from them then it will always be much slower than something which filters based on just the file name (or other basic data that comes from the directory listing, like size and modified date).
Matching that kind of tag will require reading extra data for each file, since it isn't part of the directory listing. It's inherently slower as there is a lot more disk access, seeking, contention, latency, etc. involved in reading in the data.
Is there any way to accelerate this, such as index a local database of tags, and the later taged files will add to it automatically.
You could use a Windows Search query with indexing instead, if Windows Search supports tags in the file formats you're using.