It depends what you want 4k for.
If you're tired of seeing giant pixels on your computer, while your phone has had much finer details for years, then it's not too small.
OTOH, If you want to run things without any DPI scaling and want more space, not more detail, then it is too small.
I run two 24" 4k screens at 200% scaling (so it is like two 24" 1080p screens but with 4x the pixels/detail), and that works great.
As Jon says, DPI scaling in Windows and most apps is pretty good now. There are some annoying bugs in the OS still (like Window borders have weird gaps in them if you look closely, and the odd icon in some UIs is tiny) but they are mostly minor and cosmetic.
Also as Jon says, mixed DPI in Windows is still something to be avoided. The OS still has a ton of bugs if you have two monitors set to different scaling factors. Avoid that like the plague.
I went with 24" 4K as I wanted an integer scaling factor. 200% scaling means things designed for standard pixel sizes can be "pixel doubled" (really pixel quadrupled) so there is no blurred scaling on them. They look exactly as they would have looked on a 1080p screen, other than the minor issue of ClearType antialiasing looking a bit weird (or more weird).
If I had a 27" 4K screen I'd probably run that at 200% as well, and just spread things out over a larger space. But once you get above 27" or 28" or so, 200% scaling is probably too much (assuming the screen is at a normal desk/chair distance) and you'd maybe use 150% or 175%, at which point apps designed for 100% are going to get blurry, which is horrible.
If you do any gaming, and want 60fps rather than 30fps, you can forget 4k native. But you can run games at 1080p or 1440p and they scale well to the 4k screen. The issues with things getting blurry don't seem to matter when scaling 1440p to 4k for a game, although a native resolution is always a bit nicer (and allows you to run things windowed, too). The next-gen cards might finally do 4k gaming at a good framerate. Right now the 1080ti / Titans barely does it and can't maintain 60fps in many new titles.
Remember that HDR is coming out and refresh rates keep going up. A new VESA standard for FreeSync is coming out (although who knows if NVidia will support it, or continue to make people pay extra for GSync). There are a couple of 4K + HDR + G/FreeSync monitors on the market now but they are very expensive. Whatever you buy, I'd say don't spend too much as you might want to replace it in a couple of years.
Personally, I have a pair of pretty cheap ViewSonic 24" VX2475. They cost about as much as a good 1080p screen but are 4K, and they're similar to IPS (but using an alternative; I forget the name).
Going back to my first paragraph: The main question to ask is what you want 4K for. And I have to say that when I went back to 1080p for a month recently, I did not really miss 4K. The main difference was that text was chunkier when reading the web, and the borders of things were thicker than I'd become used to, but otherwise things were about the same and everything runs a bit faster at 1080p than 4K. I still like 4K but I'm also still not sure if it's worth it yet. Pretty soon there won't be any real cost to it so it will be, other than a few annoying cosmetic bugs that Microsoft don't seem to notice or want to fix because their QA isn't what it used to be.
One caveat: If you use VMware a lot, DPI scaling is a problem. VMware's DPI support is still terrible and wrongheaded. You can get by if you only use it briefly (like we do to test code on different OS), but not if you use it all day. There will be problems with other particular apps that I/we don't use and don't know about which might be a dealbreaker for you if you do use them, too. Also, if you use Photoshop, you will need to move to their rental versions as the last retail versions of Photoshop are straight-up broken under DPI scaling, unfortunately.