First, there is a new version of Yosemite, 10.10.2. Don't think it will help your situation much, but it's still good to stay up to date.
Secondly, I purchased a late 2012 Mac Mini with a 2.3 GHz Intel Core i7 processor and 4 gig of RAM in July 2013. It also came with a (slow) 5400 rpm 1 TB hard drive. I did use it for a few months with that drive (was using the latest version of Mavericks, OS 10.9.5), but it was just too slow (I also did not need that much space. In fact, for my processing, it is WAY too much. And, I keep my machines "lean and clean"). I had been planning on replacing that drive with a 256 gig Samsung 840 Pro SSD, and that is exactly what I did in November 2013. What a difference! The machine is definitely WAY faster. I also placed the 1 TB drive in a nice, slim external case, and I use it mainly for storing movies and TV series. I also do a backup of my MacBook Air to another partition on that drive whenever we are going to travel, as I take that external drive with me. In fact, I am going to do just that on Sunday, as my wife and I are going to Hawaii on Monday for 7 days. That external drive is so, so convenient.
So, an SSD will make a WORLD of difference. For my model, it was not difficult at all to take out the 1 TB drive and install the SSD. If you tell me your exact model, and are willing to do such a project, I can provide the link that will show you how to do it. More Ram CAN help, but that really depends on your processing needs. For me, 4 Gig is enough (have the same amount in my MacBook Air). I can upgrade the Ram on my Mac Mini myself (rather easy to do), and I suspect you can do the same. But, if I were you, the first thing I would do would be to install an SSD