Photos for the Mac is a storage hog. It likes to read and write from its storage with abandon, no matter how much memory you have. Because a Photos library can become quite large—mine apparently pushes 500GB—you may have resorted as I did by storing your library on an external drive.
Or, if you have a high-capacity startup volume that holds your Photos library, that storage may be a Fusion drive, Apple’s now-retired technique of pairing a slow hard drive with a low-capacity, super-fast SSD.
You might be ready for the switch to SSD, which you can now do without breaking the bank.