If you want to achieve this, you'll need to use a pry bar to put tension on the compressor and hold it in place while you tighten the pivot bolts. If your compressor has an idler wheel, tighten the belt until it's snug, then tighten the lock nut.
Let's Get It Fixed!
HomeX virtual experts will assess your issue (for free!) and can resolve simple problems.