Food is denser than air and can retain "coldness" longer once cooled down. Most of the time the temperature leak is through the joints. In an empty fridge, all you have is cold air which is the main heat transmitter.
Let's Get It Fixed!
HomeX virtual experts will assess your issue (for free!) and can resolve simple problems.