Most sources recommend against storing olive oil in the fridge, preferring the classic "dark, cool place." If you do refrigerate it, they advise giving it 5 minutes or so to come to room temperature before use.
Other oils are apparently more forgiving of refrigeration, or maybe just beneath the notice of those who care about such things.
