Usually when this starts happening the refrigerator inlet water valve need to be replaced. It is leaking water by the diaphram when the water is supposed to be off. I have seen this probably 100 times..... almost always is the inlet water valve.
You did not provide a model number so I can not provide you with a parts link,

Thanks for choosing FixYa,

Kelly