Your wireless router model have both wired and wireless connection. when one computer connected to that router through LAN (Wired) and the other computer connected through WiFi (Wireless) simultaneously,then occasionally, both of you can be disconnected for a few seconds because of the router's DNS assigned and released when certain computer attempt to connect or disconnect the router. but its normal in router when using both wired and wireless simultaneously.

It could be your router is becoming intermittent, but also your ISP connection could be dropping and not resetting properly. I have this issue occasionally and it is the ISP generally dropping my connection. Only way to test is to try a known good router and see if it corrects the problem. With data speeds increasing it is getting more difficult for the ISP to provide reliable service especially over ADSL. I went 5 years without EVER resetting my ISP connection, then started having increasing difficulties with my connection in the past 3-4 years due to higher data rates being introduced.